Open problems

The complexity of the correspondence problem depends on the complexity of the scene. As described in the previous sections, there are constraints and schemes that can help reduce the number of false matches, but many unsolved problems still exist in stereo matching. Some of these problems are:

• occlusion: given a feature m in the first image projected by a scene point M, the ``corresponding'' feature of m may not exist in the second image because from the second camera's viewpoint, M may be occluded.
• discontinuity: this is related to occlusion. Most matching algorithms use the continuity and/or relaxation as a global constraint to reduce false matches. This constraint works everywhere in the images except at depth discontinuities.
• regularity and repetitivity: repetitive patterns in the scene cause ambiguity in matching. Most existing matching algorithms are still not robust enough to handle this.