next up previous
Next: Related Work Up: Hypothesis Verification Previous: Numerical Constraint Evaluation

Verification Performance and Discussion

The goals of verification are:

  1. no true hypotheses are rejected, and
  2. among false hypotheses, only low level (e.g. SURFACEs), symmetric or ambiguous hypotheses are accepted.

However, as tolerances are needed to allow for segmentation variations, position parameter misestimation, and obscured surface reconstruction, some invalid verifications are expected. Some invalid SURFACEs are verified because of variability in surface shape matching and having no other constraints on their identity at this point. The effect of these hypotheses is reduced performance rates and increased chances of invocation of higher level false objects. However, verified higher false hypotheses are not likely to occur as the surfaces must then meet grouping, relative orientation and location constraints in hypothesis construction, and the verification constraints discussed in this chapter.

Table 10.1 summarizes the causes for rejection of SURFACE hypotheses, and Table 10.2 summarizes the causes for rejection of ASSEMBLY hypotheses. The tables record the rejection criterion as given here, except for those designated by "$N$", which means rejection by a modeled numerical constraint, by "$H$", which means failure to establish a reference frame (Chapter 9), or by "$A$" which means all slots that should have been filled were not.

Some rejected curved SURFACE hypotheses had the correct identity but an inconsistent reference frame. Some false ASSEMBLY hypotheses were rejected in hypothesis construction because no consistent reference frame could be found for them. These hypotheses are included in the analysis of rejected hypotheses given below.

Table 10.1: SURFACE Hypothesis Rejection Summary
SURFACE IMAGE REGIONS REJECTION RULE INSTANCES
uedgeb 19,22 $N$ 2
lsidea 19,22 $N$ 1
lsideb 19,22 $N$ 1
robbodyside 9 $N$ 4
robbodyside 8 $S_2$ 2
robshould1 12 $S_2$ 1
robshould2 12 $S_2$ 1
robshoulds 27 $S_2$ 2
tcaninf 9 $S_2$ 2


Table 10.2: ASSEMBLY Hypothesis Rejection Summary
ASSEMBLY IMAGE REGIONS REJECTION RULE INSTANCES NOTE
lowerarm 12,18,31 $H$ 30
lowerarm 17,19,22,25,32 $A$ 1
lowerarm 17,19,22,25,32 $H$ 1
upperarm 17,19,22,25,32 $H$ 6
armasm 12,17,18,19,22,25,31,32 $R_3$ 2
robshldbd 16,26 $H$ 3
robshldsobj 29 $H$ 1 +1
robbody all appt. $H$ 3 +1
robot all appt. $H$ 5
+1 valid hypothesis rejection because of geometric reasoning error


Table 10.3: Incorrectly Verified Hypotheses Analyzed
MODEL USED TRUE MODEL IMAGE REGIONS NOTE
uside uside 19,22 +3
uends uends 25 +2
lsidea lsideb 12 +1
lsideb lsideb 12 +2
ledgea ledgea 18 +2
ledgeb ledgea 18 +1
lendb lendb 25 +2
robbodyside robbodyside 8 +2
robshould1 robshould2 16 +1
robshould2 robshould2 16 +2
lowerarm lowerarm 12,18,31 +2
upperarm upperarm 17,19,22,25,32 +2
robbody robbody 8 +2
trashcan trashcan 9,28,38 +2
+1 true model similar to invoked model
+2 symmetric model gives match with another reference frame
+3 error because substantially obscured

Table 10.3 lists and analyzes all remaining verified hypotheses that were not "correct". The most common causes of incorrectly verified hypotheses were symmetric models, leading to multiple reference frames, and nearly identical models. The incorrect models normally were not used in larger ASSEMBLYs, because of reference frame inconsistencies.

These results show that verification worked well. Two true ASSEMBLY hypotheses were rejected because of deficiencies in geometric reasoning. All verified false hypotheses were reasonable, usually arising from either a similar or symmetric object model. Most rejected SURFACE hypotheses failed the value constraint (usually surface area - see Appendix A). Curved SURFACEs were rejected when their curvature axis was inconsistent with other orientation estimates. Most ASSEMBLYs were rejected because no consistent reference frame could be found. (Many of these hypotheses arose because hypothesis construction has a combinatorial aspect during initial hypothesis construction.)


next up previous
Next: Related Work Up: Hypothesis Verification Previous: Numerical Constraint Evaluation
Bob Fisher 2004-02-26