Verification Performance and Discussion

The goals of verification are:

1. no true hypotheses are rejected, and
2. among false hypotheses, only low level (e.g. SURFACEs), symmetric or ambiguous hypotheses are accepted.

However, as tolerances are needed to allow for segmentation variations, position parameter misestimation, and obscured surface reconstruction, some invalid verifications are expected. Some invalid SURFACEs are verified because of variability in surface shape matching and having no other constraints on their identity at this point. The effect of these hypotheses is reduced performance rates and increased chances of invocation of higher level false objects. However, verified higher false hypotheses are not likely to occur as the surfaces must then meet grouping, relative orientation and location constraints in hypothesis construction, and the verification constraints discussed in this chapter.

Table 10.1 summarizes the causes for rejection of SURFACE hypotheses, and Table 10.2 summarizes the causes for rejection of ASSEMBLY hypotheses. The tables record the rejection criterion as given here, except for those designated by "$N$", which means rejection by a modeled numerical constraint, by "$H$", which means failure to establish a reference frame (Chapter 9), or by "$A$" which means all slots that should have been filled were not.

Some rejected curved SURFACE hypotheses had the correct identity but an inconsistent reference frame. Some false ASSEMBLY hypotheses were rejected in hypothesis construction because no consistent reference frame could be found for them. These hypotheses are included in the analysis of rejected hypotheses given below.

Table 10.1: SURFACE Hypothesis Rejection Summary

SURFACE	IMAGE REGIONS	REJECTION RULE	INSTANCES
uedgeb	19,22	$N$	2
lsidea	19,22	$N$	1
lsideb	19,22	$N$	1
robbodyside	9	$N$	4
robbodyside	8	$S_2$	2
robshould1	12	$S_2$	1
robshould2	12	$S_2$	1
robshoulds	27	$S_2$	2
tcaninf	9	$S_2$	2

Table 10.2: ASSEMBLY Hypothesis Rejection Summary

ASSEMBLY	IMAGE REGIONS	REJECTION RULE	INSTANCES	NOTE
lowerarm	12,18,31	$H$	30
lowerarm	17,19,22,25,32	$A$	1
lowerarm	17,19,22,25,32	$H$	1
upperarm	17,19,22,25,32	$H$	6
armasm	12,17,18,19,22,25,31,32	$R_3$	2
robshldbd	16,26	$H$	3
robshldsobj	29	$H$	1	+1
robbody	all appt.	$H$	3	+1
robot	all appt.	$H$	5

+1	valid hypothesis rejection because of geometric reasoning error

Table 10.3: Incorrectly Verified Hypotheses Analyzed

MODEL USED	TRUE MODEL	IMAGE REGIONS	NOTE
uside	uside	19,22	+3
uends	uends	25	+2
lsidea	lsideb	12	+1
lsideb	lsideb	12	+2
ledgea	ledgea	18	+2
ledgeb	ledgea	18	+1
lendb	lendb	25	+2
robbodyside	robbodyside	8	+2
robshould1	robshould2	16	+1
robshould2	robshould2	16	+2
lowerarm	lowerarm	12,18,31	+2
upperarm	upperarm	17,19,22,25,32	+2
robbody	robbody	8	+2
trashcan	trashcan	9,28,38	+2

+1	true model similar to invoked model
+2	symmetric model gives match with another reference frame
+3	error because substantially obscured

Table 10.3 lists and analyzes all remaining verified hypotheses that were not "correct". The most common causes of incorrectly verified hypotheses were symmetric models, leading to multiple reference frames, and nearly identical models. The incorrect models normally were not used in larger ASSEMBLYs, because of reference frame inconsistencies.

These results show that verification worked well. Two true ASSEMBLY hypotheses were rejected because of deficiencies in geometric reasoning. All verified false hypotheses were reasonable, usually arising from either a similar or symmetric object model. Most rejected SURFACE hypotheses failed the value constraint (usually surface area - see Appendix A). Curved SURFACEs were rejected when their curvature axis was inconsistent with other orientation estimates. Most ASSEMBLYs were rejected because no consistent reference frame could be found. (Many of these hypotheses arose because hypothesis construction has a combinatorial aspect during initial hypothesis construction.)