To illustrate how belief networks can be used to segment medical images, I will describe a prototype system developed by Levitt, Hedgcock et al, to automatically analyze symptoms of arthridities in hand radiographs [51]. This system was developed at the San Francisco VA Medical Center with the goal of diagnosing and tracking radiological studies involving arthridities. It demonstrates the feasibility of combining Binford's model-based image understanding approach with probabilistic inference techniques. There are three major processing activities in this system: (1) prediction of image features based on three-dimensional anatomical models of the hand (parameterized by patient population statistics), (2) extraction of local image features from regions with a high probability of pathology, and (3) accumulation of evidence from lower-level image processing and feature matching procedures to support or deny hypotheses regarding anatomical structures (the likelihood of each hypothesis is computed using a Bayesian calculus).
The authors claim that their system accurately segments ``normal'' and ``somewhat degenerated hand anatomy.'' Furthermore, the system has the ability to ``soft fail,'' that is, to recognize when it has inaccurately segmented an image set. This feature allows clinicians to take into account the accuracy of the automated results when making a diagnosis. The authors are currently investigating the quantification of rheumatoid arthritis, osteoarthritis, and other degenerative joint diseases using their system.
The Bayesian reasoning technique collects evidence to support
or deny hypotheses about the image being analyzed. In this
case, evidence refers to features such as edges, curves, vertices, and
regions that are extracted by low-level image processing operators.
Also, properties of features such as size, intensity variations,
straightness, parallelism, convexity, and closure are also measured.
The prior probability distributions of various features are derived
from the sensor-anatomy physics and are combined with the estimated
viewing orientation to generate the predicted image values and
contrasts (see Figure 
).
Hypotheses are generated by matching the evidence extracted from the images with geometric models of the hand. These three-dimensional geometric models were built using published population statistics on the lengths, widths, and relative sizes of the phalanges (bones inside the digits) and metacarpals (bones inside the palm). The generated hypotheses provide possible interpretations about the size, location, and orientations of these anatomical surfaces and boundaries. The process is repeated as newly generated hypotheses are used to update the generic three-dimensional model, which is then used to predict more features.
The system's control loop (shown in Figure )
chooses the processing actions that will rapidly converge toward a
highly probable interpretation of the image anatomy (that is, a greedy
algorithm based on highest expected probability). As each hypothesis
is generated, a Bayesian network is created and the collected evidence
is used to update the probabilities in this network. The accumulation
and computation of the network probabilities is performed by an
external software package called Quanta, which is a commercial
implementation of Schacter's belief network
algorithm [71].
Figure: Top-level control loop of Levitt's hand radiograph analyzer.
Because of the concentric layering of bone and soft tissues in the
phalanges and metacarpals, the geometric model of the hand was built
using non-deformable straight homogeneous generalized cylinders. At
the top level, the hand is modeled as a collection of digits,
metacarpal joints, and the palm (see Figure (b)). The
digits, in turn, are modeled as collections of phalanges and joints
(see Figure (b)).
Figure: The Levitt model of the hand consists of two hierarchical
levels. The first level (a) expresses the relationships between
the digits and the metacarpals. The second level (b) expresses
the relationships among the phalanges, the inter-phalangeal
joints (IPJ), and the inter-joint spaces (IJS). The arrows
indicate a consists-of relationship.
In total, there are 58 modeled anatomical parts, each with its corresponding Bayesian network. As each new piece of evidence is obtained, a new node is attached to the appropriate networks and the belief values are updated. For example, when a phalanx segmentation operation is performed, a new evidence node is created and is given a probability based on the prior distribution of P(phalanx | segmentation evidence value). By propagating the probability of the evidence node through the Bayesian network, the probability of each active hypothesis can be recomputed.
To summarize, the geometric and statistical model is used to predict surfaces and boundaries in the two-dimensional image projections. The initial model is determined from prior statistical distributions as well as from initial features extracted by low-level algorithms. The predicted structures are used to direct the search for more features on a finer scale. As new features are extracted and more surfaces and boundaries are matched, the probability distributions of each hypothesis are evaluated using a Bayesian calculus. This cycle is repeated until a hypothesis with a high probability is generated, or until the system determines that no hypothesis can be confirmed.
Levitt's system cannot segment phalanges that are subluxed (a common condition in advanced cases of arthridites whereby the soft tissues separating the articulating bones have loosened or degenerated). However, because the system evaluates the accuracy of its segmentation process, it will reject anomalies such as subluxed joints, rather than provide incorrect measurements. One of the signficant contributions of this Bayesian model-based approach is that it can evaluate hypotheses based on ambiguous, incomplete, or obscured evidence. Furthermore, the Bayesian method combines prior probability distributions from population statistics with evidence from image processing operations using a mathematically sound decision theoretic approach. Unfortunately, the authors have deferred, for various reasons, any formal clinical evaluations of the system's performance, so the validity of their approach has yet to be established.