The choice of the transition kernel q remains a difficult problem, and in practice, this must be tuned to the particular application. Amongst other possible choices, we may cite the Langevin diffusion [12, ], reparameterization methods [5], adaptive direction sampling [6]. Note that the famous Gibbs sampler [4] can also be seen as a Metropolis-Hastings algorithm, where the proposal transition kernel is the conditional distribution and the acceptance probability is always equal to 1. It is also possible to mix a Gibbs sampler and a Metropolis-Hastings algorithm.
As an illustration, we have represented on figure 3 the paths of four Markov chains with the same stationary distribution but generated by four different algorithms. It can be clearly seen that the mixing properties vary strongly with the algorithm used.
Figure 3: Markov chains generated by four different algorithms: an independant sampler, a random walk, a Langevin diffusion and a modified version of the Metropolis-Hastings algorithm (the multiple Metropolis-Hastings algorithm). Although the four chains admit the same stationary distribution, their behaviours differ strongly. Note the poor mixing of the independent sampler.
In practice, monitoring the convergence of the chain is essential, since the Monte Carlo integration is only valid if the chain has converged towards the stationary distribution. Very often, the chain shows out a stable behaviour, but this can be an indication of poor mixing rather than convergence. The convergence of the Monte Carlo integral needs also to be studied, this can be done for example by computing the integral range of equation 12 using batch means [5]. Some hints about convergence monitoring can be found in [5].
To conclude we can give some references of applications of MCMC methods in computer vision: estimation of prior parameters and image segmentation [2], feature correspondence [1], building detection [3] and stereovision [13].