Intuition

When an application knows the model a priori, its estimates the parameter that most accurately defines the data. When choosing between different models, the higher the model order, the more accurately the estimated parameters fit the data. Thus, accuracy as a sole measure of fit quality is rendered ineffective when comparing best fits from different models; fit accuracy must be combined with other fit characteristics in order to choose the correct model. Model selection is sometimes seen as a compromise between accuracy of the fit corresponding to the model and the stability of the model to small perturbations in the data. The accuracy of a fit may be described by the residual sum of squares or the maximum likelihood, and the stability of a model can be measured by the covariance matrix of the estimated parameters [5]. This intuition also shows that simpler models are stabler, making model selection a compromise between accuracy of the fit and simplicity of the model. Another intuition views a good model as one that not only fits well to the current data, but also to data of the same object collected later by the same sensor [12]. Finally, the third intuition is based on the pattern made by residuals from a fit corresponding to a given model [2,9].