Important Request

[mark4flies]

Model selection is a long-standing problem in statistics and machine learning. It is still an very active research topic. There is no superior method, approach, or criterion. (That is, one that always returns the correct answer.) One of the best criteria today is Akaike's Information Criterion, corrected for small sample sizes (AICc). It is a combination of a measure of the reduction in the bias (under-fitting) and a measure of the complexity of the model (over-fitting, which leads to variance). The best model is a trade off between bias and variance that is achieved by minimizing AICc.
AICc can be computed directly from the likelihood function of the model parameters when using maximum likelihood estimation or from the sum of the squared errors when using a least squares method.