Regularization techniques are used to prevent statistical overfitting in a predictive model. By introducing additional information into the model, regularization algorithms can deal with multicollinearity and redundant predictors by making the model more parsimonious and accurate. These algorithms typically work by applying a penalty for complexity such as by adding the coefficients of the model into the minimization or including a roughness penalty.
Techniques and algorithms important for regularization include ridge regression (also known as Tikhonov regularization), lasso and elastic net algorithms, as well as trace plots and cross-validated mean square error. You can also apply Akaike Information Criteria (AIC) as a goodness-of-fit metric.
For more information on regularization techniques, please see Statistics Toolbox.
See also: Machine Learning