Regluarization
Notes and Ideas:
Regluarization performs Feature Selection by shrinking the contribution of features.Addjusted cost function
M(w): Model Error
R(w): Function of Estimated Parameter(s)
: regularization strength parameter, adds a penalty proportional to the size of the estimated model parameters, or a function of the parameter Link to original
For L1-regularization (Lasso Regression), this is accomplished by driving some coefficients to zero.
Reducing the number of features can prevent overfitting.