Answer :
Ridge Regression is a type of linear regression that includes an L2 regularization term, which helps prevent overfitting by adding a penalty equal to the square of the magnitude of coefficients. Here's an in-depth explanation:
Ridge Regression is particularly useful when there is multicollinearity (when independent variables are highly correlated) in the dataset. In such cases, ordinary least squares estimates can be unbiased, but they can have large variances, leading to overfitting. Ridge Regression addresses this by introducing a regularization term, specifically the L2 norm.
Explanation of Ridge Regression:
Dependent and Independent Variables: Like regular linear regression, Ridge Regression aims to model the relationship between dependent (or response) variable and one or more independent variables (or predictors).
Regularization: Regularization is a technique used to prevent overfitting by adding a penalty to large coefficients. In Ridge Regression, this takes the form of L2 regularization.
Mathematics Behind Ridge Regression:
- The ordinary least squares objective function minimizes the squared differences between the observed and predicted values.
- Ridge Regression modifies this by adding a penalty term:
[tex]\text{Cost Function} = \text{Minimize} \sum (y_i - \hat{y}_i)^2 + \lambda \sum \beta_j^2[/tex]
Here, [tex]\lambda[/tex] is a tuning parameter that controls the amount of shrinkage: setting [tex]\lambda = 0[/tex] results in ordinary least squares, and larger values of [tex]\lambda[/tex] increase the amount of regularization, causing greater coefficients shrinkage.
Why Use L2 Regularization?:
- L2 regularization penalty is applied, which is the sum of squares of coefficients (excluding the intercept). This penalty is also referred to as the L2 norm.
- It helps to keep the model simple and reduces the risk of overfitting by shrinking the coefficient estimates.
Therefore, the type of regularization inherent in the Ridge Regression case is L2 regularization. The correct answer is (D) L2.
