Back to Problems

Implement Ridge Regression Loss Function

Write a Python function `ridge_loss` that implements the Ridge Regression loss function. The function should take a 2D numpy array `X` representing the feature matrix, a 1D numpy array `w` representing the coefficients, a 1D numpy array `y_true` representing the true labels, and a float `alpha` representing the regularization parameter. The function should return the Ridge loss, which combines the Mean Squared Error (MSE) and a regularization term.

Example

Example:
import numpy as np

X = np.array([[1, 1], [2, 1], [3, 1], [4, 1]])
w = np.array([0.2, 2])
y_true = np.array([2, 3, 4, 5])
alpha = 0.1

loss = ridge_loss(X, w, y_true, alpha)
print(loss)
# Expected Output: 2.204

Ridge Regression Loss

Ridge Regression is a linear regression method with a regularization term to prevent overfitting by controlling the size of the coefficients.

Key Concepts:

  1. Regularization: Adds a penalty to the loss function to discourage large coefficients, helping to generalize the model.
  2. Mean Squared Error (MSE): Measures the average squared difference between actual and predicted values.
  3. Penalty Term: The sum of the squared coefficients, scaled by the regularization parameter \( \lambda \), which controls the strength of the regularization.

Ridge Loss Function:

The Ridge Loss function combines MSE and the penalty term:

\[ L(\beta) = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 + \lambda \sum_{j=1}^{p} \beta_j^2 \]

Implementation Steps:

  1. Calculate MSE: Compute the average squared difference between actual and predicted values.
  2. Add Regularization Term: Compute the sum of squared coefficients multiplied by \( \lambda \).
  3. Combine and Minimize: Sum MSE and the regularization term to form the Ridge loss, then minimize this loss to find the optimal coefficients.
import numpy as np

def ridge_loss(X: np.ndarray, w: np.ndarray, y_true: np.ndarray, alpha: float) -> float:
    loss = np.mean((y_true - X @ w)**2) + alpha * np.sum(w**2)
    return loss
Contributors

Want to help out? Contribute here.

Your Solution

Output will be shown here.