Implement Lasso regression with ISTA (proximal gradient descent): soft_threshold(w, threshold) and l1_regularization_gradient_descent(X, y, alpha, learning_rate, max_iter, tol).
Objective: J(w,b)=2n1∑i=1n(yi−y^i)2+α∑j∣wj∣
- Soft-thresholding, element-wise: S(w,λ)=sign(w)⋅max(∣w∣−λ,0).
- Start from zero weights and bias 0. Each iteration: take one gradient step of size
learning_rate on the MSE term for both w and b (the bias is not regularized), then set w←S(w, learning_rate⋅α).
- Stop after
max_iter iterations, or earlier when the L2 norm of the weight change in an iteration is below tol.
- Return
(weights, bias): an array of shape (n_features,) and a float.