Back to Problems
## Single Neuron with Backpropagation (medium)

#### Example

## Neural Network Learning with Backpropagation

This question involves implementing backpropagation for a single neuron in a neural network. The neuron processes inputs and updates parameters to minimize the Mean Squared Error (MSE) between predicted outputs and true labels.
### Mathematical Background

### Practical Implementation

This process refines the neuron's ability to predict accurately by iteratively adjusting the weights and bias based on the error gradients, optimizing the neural network's performance over multiple iterations.

Write a Python function that simulates a single neuron with sigmoid activation, and implements backpropagation to update the neuron's weights and bias. The function should take a list of feature vectors, associated true binary labels, initial weights, initial bias, a learning rate, and the number of epochs. The function should update the weights and bias using gradient descent based on the MSE loss, and return the updated weights, bias, and a list of MSE values for each epoch, each rounded to four decimal places.

Example: input: features = [[1.0, 2.0], [2.0, 1.0], [-1.0, -2.0]], labels = [1, 0, 0], initial_weights = [0.1, -0.2], initial_bias = 0.0, learning_rate = 0.1, epochs = 2 output: updated_weights = [0.1036, -0.1425], updated_bias = -0.0167, mse_values =[0.3033, 0.2942] reasoning: The neuron receives feature vectors and computes predictions using the sigmoid activation. Based on the predictions and true labels, the gradients of MSE loss with respect to weights and bias are computed and used to update the model parameters across epochs.

**Forward Pass**:- Compute the neuron output by calculating the dot product of the weights and input features and adding the bias: \[ z = w_1x_1 + w_2x_2 + ... + w_nx_n + b \] \[ \sigma(z) = \frac{1}{1 + e^{-z}} \]

**Loss Calculation (MSE)**:- The Mean Squared Error is used to quantify the error between the neuron's predictions and the actual labels: \[ MSE = \frac{1}{n} \sum_{i=1}^{n} (\sigma(z_i) - y_i)^2 \]

**Backward Pass (Gradient Calculation)**:- Compute the gradient of the MSE with respect to each weight and the bias. This involves the partial derivatives of the loss function with respect to the output of the neuron, multiplied by the derivative of the sigmoid function: \[ \frac{\partial MSE}{\partial w_j} = \frac{2}{n} \sum_{i=1}^{n} (\sigma(z_i) - y_i) \sigma'(z_i) x_{ij} \] \[ \frac{\partial MSE}{\partial b} = \frac{2}{n} \sum_{i=1}^{n} (\sigma(z_i) - y_i) \sigma'(z_i) \]

**Parameter Update**:- Update each weight and the bias by subtracting a portion of the gradient determined by the learning rate: \[ w_j = w_j - \alpha \frac{\partial MSE}{\partial w_j} \] \[ b = b - \alpha \frac{\partial MSE}{\partial b} \]

import numpy as np def sigmoid(x): return 1 / (1 + np.exp(-x)) def train_neuron(features, labels, initial_weights, initial_bias, learning_rate, epochs): weights = np.array(initial_weights) bias = initial_bias features = np.array(features) labels = np.array(labels) mse_values = [] for _ in range(epochs): z = np.dot(features, weights) + bias predictions = sigmoid(z) mse = np.mean((predictions - labels) ** 2) mse_values.append(round(mse, 4)) # Gradient calculation for weights and bias errors = predictions - labels weight_gradients = (2/len(labels)) * np.dot(features.T, errors * predictions * (1 - predictions)) bias_gradient = (2/len(labels)) * np.sum(errors * predictions * (1 - predictions)) # Update weights and bias weights -= learning_rate * weight_gradients bias -= learning_rate * bias_gradient # Round weights and bias for output updated_weights = np.round(weights, 4) updated_bias = round(bias, 4) return updated_weights.tolist(), updated_bias, mse_values

There’s no video solution available yet 😔, but you can be the first to submit one at: GitHub link.

Output will be shown here.

Solution copied to clipboard!