An RBF network is a one-hidden-layer net whose hidden units are Gaussian kernels around fixed centroids μ1,…,μK:
ϕ(x)=[1, K(x,μ1),…,K(x,μK)],K(x,μ)=exp(−2σ2∥x−μ∥2)
(a leading constant feature serves as the bias). With the centroids fixed, only the output weights w are learned.
Implement:
rbf_design(X, centers, sigma): the N×(K+1) design matrix Φ with rows ϕ(xn).
rbf_fit(X, y, centers, sigma, lam, task, n_newton=20): returns w of length K+1. The penalty 2λ∥w1:K∥2 applies to every weight except the bias w0.
task="reg": minimise 21∥y−Φw∥2+2λ∥w1:K∥2 exactly.
task="clf" (labels in {0,1}, pn=sigmoid(ϕ(xn)⊤w)): minimise the negative log-likelihood plus the same penalty with n_newton Newton steps starting from w=0, each step w←w−H−1g with g=Φ⊤(p−y)+Λw and H=Φ⊤diag(p(1−p))Φ+Λ, where Λ=λdiag(0,1,…,1).
rbf_predict(X, centers, sigma, w, task): Φw for regression, sigmoid(Φw) for classification.