Two cheap variants of convolution from Murphy's book. Inputs are (H, W, C) arrays, convolutions are valid (no padding) cross-correlations with stride 1.
Dilated convolution with rate and kernel of shape (kh, kw, C, D):
The output has spatial size .
Depthwise separable convolution: each input channel is convolved with its own 2-D filter w_dw[:, :, c] (shape (kh, kw, C) in total), then a convolution w_pw of shape (C, D) mixes channels:
Implement:
dilated_conv2d(X, w, b, rate) and separable_conv2d(X, w_dw, w_pw, b), returning (H_out, W_out, D) arrays.conv_costs(H, W, C, kh, kw, D) returning (std_params, std_macs, sep_params, sep_macs) for a valid stride-1 layer: weight counts (biases excluded) and multiply-accumulates. The standard layer has weights and spends MACs per output pixel; the separable layer has weights and spends (depthwise) plus (pointwise) MACs per output pixel.