Mamba's state space model is written in continuous time, , and turned into a discrete recurrence using a step size . Because and are selective (computed from the input), every token and every channel gets its own discretisation. Implement it for a diagonal .
discretize(delta, A, B_in, method="zoh")
delta has shape (B, L, ED) (batch, sequence length, inner channels) and is positive.A has shape (ED, N): row d is the diagonal of channel d's state matrix.B_in has shape (B, L, N): one input vector per token, shared by all ED channels.(A_bar, B_bar), two float arrays of shape (B, L, ED, N). For both methods:method="zoh" (zero-order hold, exact for a diagonal ). Where use the limit of this expression as ; the result must be finite.method="euler" (the first-order simplification used for in the reference implementation):delta broadcasts against A over N and B_in broadcasts over ED; with ED == N a wrong axis gives no error, only wrong numbers.np.where evaluates both branches, so a masked division by zero still warns. Replace the zeros in the denominator before dividing.np.expm1 keeps accurate when is tiny.