Architecture
aka SSM
State Space Model
Sequence model that uses continuous-time linear dynamics instead of attention.
Definition
State-space models (S4, Mamba) propagate a hidden state with linear recurrences, yielding O(n) sequence length scaling versus attention's O(n²). They match transformer quality at moderate scale and excel on very long sequences. Recent hybrids combine SSM blocks with attention.
Common use cases
- Long-context modelling
- Genomics
- Time-series