Architecture
aka SSM
State Space Model
Sequence model that uses continuous-time linear dynamics instead of attention.
Definition
State-space models (S4, Mamba) propagate a hidden state with linear recurrences, yielding O(n) sequence length scaling versus attention's O(nÂČ). They match transformer quality at moderate scale and excel on very long sequences. Recent hybrids combine SSM blocks with attention.
Common use cases
- Long-context modelling
- Genomics
- Time-series