Architecture
Mamba
Selective state-space model with input-dependent dynamics, competitive with transformers.
Definition
Mamba (2023) augments classical SSMs with selectivity — its state-transition matrices depend on the input — letting it focus on relevant tokens like attention does. Mamba-2 and hybrid Jamba/Zamba show that SSM blocks can replace or complement self-attention at scale.
Common use cases
- Long-context LLMs
- Audio
- DNA modelling