Architecture
Sparse Attention
Attention variants that restrict each query to a subset of keys, reducing compute.
Definition
Sparse attention patterns — sliding window, dilated, block-sparse, sink — limit each token's view to a fixed-size neighbourhood or a few global tokens. This brings memory and compute down from quadratic toward near-linear, enabling 100k+ context windows.
Common use cases
- Long-context models
- Mixed-context inference
- Efficient serving