Architecture
Perceiver
Transformer variant that cross-attends to a fixed-size latent array, decoupling from input size.
Definition
The Perceiver introduces a small set of learned latent queries that cross-attend to arbitrarily large inputs (images, point clouds, audio). This breaks attention's quadratic cost in input length and makes the architecture modality-agnostic.
Common use cases
- Multimodal fusion
- Point clouds
- Long sequences