Architecture
Decoder-only
Transformer variant with only a stack of causal-attention decoder layers, no encoder.
Definition
Decoder-only architectures, popularised by GPT, drop the encoder and use a single stack of causal transformer blocks. They train by next-token prediction on raw text and have become the dominant design for general-purpose LLMs because of their simplicity and scaling behaviour.
Common use cases
- Chat
- Code
- Reasoning
Related models
gpt
claude
llama
mistral