Concept
Encoder
Network half that compresses an input sequence into contextualised representations.
Definition
An encoder reads the full input at once and emits a vector or sequence of vectors that downstream layers (or a decoder) consume. BERT-style models are encoder-only and excel at classification and embeddings. Encoder-decoder models like T5 use an encoder for the source and a decoder for the target.
Common use cases
- Classification
- Embeddings
- Translation source side