Architecture
Encoder-Decoder
Transformer with both an encoder reading inputs and a decoder generating outputs.
Definition
Encoder-decoder transformers (T5, BART, mT5) separate the read and write stages: the encoder ingests the source, the decoder generates conditioned on the encoder's output via cross-attention. They remain strong for translation, summarisation and seq2seq tasks where input and output are clearly distinct.
Common use cases
- Machine translation
- Summarisation
- Speech recognition