Encoder–Decoder Model

Encoder–Decoder Model is a neural network architecture that converts variable-length input sequences into variable-length outputs by chaining two components: an encoder that turns the input (text, audio, image) into a sequence of contextual representations and a decoder that generates the target sequence one step at a time, attending to those representations through cross-attention; in the earlier RNN-based seq2seq models the encoder instead produced a single fixed-size context vector. Introduced for neural machine translation, it underpins tasks such as speech recognition, text summarization, and image captioning. Variants range from RNN-based seq2seq with attention to fully Transformer encoder–decoders like T5 and MarianMT. Key metrics—BLEU, ROUGE, Word-Error Rate—measure fluency and fidelity, while beam search, teacher forcing, and scheduled sampling refine training and inference. By cleanly separating understanding from generation, an Encoder–Decoder Model handles length mismatches, supports multilingual transfer, and can act as the generator in Retrieval-Augmented Generation (RAG) pipelines, where retrieval normally relies on a separate embedding model and generation is handled by either a decoder-only or an encoder–decoder model.

Related terms

• Decoder-Only Model

Related services: Agentic AI consulting.

← Back to the AI Glossary