Attention Mechanism

Neural Networks

Focusing on relevant parts of input

Allows models to focus on specific parts of input when making predictions, like humans focusing attention on important details.

Core component of modern transformers and seq2seq models

Learn more about concepts related to Attention Mechanism

Transformer

Attention-based architecture for sequence processing

BERT (Bidirectional Encoder Representations from Transformers)

Language model that understands context from both directions

GPT (Generative Pre-trained Transformer)

Large language model for text generation