How a Large Language Model (LLM) Answers a Question
End-to-End Inference Architecture & Retrieval-Augmented Generation (RAG) Flow
1. Input & Encoding Pipeline
2. Core Model & RAG Augmentation
3. Decoding & Output
Input
Tokens
Embeddings
Processed Data
Query for context
Generated Tokens
Text Output
User Interaction/Evaluation
Future Model Updates
1
User's Question
• Raw natural language
• Prompt input query
2
Tokenization
• Breaks input into
smaller units (tokens)
3
Embedding
• Numerical vectors
• Captures semantics
4
Transformer Layers
• Processes embeddings
• Self-attention mechanisms
5
Next Token Prediction
• Next-token probabilities
• Autoregressive generation
6
RAG (Optional)
• Retrieves relevant passages
• Augments prompt context
7
Detokenization
• Converts tokens to
human-readable text
8
LLM Answer
• Final text output
• Delivered to user
9
Feedback Loop
• Refines & improves future responses
STAGE 1 / 6
Prompt Input & Tokenization
The user submits a question, which the tokenizer breaks down into smaller units called tokens (words, subwords, or characters).
◀ Prev
Pause
Next ▶
Sources:
cruxdigits.nl • medium.com (@bervice, @manavg) • maryayaqin.com • scaler.com • quantumobile.com • aws.amazon.com/what-is/retrieval-augmented-generation