Understanding LLM Hallucinations and Fixes

Overview

Topic: LLM hallucination — why large language models produce incorrect confident answers.
Goal: Explain causes, examples, and mitigation strategies before diving into RAG (Retrieval-Augmented Generation).
Format: Short lecture-style explanation with practical suggestions for application development.

Definition: LLM produces fluent but factually incorrect or fabricated answers.
Typical symptom: Model asserts incorrect statistics, facts, or examples confidently.
Analogy used: LLM like an "arrogant friend" who answers even when unsure.

Training Cutoff Date
- Models only know data up to a cutoff; they lack knowledge of newer facts.
- Example: A model trained up to a date cannot answer events after that date.
Insufficient Training Data
- Lack of sufficient examples for certain tasks causes incorrect or unstable outputs.
- Example: Basic arithmetic or uncommon queries may be answered incorrectly.
Inherent Model Behavior
- LLMs generate plausible continuations, which can include fabricated facts.
- They may produce statistics or citations that sound believable but are invented.

For casual users it is acceptable sometimes, but in production or company settings:
- Incorrect outputs can cause harm, wrong decisions, or loss of trust.
- Enterprise apps need reliable factual grounding and accountability.

Integrate External Tools
- Connect LLMs to web search, databases, third-party APIs, or RAG systems.
- Use these tools to supply up-to-date and authoritative context.
Retrieval-Augmented Generation (RAG)
- Store company data in a vector store or vector database.
- At query time, retrieve context and feed it to the LLM to ground answers.
- RAG reduces hallucination by providing factual context; it cannot eliminate it.
Fine-Tuning
- Fine-tune models on domain-specific data to reduce mistakes.
- Constraint: fine-tuning can be expensive and resource-intensive.
Prompt Engineering
- Helpful but not sufficient alone to fully stop hallucination.
Human Verification / Feedback
- Add a human-in-the-loop to verify or correct critical outputs.
- Collect feedback to improve the system iteratively.

Architecture: LLM + Retriever + Vector Store
- Query -> Retriever fetches relevant documents -> LLM generates answer with retrieved context.
Benefits
- Grounds responses in company data and reduces fabrication.
- Improves factual accuracy by about 5–10% in many cases; overall hallucination can be reduced but not eliminated.
Limitations
- If no relevant context is found, the LLM may still fabricate answers.
- Expect partial improvement (e.g., 20–30% reduction in hallucination in typical scenarios).

Term	Definition
LLM Hallucination	When an LLM generates plausible but incorrect or fabricated information.
Training Cutoff Date	The latest date of data used to train an LLM; model lacks later information.
RAG (Retrieval-Augmented Generation)	Approach combining retrieval (vector stores/search) with LLM generation to ground answers.
Vector Store	A database storing embeddings for retrieval of context relevant to queries.
Fine-Tuning	Re-training or adapting an LLM on domain-specific data to improve accuracy.

Learn RAG architecture and components: retrievers, vector stores, and prompting strategies.
Practice integrating an LLM with a vector store and simple retrieval pipeline.
Experiment with human-in-the-loop verification for critical outputs.
Investigate cost and feasibility of fine-tuning for your domain before choosing it.
Follow up: next lecture will cover RAG applications and implementation details (e.g., using LangChain or LangGraph).