Lecture Notes: Retrieval Augmented Generation (RAG) and Agentic RAG

Introduction to Retrieval Augmented Generation (RAG)

• Definition: A pipeline that improves LLM responses by incorporating relevant data from a vector database as context.
• Process:
  • User/application sends a query.
  • Query is interpolated into a prompt for the LLM.
  • LLM generates an output based on the prompt.
• Enhancement:
  • Use a vector database to retrieve relevant data and add it as context to the LLM prompt.
  • Improves quality and reliability of LLM responses by grounding them in concrete information.

Agentic RAG

• Evolution of RAG: Uses the LLM not just for responses but also for decision-making tasks.
• Capabilities:
  • LLM can decide which vector database to query if multiple are available.
  • Determines type of response needed (text, chart, code snippet) based on query context.
• Agent Role: LLM acts as an agent to improve relevance and accuracy of retrieved data.

Augmenting the Process with an Agent

• Multiple Data Sources:
  • Internal documentation (policies, procedures, guidelines).
  • General industry knowledge (standards, best practices, public resources).
• Agent’s Functionality:
  • Intelligently decides which database to query based on user question.
  • Utilizes LLM's language understanding to interpret query and determine context.
  • Routes queries to the most relevant database:
    • Company policy questions to internal documentation.
    • Industry standard questions to general knowledge database.
  • Recognizes irrelevant queries and routes to a failsafe.

Use Cases and Applications

• Fields:
  • Customer support systems.
  • Legal tech.
  • Potentially any field as technology evolves (health care, etc.).
• Examples:
  • Lawyers accessing internal briefs or public caseload databases.
  • Systems retrieving real-time data or incorporating third-party services.

Conclusion

• Agentic RAG: Enhances RAG pipeline by enabling intelligent decision-making.
• Benefits:
  • More responsive, accurate, and adaptable pipeline.
  • AI systems that understand context and deliver value to users.

These notes provide a summary of the lecture on RAG and Agentic RAG, highlighting their processes, enhancements, and applications in various fields.