Practitioner's Guide to Data for Generative AI

Introduction

• Speakers: Jonathan Katz & Siva Raghupathi (AWS since 2009)
• Presentation Context: Focus on data flows for generative AI, specifically Retrieve Augmented Generation (RAG).
• Goal: Provide strategies for building generative AI applications using private data sources.

Example Scenario

• PAT and Car Insurance: Used to illustrate how generative AI can personalize and deliver responses.
• Data Sources Needed: Driving history, payment history, accident records, etc.
• Challenge: Understanding nuances in language and tailoring responses accordingly.

Retrieve Augmented Generation (RAG)

• Basics: Involves integrating various data sources to enhance foundation models.
• Components of a Prompt:
  • Instructions for the model (e.g., don't fabricate information).
  • Situational context (facts about the user).
  • Semantic context (meaning of facts).
  • User input (depends on model).

Workflow

• Initial Steps:
  • User application receives a question.
  • Query a prompt repository.
  • Retrieve situational and semantic context.
  • Vector embedding for user input to match semantically related information.
• Execution: Send compiled information to the generative AI model to get a response.

Contexts in RAG

• Situational Context: Facts about the current user, typically sourced from databases.
• Semantic Context: Provides meaning, often from transformed data sources or vector databases.

Advanced RAG Techniques

• Naive RAG: Basic workflow with initial setup. Suitable for simple queries.
• Advanced RAG: Incorporates hybrid search, GraphRAG, context summarization, and natural language queries for enhanced responses.
• Modular RAG: Uses query routing to manage multiple data sources.

Optimizing Data Queries

• Semantic Caching: Avoids repetitive calls to the model by caching similar queries and contexts.

Data Preparation and Management

• Types of Data:
  • Structured (defined schema, situational context).
  • Semi-structured (e.g., JSON, may evolve over time).
  • Unstructured (requires transformation for semantic meaning).
• Preparing Data:
  • Chunking strategies for unstructured data.
  • Natural language queries for structured data.

Vector Search

• Selection Criteria: Familiarity, performance needs, scale.
• Indexes: Hierarchical Navigable Small Worlds (HNSW) is popular for its ease and performance.

Data Architecture for RAG

• Serving Layer: Essential for low-latency data access.
• Backend Pipelines: Ensure data freshness and timely updates.

Key Points in Data Governance

• Data Sharing & Lineage: Ensures quality and traceability.
• Automated Systems: Minimize manual intervention and errors.

Conclusion

• Takeaways:
  • Work backwards from workflow to data flow.
  • Use existing data effectively.
  • Leverage automation wherever possible.

Feedback and Closing Remarks

• Encouraged audience feedback and thanked attendees.