Lecture Notes: Adding Knowledge Files to Large Language Models

Introduction

• Presenter: Dave
• Topic: How to add your own knowledge files and documents to a large language model (LLM) both locally and online.

Key Methods for Adding Information

1. Retraining a Model
2. Retrieval Augmented Generation (RAG)
3. Uploading Documents to the Context Window

Demonstration

• Demo of a modest-sized LLM running locally on Dual Nvidia RTX 60008 setup.
• Comparison between running a 1 billion parameter model and a 70 billion parameter model.
  • 1 billion model: Sustains over 300 tokens/second.
  • 70 billion model: Around 20 tokens/second.

Adding Information Methods Explained

1. Retraining a Model

• Analogy: Like sending a student back to school with new books for permanent knowledge.
• Pros: Long-term, permanent update of knowledge.
• Cons: Requires significant resources and computing power.

2. Retrieval Augmented Generation (RAG)

• Analogy: Student consulting a library for the latest information.
• Pros: Agile and dynamic; no deep permanent learning.
• Cons: Requires a database for information retrieval.

3. Uploading Documents to the Context Window

• Analogy: Student using a cheat sheet during an exam.
• Pros: Quickest way to provide immediate knowledge.
• Cons: Temporary; only lasts during the session.

Why Not Retrain?

• Openness: Lack of access to modify models like ChatGPT.
• Hardware & Software Requirements: Need for serious hardware and programming skills.

Focus on RAG and Context

• Detailed steps for using RAG and context documents.

Using ChatGPT

• Uploading documents using the upload button to enhance current context.
• Demo: Uploading PDP 1134 manual and querying for specific information.

Creating Custom GPTs with Context Documents

• Steps:
  1. Use "Create" function in ChatGPT to form a specialized GPT.
  2. Upload relevant documents (e.g., PDP manuals).
  3. Query the custom GPT for specific knowledge.

Local Usage with oLama and Open WebUI

• Demonstration of setting up context documents locally.
• Steps:
  1. Run Open WebUI on a local machine.
  2. Upload files and set as context.
  3. Query using context documents.

RAG Setup

• Benefits: Efficiently handles large data, pulls necessary information.
• Steps:
  1. Configure RAG by scanning and indexing documents.
  2. Use in queries to combine dynamic retrieval with existing knowledge.

Conclusion

• Summary: Retraining, RAG, and context window usage each have unique strengths for different user requirements.
• Final Note: Follow Dave's channel for more episodes and insights.

Additional Resources

• Dave's Attic: Weekly podcast covering viewer questions and further discussions.