Generative AI Mini Course Lecture Notes

Introduction

• Topics Covered:
  • Gen AI Fundamentals
  • LangChain framework (Python)
  • Two end-to-end Gen AI projects
• Projects:
  • Equity news research tool using commercial GPT model
  • Q&A tool in Retail industry using open-source LLM model

Generative AI Fundamentals

• Types of AI:
  • Generative AI
  • Non-generative AI
• Non-generative AI: Decision-making using existing data (e.g., medical diagnosis, credit scoring).
• Generative AI: Creates new content (e.g., ChatGPT, image generation).
• Applications: Text, images, video, audio.

Evolution of AI

• Early Days: Statistical Machine Learning
  • Home price prediction using features like area, bedroom count, etc.
• Image Recognition: Complex features like whiskers, pointy ears for cats
• Deep Learning: Neural networks for complex feature identification
  • Gave birth to deep learning
• Recurrent Neural Networks (RNN): Used for language translation.
  • Feed each word and previous translation to the same network.
• Transformers: Key breakthrough with paper “Attention is All You Need”
  • Basis for models like BERT, GPT-3, GPT-4.
  • Enabled sophisticated generative tasks like autocomplete, Q&A.
  • Varieties: Google’s BERT, OpenAI’s GPT (e.g., GPT-4 for ChatGPT), image models (DALL-E, Stable Diffusion).
  • Applications in text generation, image creation, and video generation.

Language Models and LLMs

• Language Models:
  • Predict the next word in a sequence.
  • Training using Wikipedia, books, news articles (self-supervised learning).
• Large Language Models (LLMs):
  • Capable of more complex tasks.
  • Example: GPT-4 with 175 billion parameters.
• Breakthrough with Transformer Architecture:
  • Varieties like BERT, GPT, DALL-E.

Other Key Concepts

• Stochastic Parrot: Mimicking probability-based language without understanding.
• Embeddings and Vector Database:
  • Numeric representation of text for capturing meaning.
  • Used for semantic search (e.g., Google search differentiation between 'Apple' as fruit or company).

Vector Databases

• Purpose: Efficient search and storage of embeddings.
• Examples: Pinecone, Milvus, Chroma, FAISS.
• Applications: Semantic search, similarity matching.

Retrieval-Augmented Generation (RAG)

• Concept: Use of external data sources to answer questions.
• Analogy: Open-book exam technique.
  • Fine-tuning model on specific data sets (open-book concept).

Tools for Generative AI Applications

• Examples: GPT-4, BERT, DALL-E, LangChain, Hugging Face Transformers, PyTorch, TensorFlow.

Project 1: Equity News Research Tool

• Goal: Use GPT-4 to build a research tool for equity news.
• Steps:
  • Load news articles.
  • Split text into meaningful chunks.
  • Use embeddings to store in a vector database.
  • Summarize and answer queries based on retrieved chunks.

Project 2: Q&A Tool in Retail Industry

• Goal: Build Q&A system using an open-source LLM.
• Steps:
  • Load retail data (e.g., inventory, discounts).
  • Convert user questions into SQL queries to fetch relevant data.
  • Fine-tune model using few-shot learning for complex queries.

Conclusion

• Generative AI has broad applications across industries.
• Practical, end-to-end projects using frameworks like LangChain complement theoretical knowledge.
• Continuous learning and application of concepts are crucial.