Lecture on Generative AI and LLMs

Introduction

Audibility and Visibility Check: Initial checks to ensure the speaker is audible and visible to all participants.
Session Start: Aim to start the session after initial checks.

Schedule: Sessions will occur over the next two weeks, daily from 3 PM to 5 PM.
Content Breakdown: Starting from basic to advanced generative AI concepts. Incorporates theoretical and practical application development.
Teaching Method: First the theory, then practical application development (open AI, LLMs, etc.).

Dashboard Usage: All lectures, videos, assignments, and quizzes will be uploaded here.
Enrollment: Free enrollment for participants.
Inon YouTube Channel: Videos will also be available on YouTube.
Contributors: Sessions conducted by Sunny Savita and Buppi.

Generative AI Overview: What is generative AI, types of applications we can create, and foundational theory.
Large Language Models (LLMs): History, functionality, and applications of LLMs.
OpenAI & LangChain: Detailed walkthrough, dashboard functionality, and practical demonstrations using Python API.
Vector Databases: Need for vector databases, embeddings, and their role in AI applications.
Open Source Models: Llama, Falcon, Bloom, and others will be discussed.
End-to-End Project Development: From theory to deployment, including MLOps concepts.

Basic Knowledge: A foundational understanding of Python, machine learning, and deep learning concepts.
Course Level: No advanced prerequisites required, basic knowledge of Python and ML/DL will suffice.

Generative AI: Types like Image-to-Image, Text-to-Text, Image-to-Text, and Text-to-Image generations.
Deep Learning Overview: ANN, CNN, RNN, types of neural networks, and their uses.
RNN and LSTM: Sequence data processing, memory units in LSTMs, and GRU.
Encoder-Decoder Paradigm: Context vector, sequence-to-sequence mapping issues, attention mechanisms.
Attention Mechanisms: Overview of the 2014 paper introducing attention in sequence models.
Transformers: Base architecture for most modern LLMs. Overview of the paper "Attention is All You Need." Transformer encoders and decoders, multi-headed attention, positional encoding.

Discriminative Models: Classical supervised models like RNNs for fixed input-output lengths.
Generative Models: Training involves unsupervised learning, supervised fine-tuning, and reinforcement learning; capable of generating new data.

Definition: Trained on huge datasets, capable of multiple tasks (text generation, summarization, etc.).
Bases of LLMs: Large data needs, complexity in neural networks, unsupervised learning, supervised fine-tuning.
Model Types: Encoder-only (BERT), Decoder-only (GPT), Encoder-Decoder (T5).
Applications: Various tasks like transcription, translation, question answering.
OpenAI and Open Source Models: Overview of models like GPT-3, GPT-4, Bloom, Llama 2.

Session Plans: Practical implementation starts tomorrow, focusing on OpenAI's Python API.
APIs and Platforms: How to utilize various tools like Hugging Face, OpenAI, and AI21 Labs for different AI tasks.

Next Session: Tomorrow at 3 PM focusing on practical applications using OpenAI.
Final Remarks: Resources and recorded sessions will be available on the dashboard.

QA Session: Addressing prerequisites, applications in different domains, mathematical intuition.