Transcript for:
Generative AI Mini Course Lecture Notes

welcome to this generative AI mini course first we will understand the Gen AI fundamentals then we will learn Lang chain which is a python framework used for building gen application and in the end we will build two endtoend gen AI projects the first project will be using commercial GPT model where we will build equity news research tool the second project will be using open-source llm model where we will build a Q&A tool in retail industry let's start with the definition of gen ai ai can be categorized into two sections generative ai non-generative ai when you talk about non-generative AI you are dealing with problems such as you have a chest x-ray and you want to find out if this person has pneumonia or not or maybe you have some data on person's credit history and you want to figure out if the person should be given a loan or not in these problems you are not creating new content you have data and based on that data you are making certain decisions in the case of generative AI however you are generating new content the classical example is chat GPD chat GPD is a gen application here you can write your resume you can plan your trip you can even create an image of Hulk playing Gujarati dandia in summary generative AI is a category of AI that is associated with generating new content and this new content can be text images video audio Etc let's now look at the evolution of jna in the early days of AI the kind of problems that we used to solve was predicting home price based on factors such as the area the bedrooms the age Etc this is called statistical machine learning and the factors which determine the price of the home such as area bedroom age Etc they were called features now these were simple features when it comes to image recognization such as identifying if the image is cat or dog the feature was kind of complex see here the features are the whiskers or the pointy ears that the cat has and the this data which is an image data is just a bunch of pixel so it is unstructured data in the case of house price prediction you have structur data area bedroom Etc so therefore these features in an image they are called complex features the cat's face could be in a different angle and this ear can be in let's say in this corner or let's say the legs are not present at all so that makes uh image detection problem much complex and sometimes you can't use statistical ml for this therefore neural networks were invented and that gave birth to deep learning so there was a statistical machine learning before then came deep learning where neural networks was the main approach that we used used after that came recurrent neural network so in recurrent neural network what you have is this kind of network and let's say you are trying to translate this particular sentence in a different language let's say you want to translate from English to Hindi what you do is you feed the first word to your neural network so I will feed D to a neural network and it will give me this translation this A1 is nothing but the translation of D in let's say Hindi after that you feed the second word and the translation of previous word to the same network so these are not four different networks okay this is just a time exess it's the same network where you feed the translation of the previous word or previous sentence so that created a kind of a loop within the neural network and it is called recurrent neural network this was used for solving problems like language translation and then came more sophisticated problem which was kind of generative in nature so here is the email which I got and in Gmail when I try to respond it will try to autocomplete you just notice that when I say thanks for reaching out here's my availability see it is autoc completing all these words and the way it works is it looks at the content of the email so let's say Angela said hey we have a potential collaboration opportunity do you have time to talk so the network will read this sentence and then uh it will try to predict the probability of the next word so when I'm saying Angela thanks for the probability of uh saying contacting me is higher and probability is generally between zero and one let's say contacting me has a probability of 091 I can also say thanks for reaching out and probability of that could be higher I can say thanks for finishing the project but uh in the context of the previous email that Angela send me the probability of this might be lower I'm putting these numbers randomly but you get an idea if Angela had said that uh hey D I have finished the project please check it out in that case I can say Angela thanks for finishing the project so if we had that kind of context then the probability here would be higher but since we're talking about the collaboration the probability of finishing the project is lower so you get an idea here if you pass huge Corpus of text and try to build a language understanding you will be able to predict the next word that comes in the sentence and this is called language model it is an AI model that can predict the next word or set of words for a given sequence of words so here I have text from Wikipedia okay and just look at it uh this is the Wikipedia article on India's Freedom Movement and here to train the neural network Network we can create the training pairs from this paragraph we can uh come up with uh a problem that we want to solve and the problem is uh filling the missing word so here I'm saying first Indian nationalist so first Indian nationalist to embrace okay so what is the missing word here well nationalist similarly we can come up with this kind of training Pairs and then we give these training pairs to neural network for training okay and this approach of training neural network is called self-supervised learning so the good point here is that when you want to train a language model you don't need a lot of label data you can take Wikipedia text text from uh news articles text from variety of books and generate these training Pairs and then train the neural network and after that neural network will be able to predict the missing word or the word that comes next in the sentence just like how Gmail autocomplete is working and when you feed huge amount of data to this neural network and let's say when you have so many layers in the neural network let's say that neural network is Big what we get is large language model gp4 a model behind chat GPT is a large language model it has 100 75 billion parameters and when I say parameters is nothing but all these weights in the neural network so see this neural network has only 10 parameters imagine a huge network with so many layers having 175 billion parameters the critical breakthrough in uh jni came when this particular paper called attention is all you need was published which gave rise to a spatial neural network architecture called Transformer so just to summarize previously we had statistical machine learning then came deep learning which was based on neural network then came recurrent neural network and then came Transformers which is very powerful and talking about Transformers we have variety of architectures or variety of models for example Google has this model called bird open AI came up with this model called GPT if you look at your chat GPT application you will find this model right now it's gp4 so that is the model that they use and it's called generative pre-rain Transformer that's a long form of GPT similar to text model we have image models too for example Del stable diffusion Etc this is the image generated by a model called stable diffusion and if you have a chat GPD pro version you can generate image for example generating an image of a person selling what on mask look at this image I mean is quite better I know this person is not wearing the mask but still it's doing pretty decent job to summarize we have text to text model such as bird GPT we have text to image model such as di stable diffusion where you give text and generates image and open a SORA is an example of text to or video model you can just Google open as s you'll find the demo you can give a text prompt and it will generate an entire video out of it so all of this has become possible because of Transformer architecture and now that you have good understanding of variety of models in geni space let's look at an analogy based understanding of llm what I'm trying to do here is clarify all these Concepts such as llm Vector DB Etc because these concepts are used heavily in the field of gen okay so next I'm going to play an analogy based animated video video of [Music] llm Peter Pand has a curious parrot called buddy buddy has a great mimicking ability and a sharp memory Shar memory buddy listens to all the conversations in Peter's home and can mimic them very accurately now when he hears feeling hungry I would like to have some Biryani for this case the probability of him saying Biryani cherries or food is much higher than the words such as bicycle or book Budd he doesn't understand the meaning of Biryani or food or cherries the way humans do all he's doing is using statistical probability along with some Randomness to predict the next word or set of words purely based on the past conversations he has listened to we can call Buddy is stochastic parot stochastic Means A system that is characterized by Randomness or probability a language model is somewhat like a stochastic parrot there are computer programs that use a technology called neural networks to predict the next set of words for a sentence for a simple explanation of a neural network please watch this particular video just like how Budd is trained on Peter's home conversations data set you can have a language model that is trained on for example all movie related articles from Wikipedia and it will be able to predict the next set of words for a movie related sentence Gmail autocomplete is one of the many applications that uses a language model underneath now that we have some understanding of a language model let's understand what the heck is a large language model let's go back to our bir example allbody got some Divine superpower and now he can listen to Peter's neighbor conversations conversations that are happening in schools and universities in the town in fact not only in his town but all the towns across the world with this extra power and knowledge nowbody can complete the next set of work words on a history subject give your nutrition advice or even write a poem like our powerful parrot buddy large language models are trained on a huge volume of data such as Wikipedia articles Google news articles online books and so on if you look inside the llm you will find a neural network containing trillions of parameters that can capture more complex patterns and nuances in a language chat GPT is an application that uses llm called gpt3 or gp4 behind the scenes other examples of llms are palm2 by Google and Lama by meta on top of statistical predictions llm uses another approach called reinforcement learning with human feedback rlf let's understand this once again with Buddy one day Peter was having a conversation with his cute little 2-year-old son son don't eat too much bananas else I will punish you with an iron Rod hearing this Peter realized that buddy has been listening to the conversation from abusive parents in his town what he said was the effect of that Peter then starts keeping a close eye on what buddy is saying for the same question buddy can produce multiple answers and all Peter has to do is tell him which one is toxic and which one is not after this training budy doesn't use any toxic language while training CAD GPT open a used a similar approach of human intervention rlf open a used a huge Workforce of humans to make Chad GPT less toxic while LMS are very powerful they don't have any subjective experience emotions or Consciousness that we as humans have llms work purely based on the data that they have been trained on now that you have some understanding of llm let let's cover two other important topics called embeddings and Vector database embedding is nothing but a numeric representation of text in form of a vector such that you can capture the meaning of that text once you create embeddings for a given text you can do math with the words and sentences such as Paris minus France plus India equal to Delhi or apple minus Tim Cook plus sat Nela equal to Microsoft this sounds crazy right but it is actually possible due to embeddings and Vector database allows to store these embeddings and perform efficient search on these embeddings so let's try to understand these Concepts in a bit more detail we'll start first with the startup boom that is going on in the field of vector databases there are some AI startups that have raised millions of dollars of funding and they have one product in common which is Vector database let's try to understand what exactly is Vector database today when you search in Google calories in apple versus employees in apple Google figures out that the first Apple means fruit and the second one is a company have you ever wondered how does Google does this it uses a technique called semantic search semantic search means not searching using the exact keyword matching but understanding the intent of a user query and using the context to perform the search for doing semantic search internally it uses the concept of embedding word embedding or sentence embedding is nothing but a numerical representation of text let's first understand how exactly embedding Works let's figure out how you can represent this word Apple into a numeric presentation given this particular context one way is to think about different features of properties of words here you can have related to phones e location has talk Etc as properties and then you assign value for each of these properties Revenue here means $82 billion you get a sequence of numbers as a result and that is nothing but a vector so this Vector is a word embedding for the word Apple for this particular context if you're talking about apple the fruit then the embedding might look something different because the value of these properties is different and when you have the embeddings for different words looking at the embedding you can say that the second apple and the word orange they are similar because look at their values they're matching of course there are some values which are not matching but compared to this vector and the first Vector second and third Vector are kind of similar same way if you have let's say Samsung as a word you can represent that into a numeric uh presentation is it related to phones yes is it a location no and when you look at again all the vectors you can figure out that the first and the fourth vectors are kind of similar so using these vectors you can figure out the similarity not just similarity you can actually do a complex arithmetics such as this this is a famous example in the NLP domain where you can perform this mathematics using a technique called word to word to is a technique to represent word into a numeric representation I have made a separate video so if you want to know more about it you can go and look at this video in that video I have explained how you can generate handcrafted features for each of these and you can do this particular math now just for intuition I explain everything using handcrafted features but in reality you use some complex statistical techniques to generate these word embeddings again if you have curiosity you can watch those two videos or this particular video on bir so far let's say you have this understanding that there are variety of these techniques that you can use to represent a word or a sentence or even a document into an embedding and here are just different techniques which are being used uh in chat GPT era obviously Transformer based embedding techniques are getting popular so when you're using open AI API for embedding uh you know what technique it is using underneath when you're building any text based AI application you might have thousands or even millions of embedding vectors and you need to store them somewhere when you think about storing them the first option that comes to mind is a traditional relational database so let's say for our use case we have these four articles the first two red are related to apple the fruit the remaining are Apple the company you will first generate the embedding let's say using open AI API and then you will save that into let's say your SQL database now when you have a search query you will also generate embedding for that and you will try to compare this embedding with the stored embedding and try to retrieve the relevant documents here you will use a concept of cosign similarity to retrieve the matching vectors and you can display it in your Google search result this in theory works okay but in reality you will have millions of Records or even millions of Records in your database and that's when things starts getting interesting because just think about matching this Vector for a query Vector if you want to match with these stored vectors then one of the approach you can use is linear search where you go one by one and if cosine similarity is close to one then you will put that Vector into your result data set and then you can keep on going and store your result vectors now you already realize the problem if there are millions of stored Vector embeddings your computation is going to be too much you know your head will be raised because you can't handle delay and computational uh requirements for such a use case you need to do something smart how do we do this in a traditional database well we use a thing called index database index helps you search things faster similarly in this particular use case we can use one hashing function we don't need to go into detail what that hashing function is but let's say this hashing function is creating buckets of s similar looking embeddings okay and then when you have a search query you can let that go through the same hashing function which will bucket it into one of these three buckets and then within that bucket you can do individual linear search this way you are only matching with those vectors which are in bucket one you don't have to match it with bucket two and bucket three this will speed things up and this technique is called locality sensitive hashing this is one of the Techni techniques that Vector databases is using there are many such techniques and those techniques are outlined in this beautifully written article I'll provide a link to this article you can uh read through it so far you have realized that Vector databases help you do faster search they also help you store things in an optimal way so these are the two big benefits why Vector databases are gaining popularity after databases we need to understand the concept of retrieval augmented generation also known as rag in my company at lck Technologies where we work on multiple AI client projects one of the common requirements is we have this private organizational data or we have this public custom data set can we fine-tune chat GPT on this or can we build chat GPT like Solution on this specific data set and the way you can do that is by using rag let me explain this by giving you an analogy let's say you have a college student called mea mea is a very smart individual she's uh generally good with things she's studying in computer science now we want her to appear in some competitive biology exam which will be based on this book called How microbes rule the world the idea here is to not make Mera a biologist but she should just go and appear in that competitive exam and win the prize now you can do this thing in two ways option one is full-fledged training where she goes to a college attends biology classes for one entire year and becomes expert in biology option number two is if the exam committee is allowing open book exam mea can take this book with her you might be aware about open book exam concept where you can take a book in the examination and you can refer to the book to write the answer which option will you go for obviously option number two because you will save time and money both mea doesn't have to wait for one entire year to uh finish her biology studies instead she will take this book and since she's very good in terms of Reading Writing and generally she's very smart so she can quickly figure out the answers and she can uh write those answers during the exam similar to this approach when you're building rag based jna application what you do is you take question uh from a user and the llm or the large language model will refer to our database now this database could be Excel files PDF documents SQL database anything this could be either your private internal organizational data or it could be a public data the idea is you want llm to pull the answer from this particular data sources and llm can do that using this concept called rag okay we will go over technical understanding of rag when we do our end to end project later on in this video but let's first look at the tools that we can use for building gen AI applications Chad GPT is a gen AI application okay which uses gp4 as a large language model now this AI model is similar to human brain which is train see if I look at my brain it has been trained through my experiences through the college studies through the books that I have read Etc gp4 is a similar model that is trained on your Wikipedia articles books and so much of that data now having brain alone is not enough you need a body correct so I have this brain but uh if you want to uh get benefit of the knowledge in this brain you need this body like this eyes this voice which can interact and solve variety of problems similar to that gp4 model is just a just like a human brain you need body around it and that body is nothing but a backend server if you have done software engineering you might be aware about this web servers which are based on nodejs Jango fast API Etc so you build this kind of a server now when you're using chat GPT application uh the server runs in open AI Cloud but let's say tomorrow you want to build your own application let's say this kind of uh database Q&A system for retail store where you ask a question and it will pull the answer from your internal private database we are going to build this tool by the way later on okay so stay tuned but here this UI or the front end will be making a call to a backend server which can be using this gp4 model and it will use retrieval augmented generation rag to pull the answer from my SQL database we will see in when we do that project how exactly this is done but I'm just giving you an overview right now so the answer that you looking for that how much total price of inventory for small size t-shirt this answer is is available in this database and gp4 is converting your uh question into SQL query and it is pulling the answer and you can deploy this backend server into Azure Cloud uh Azure has this service called aszure open AI service where you can host your private gp4 model so that your data is protected it doesn't go out if you don't want to use Azure you can use things like Amazon badrock now Amazon badrock doesn't have GPT model but it has other foundational models such as uh Cloe and I think it has this model called llama see llama 2 mistal stable diffusion it has so many models so to summarize there are commercial models such as GPT jini Etc but there are open- Source models such as MRA Lama Etc now let's say you build your application using GPT model and tomorrow you are not able to afford the gp4 bill because they charge per token okay and let's say you want to save your cost what do you do well you use open source model such as Lama 2 so in this case you might have build time to uh develop this code let's say you have 20,000 lines of code and now you have to redo all these things wouldn't it be nice if we have some way where you write an application you write some code and you plug your gp4 model and tomorrow if you can't afford the bill you plug a different model okay so that way is Lang chain Lang chain is a python framework that is used to build J applications and it's a very popular framework nowadays so if you are building J application you'll have to learn this framework because that makes building llm apps easier to summarize the tooling for Gen first thing you need is some model some llm model it can be a commercial model such as gp4 or or open source model such as llama or mistal it can be image models as well then you need cloud service so you can use aure open AI Amazon badrock Google Cloud there are so many Cloud options that you have and then you need framework like Lang chain you can also use hugging face Transformer library to use variety of Open Source model and then in terms of deep learning libraries you can use py torch tensor flow Etc now is the time that we learn about Lang chain uh framework so I had this other Lang chain crash course on my YouTube channel uh in which I went through the Lang chain framework so I'm just going to uh play that now it is a crash course on Lang chain all right so let's get started with Lang chain now Lang chain is a framework that allows you to build applications on top of llm or large language model in this crash course video we are going to go over all the basics of Lang chain and then we will build a restaurant idea generator application using streamlet where you can input any cuisine Indian Mexican Etc and it will generate a fancy restaurant name along with the manual items first let us understand uh what is Lang chain and what kind of problem does it addressed when you are using Chad GPD as an application internally it is making call to open AI API which internally uses uh any llm such as GPD 3.5 or 4 in this case chat GPD itself is not an l l m it is an application whereas GPD 3.54 these are large language models now let's say you want to build an application for restaurant idea generator where you give a Cuisine and then it will generate a fancy name such as sto Temptation for Mexican and manual items as well let's say you give Indian Cuisine it will say Okay Curry K Palace or Sahara Palace for Arabic along with this manual items so this is a sample application which we are going to build but this is an llm based application and for this we can use the same architecture as jet GPT where we can directly call open AI API and here I have provided a screenshot of their main API uh so you can call it and you can get a behavior similar to chat GPT internally it will use GPT 3.5 or GPT 4 model and this case once again Resturant idea generator is an application similar to chat GPT but internally you are using open AI API and the llms now there are couple of limitations of following this approach and by the way uh the reason I'm telling you this is nowadays there is a big boom in the industry where every business wants to build their own llm you would think why they can't use chat GPD because chat GPD has no access to your internal organization data so people want to build applications which are based on llm okay so there is a clear demand and clear boom in the industry for this and why do business not use this kind of architecture well there are couple of things to consider first of all calling open AI API has a cost associated with it for every th000 token they will charge2 or something you can check open a pricing page but there is a cost associated with it and if you're a startup who is having funding issues and you know your budget is limited this is going to be a bottleneck for you another thing is you might have noticed Chad GP doesn't answer latest question its knowledge is limited to September 2021 as of this video recording so if you want to incorporate some latest information let's say from Google Wikipedia or somewhere else you can't get that here the other issue is attick is my own software development and data science company if I want to know how many employees joined last month chat GPT can't answer because it doesn't have access to my own internal organization data so if you use this kind of architecture for building your application uh you will hit some roadblocks or you will rather have some limitation and look open AI guys are pretty smart actually if they want they can address all of this but their stance is very clear we will provide foundational apis and building framework is something that other people should do and that's what happens see if you have just open a API it is not enough to build llm therefore you need need some kind of framework where you can call open AI gpt3 gp4 or maybe if you want to save the cost you call some open-source models such as hugging face Bloom there are so many models out there let's say you want to use them you don't want to uh spend money on open AI gpt3 model uh then this framework should provide that plug-and play uh support you know where you can integrate to one of these models and your code kind of Remains the Same this framework should also Pro provide integration with Google search Wikipedia or even integration with your own organizational databases so that the application can pull information from these various sources as well and this framework is Lang chain that is what it does it's a framework that allows you to build applications using llm okay let's install Lang chain now and uh do some initial setup let us first create an account on open AI you can go to open a website click on login and create a login using Google or individual email credentials and once your login is created you will come to a dashboard so let me just show you so you'll go to open AI say login you're logged in click on API and then from your account you can go to your manage account and API Keys you will find a key here which will look something like SK hyphen something that is like a password so you need to use that key in our code for Lang chain you can also create a separate keys for separate projects so I have some client projects going on a YouTube tutorials so for each of them I have a separate key in your case you can just use the one key by the way you can generate a new key here as well uh so let's say you copy that key to some secure place after that you won't be able to access it here so you have to delete and create a new key okay so let's say you have that key ready with you uh here then you can just import OS module and then in OS module you can create a environment variable with that particular key your key will be SK something okay uh in my case I have stored that key in one python file okay I don't want to share that key with all of you that is a reason and that python file look something like this you know secretor key. Pi uh it will have my own internal key I can have n number of keys here and I'm just importing that variable here and just setting it here contrl enter so that thing is set now let's go to the terminal and install some modules so you're going to install Lang chain module that's number one and the second module you are going to install is called pip install open AI once you have installed those modules let's now import few important things from Lang chain uh we are going to import the llm called open a now we are using open a because open AI I know it cost some money but it is the best one uh if you want some other ones then just just hit Tab and it will just show you hugging phas whatever the the other type of whatever other llms that it has available it will show you all of that we are right now happy with open Ai and then I will create my open model it has a variable called temperature now what temperature means is how creative you want your model to be so if the temperature is set to let's say zero it means it is very safe it is not taking any bets but if it is one it will take risk it might generate wrong output but it is very creative at the same time I tend to set it to 6.7 things like that and now in that llm you can pass any question so let's say I want to open a restaurant for Indian food and I want some fancy name for it I'm not able to come up with that product name idea or restaurant name idea and let's see what this guy does and I typed in the same question in uh here also see I want to open a restaurant for Mexican food it told me this uh if you say Indian food it will tell you something else so we are using essentially the same concept here okay so here uh it says okay Maharaja Palace Cuisine uh if you say Italian food see the name sounds real as if it's an Italian restaurant so we have imported that open class which created an llm and in the llm we are just passing a simple text now I don't want to keep on changing this same string so I will now go ahead and create something called a prompt template so from Lang chain. prompts you can import a prompt template and in that prompt template you can pass some variables such as input variable what will be the input variable by the way variables it will be a Cuisine and then the template that you want looks something like this so I'll just copy paste here and what we're doing is just changing that Italian Etc with that kuin variable and this template is called prompt template and let's say this is for a restaurant name that's why I'm saying name here uh and once that template is created you can just say prompt template name. format and in that format you can pass cuisine as let's say Mexican and see I want to open a restaurant for Mexican food if you say Italian it will say Italian food this is more like a python string formatting you would be wondering why you don't use Python string formatting well let me just show you that using something called chain so we going to use this concept of chain in Lang chain and it is one of the most important objects in in in Lang chain framework you can figure out from the name of the framework itself uh and we are importing llm chain and llm chain is essentially a very simple object where you are saying my llm is this whatever you created here okay that is my llm and my prompt is this prompt template and this is my chain and in the chain you can say chain. run let's say I want to open a American restaurant see the All American Grill and bar so now here I don't have to pass the whole sentence I want to open a restaurant for is that I just pass the cuisine the variable and it will just work every time Mexican see so internally it is calling openi API uh and we made that connection via this module here so if you are using hugging phas then you'll have to do the hugging facee setup and it will call hugging phas okay here we are kind of paying that cost but by the way uh when you created that open API account you got $5 free credit so you should be okay $5 is more than enough for initial learning and exploration and after that if you like it you can go ahead and pay money so that is the simple chain that we got here now let's look at something called a sequential chain uh so let me just explain the concept first so far what we did is we had this chain for generating restaurant name and it was generating it but let's say for that restaurant you want to generate a manual item food manual items so you can have this second component or a second chain where you pass restaurant name as an input and it should give you the menu items that you should include in that restaurant so if it is Indian restaurant it will say p Mangus Etc if it is Mexican it will say quadia burrito things like that this thing is called Simple sequential chain and let's uh code that up but just to clarify the idea here you have one input and one output and you can have intermediate steps where the input of the second step is the output of the first step it is as easy as that so here uh once again I'm generating everything from scratch I have generated the name chain the same way that we did before this is the exit copy paste of the previous code so nothing fancy here and then we are going to create another chain and I'm just copy pasting just to save your time where the input is restaurant name and we are saying suggest me some food menu items for restaurant that so it is like saying this see uh you you are saying I want to open a restaurant for Indian food suggest a fancy name for this only one name please so let's say you talk to CH GPT and CH GP generated this now you are saying that generate food menu items for seon spice and then return it as a comma separated list see this this is what you want you you want to generate all this list and you have now two chains which we are going to do control enter and execute this code and now from Lang chain from Lang chain dot chains it will show you all kind of chains you're going to import a simple sequential chain okay and that simple sequential chain will contain these individual chains that we created and by the way the order matters here so this is a restaurant name chain and then you have a food item chain and that's it and now you will say chain. run let's say you want to generate it for Indian food you're getting a response here and you print a response sometimes it takes time so you have to wait for a few seconds but it will generate the manual items see Lamb kma vegetable CH FR Sak paner yummy huh you're probably getting a water in your mouth uh let's do Mexican for all those Mexican food lovers so while this chain looks good it is generating those food menu items uh I'm not getting the restaurant name as such because if Mexican food it is saying all these item but what is the restaurant name that was that intermediate step here that was the intermediate step but in the output in the simple sequential chain it gives you just one output but I want the rest name and the menual items both for that we have to use a different chain called sequential chain and this sequential chain can have multiple input multiple outputs so I can just say okay give me a name for Indian restaurant which is wagan and then in the output I can say give me restaurant name and items both as an output all right so let's try something like this here I think this whole code kind of Remains the Same I'm just going to add one or two extra things here so see this is my first chain and the extra thing that I have added is the output key so the output of the first uh chain is the restaurant name and the second chain looks something like this where the output key is manual items okay and now let's create the simple sequence and chain so I'm going to say from Lang chain. chains import sequential chain we already used simple sequential chain this is a sequential chain which is little kind of generic so sequential chain will take what kind of parameters will it take well first of all it will say chains and these are the two chains I have and then my input variables you can specify all the input variables so input variables I'm not including that veon Etc let let's keep things simple all I want is two output remaining things are same okay so in the output variables I should say that I want a restaurant name and the manual items as my output variable and let's call this a chain and by the way when you run this chain you can't just say Mexican because you might have multiple input variables that's why you need to give a dictionary you will say Cuisine is let's say Arabic uh run not supported okay run is not yes so run is not supported you have to just call it just like that no function just chain and then bracket just give the argument see Hummers with P bread falafal and the name of the restaurant is the Arabian Bistro so it's giving in fact it's giving input as well so it's giving input and both the outputs whatever code we have written so far we will use that code and create a streamlit based application for restaurant name generator I'm in my C code directory I have created this empty folder called restorant name generator you see there are no files here and I'm going to Now launch Pam uh which is a free uh python code editor and in this you can select open and you can open that particular folder so I will go to my C code directory and in that I will locate Resturant name generator hit okay and it will create a I think empty main.py file and I can just remove the content here and I'm going to just import of streamlit Library first now if you don't know about streamlit it is a library that allows data scientist to build POC application proof of concept application simple applications very very quickly you don't have to use front end Frameworks such as reactjs Etc this Library will allow you to do all of this things very very fast so let me just show you so you can um just create a simple application with with a title restaurant name generator and by the way you have to do pip install pip install streamlit before you start using it otherwise you'll get an error so make sure you have run that uh and this is the simple app with one title now I can go to terminal and I can just say streamlet run Main pi and it is going to open up an application in my browser see simple application with this particular title now I can create uh the Picker where you can pick the cuisine and for that I'll use the sidebar so in stream LD there is something called sidebar where you can create a select box and give a name to that box so you will say pick a Cuisine and give all the options that you want in that particular drop down so I will just put bunch of cuisin indan American Mexican and so on and then here you will say if or let me just show you how this looks so just say hit contrl s save and click on rerun you can click go here and rerun or just press R key and it will show you see you get this kind of nice picker uh and if someone picks any entry let's say someone picks Mexican what's going to happen is that call is going to return that value in a variable which we will store in this Cuisine okay so I I will just store it in this particular variable and you can say if Cuisine then do something okay what do I want to do I want to generate the restaurant fancy restaurant name and list of manual items here for that let me just write a a dummy code here so I will just call that function let's say get restaurant name and items where you supply the cuisine as input and it returns let's say restorant name it is always a good idea to write this kind of stub function or empty function so that you can check your wiring and then you can write the actual code in that function so let's say my restorant name is Cur delight and my menu items is whatever items you like okay so if if Cuisine then get those things as a response and then from the response let's say the restaurant name I can show it here on the right hand side see here somewhere below this header I want to show it and I will use um maybe let's say st. header I will use st. header as a UI control so let's do this s3. [Music] header and when I get the menual items so let me get the menual items here so the menu items are going to be this uh Manu items here and whenever you have comma separator string uh you can call this split function and specify the separator which is comma here and this should return you a list and once you get that list you can iterate over that list and you can write those items here uh um I can just say item and maybe I can put some character just to indicate this is an item you can also uh write like kind of like a header where you'll say okay these are manual items menu items okay hit save this code is ready you go back to your UI rerun and see you're getting the restaurant name in the menual items now when you change this selection it's not going to change because obviously we are returning the hardcoded response and the next step is to write that code which we wrote in our Jupiter notebook and put that code here okay and since I like to modularize my code I'm going to create a new python file let's call it Lang chain Helper and in this file I will copy paste this function and here you can import that module and you just simply call that here see now let's focus on Lang chain helper so what do we need to do here well the same thing that we did in our notebook so I'm just going to copy paste some code from my notebook here okay we don't need to go over it because we have already uh return that code I will also create create a file for my secret key so I will call it secret key and in that secret key file I will place my open a secret key now I'm not going to show you my key because of course it's private thing but you will type whatever key you got uh remember you got $5 credit so you can do a lot of things with $5 is more than enough uh so put that uh that thing here and then you are using that key see you importing that variable from that python file here directly okay so my key is ready what else do I need to do well again copy paste business folks copy paste is a boon for any programmer or a data scientist so we just copy pasted the code for the sequential chain that we wrote in our notebook see here we create a restaurant name chain here we create manual items chain and we just return this response folks this is this is so straightforward okay and I have this habit of creating this function main function just so that I can taste it so I will say if name isore meain uh then print generate Resturant name let's say Italian okay and now what I'll do is I'll pause the video and put my real secret key here all right my secret key I have placed it uh and now I can run it and see what happens so we are generating Italian food restaurant name in the manual items perfect so the restaurant name is La do V whatever and manual items are margara Piza alfredo lasagna and all that one thing I'm noticing here is I see some extra slen characters here so maybe we need to remove them so the way to do that would be let's go back to our streamlit code and instead of just saying respons restaurant name we can call this St function that will remove the leading and trailing white spaces including those slash and characters and you can use that same thing here as well before calling split so hit contrl s save let's go back rerun see Italian food this is my restaurant name manual item you can change it to Mexican change it to whatever just play with it and folks The Art of Getting skillful at coding is to practice just by watching this video you're not going to learn it so make sure you're practicing while watching this video all right our streamlit application is ready as a next step we are going to look into something called agents which is a very powerful Concept in Lang chain and by the way all the code that we are writing check video description we're going to give you all of that code agents is a very powerful Concept in Lang chain what happens when you type this in jat GPD when you say give me two flights options from New York to Delhi on a given date obviously it won't be able to answer because it has knowledge till September 2021 but if you have chat GPD Plus subscription there is this thing called plugins and I have installed those plugins especially the xedia plugin xedia is a website which helps you find tickets and when I give the same question now with the plug-in enable magically it will start working so it will go to xedia plug-in try to pull the information on the flights for a given date and given source and destination and then it will start typing those two options see number one option is only I think $500 ticket $518 ticket which is a pretty good deal by the way I should I think I should book it for my next India and there is the second option and it will give you a link where you can go and book those tickets on xedia so what exactly happened when we enable uh this plugin let's try to understand that so when you think about llm many people think that it is just a knowledge engine it has knowledge and it just Tred to give answer based on that knowledge but the knowledge is only limited till September 2021 the thing that we miss out is it has a reasoning component so it is a reasoning engine too and using that reasoning engine it can figure out that when someone types this kind of question see when as a human when we look at this questions what do we think let's say if we have to go to Wikipedia or not Wikipedia xedia and if you have to type uh convert this question let's say if your friend asks you this question and let's say you are that reasoning engine you go to xedia and in the source you will put New York destination you will put Delhi date you will put 1st August how can you do that because you have that reasoning engine in your brain similarly llm has a reasoning engine using which from that sentence it will figure out source is this destination is this that and it will call the Xperia plug-in and that will return the response back let's look at some other question when was Elon mus born what is his age now in 2023 now maybe this can be answered by the llm knowledge but let's say you are asking some question which is related to an event which happened in 2022 now this guy doesn't have knowledge after September 2021 but once again it has a reasoning capability so it will say okay in order to answer that question first I need to find out when was Elon Musk born for that it can use things like Wikipedia so agents essentially do this thing agents will have tools and using that tool it will try to fetch the answer silon mus was born in 1971 and then there could be another tool which will tell you 2023 minus 1971 how much is that so there there is a math tool uh that it can use to compute that and it will in the end say Elon Musk is 52y old so this is what agents are agents will connect with external tools it will use llm reasoning capabilities to perform a given task let's look at a different question how much was US GDP in 2022 plus five I'm doing just like a it's a it's a silly operation no one cares uh but US GDP in 2022 llm doesn't know because it knowledge is still 2021 so it will go to Google it will find that answer and then it will use math tool to do plus five all these tools like Google Search tool math tool and Wikipedia tool are available as part of Lang chain and you can configure your agent so your agent is nothing but using all these tools and lm's reasoning capability to to perform a given task that is your agent and this agent can be used in our jupyter notebook so that's what I'm going to show you next so let's first import couple of uh important modules and classes and once I have imported them I will will create tools so I will say load tools and I will give list of tools now if you do Google search On Tools here so let's say if you do Google search L chain agent load tools you'll come here you will see list of tools see I have Wikipedia as a tool I have twio I have all these tools that I can use so we're going to use Wikipedia tool here is called Wikipedia Wikipedia and the math tool is called llm math and here you need to provide the llm variable is the one which we created above somewhere here see this is the variable okay so this thing is called tools and then you can create an agent using this initialize agent method okay so initialization method will take two tools it will take llm and it will take agent and in the agent I will give this zero now see hold on zero short react uh description react means uh thought and action so when we are reasoning we first have a thought then we figure out where to go and we take an action so it mimics that particular concept here I will call this an agent and and then I will ask the question agent. run when was Elon mus born and what is his AG in 2023 so let's see what this gives us see perfect it says 52 year old in 2023 if you want to go uh step by step in the reasoning process you can say verbos is equal to True uh and it will tell you by the way verbos is equal to True is the variable that you can use here in any function to kind of figure out the internal steps that it is taking so here the first step when it Encounters this question it knows actually that it has to go to Wikipedia to get the birth date of Elon Musk so it went to Elon Musk Wikipedia page and which will have this particular date here and then um I think it uses the Matt tool sometimes I don't know it should have used the mat tool was it doesn't show but if you rerun it again okay I don't know why this is not working but previously I was seeing that let me show you a previous snapshot that I have um here it say okay it went to Wikipedia for Elon mus birth dat and then it use C action as a calculator so there it is using the llm math tool and it is just uh calculating the final answer it is saying it is 52 year old let's try a different option so this time we are going to use U Sur API so if you don't know about Sur API it is Google search API whatever you do uh in Google and what results uh it gives you if you want to access those results programmatically you can use this particular API you can log in using your Gmail account I have already logged in and when you go to dashboard it will give you this API key so this is similar to our open AI API key it will be a big big string I have stored that API key into my private file that secret key file that I have and I'm going to initialize some environment variable so for Ser API you need to initialize this variable and this is the key which I got from there okay so you can just copy paste this key if you want to keep things simple I don't want to show that key publicly here that's why I I have this thing here and once I have this thing the next steps are kind of similar so I can just copy paste pretty much everything here and I will just say Okay initialize the agent sir PPA and llm math are the two tools I'm going to use and in my agent I will say agent. run and what was the US GDP in 2022 and plus + 5 okay and while it is executing it let's do Google here so when you do Google US GDP 2022 it will tell you 25.46 so this Sur API this API will do Google search and it will tell you the answer that it is this so check this so first it is searching US GDP this and then it is adding F number to this and it is giving you uh this particular result we'll talk more about agents in our future videos uh one thing I noticed is agents are not perfect sometimes they give stupid answer this whole thing is evolving so in the future it will get better but for agent this is what we have and now we will talk about memory when you look at any chatboard application such as chat GPT you will notice that it remembers the past conversation exchange here I asked who won the first Cricket World Cup then a it's totally irrelevant conversation what is 5 + 5 then I'm asking who was the captain of the winning team now see here I did not say which match which game cricket football Etc but it remembers that I'm talking about cricket and it is giving me a relevant answer same thing happens with human conversation we start a topic then we keep on saying things but we remember what the topic is about if you look at the llm Chain by default These Chains do not have memory they are stateless and if you look at the available methods that this chain has you'll find that it has an element called memory here see memory uh and if you check the memory so here if you try to print the memory see you don't get anything uh because the object is set to none now if you want you can attach memory to it so for memory you have to create an additional object and attach it so that it remembers all these conversations this is useful especially if you're building a chat boat let's say for your customer uh care Department uh in that many times they need to save the transcripts of those conversations for legal and compliance reasons so here uh I'm going to import uh an object called conversational buffer memory which is a very common type of memory in Lang CH module and create an object of that class so I will just say this is a conversation buffer memory and then the same chain uh I will print here but I will just pass memory as an additional argument okay and then I'll run the same chain one more time uh for a different question now when you look at chain. memory see there is a memory attached to it we explicitly attach this conversion memory to it and if you look at the buffer and if you just print it for nice alignment Etc see human then AI this human and AI this now this looks good you can save this into your database as a saved transcript of your customer service center conversation uh but one problem with this particular object which is conversation buffer memories that it will keep on growing endlessly so let's say you have 100 conversational exchanges and by that what I mean is one question answer pair so one question answer pair is one conversational exchange this is second so in total this is two conversional exchange so here if you have 100 conversational exchange what's going to happen is next time when you ask a question to open AI when you say chain run it is going to send all this past history to open a and open a charges you per token so this is one token second token third four and so on for th000 token they charge like2 something based on on the model so your cost is going to go up so if you want to save the cost and kind of do things in an optimized way uh you need to restrict this buffer size you can say just remember last five conversational exchanges okay and this thing can be done using something called converation chain so open AI Lang chain provides this conversation chain which is just very simple object so let me just create that conversation chain where you can just pass llm is equal to open AI temperature is equal to 0.7 and I'll just call it convo and let's check the default prompt that is associated with it see default promp is this uh let me just check the template let me just print the template associated with this this is the default template that comes and it says that the following is a friendly conversation between human and Ai and there is history and there is input so if you look at this uh conversation window here there is a history this is a history and the next question you're going to type is the input so input history same way input history okay now when you ask bunch of questions through this so I'm just going to copy paste uh those questions here uh what is 5 + 5 and then who was the captain of the winning team now when you do con. memory it won't be empty because by default conversation chain has this memory associated with it so by default this conversation chain object comes with inbuilt conversation buffer memory and if you print the buffer you will see the entire transcript of our conversation now while this looks good once again think about the open AI token cost if this keeps on building the buffer endlessly then you might have 5,000 tokens in one conversation and when you make a next call like this it's going to actually send entire history entire conversation to open a and that will increase your bill on the API call to tackle that problem maybe what you can do is you can say just send only last 10 or 20 conversational exchanges because that's what I care about that might be enough based on the use case that you're dealing with uh and for that there is an object call from Lang chain. memory import there is a conversational bu for window memory you are restricting the window you are saying let's say my window and K is the parameter you're saying just remember only last one conversational exchange which is one question answer pair okay so let's try this out and let's see uh how this goes so I'm going to copy paste some code here created a same conversational chain object asking my first question asking my second question now when I ask this second question here what is 5 + 5 it remembers the previous exchange but when I I asked the third question here it remembers only what is 5 + 5 it doesn't remember this it's like a shortterm memory lost like momento or gz movie it just forgot what happened here so when when I ask this question it will say I'm sorry I don't know because it doesn't know which game you're talking about which particular match you're talking about okay so here I know it's probably not the best example uh but the idea is I wanted to demonstrate this K parameter here and Bas on this use case uh you might see benefit in using conversational buffer window memory that is all we had in terms of Lang chain fundamentals Now using these fundamentals which you just learned now we are going to build build two end to end projects first project is in finance domain second project is in retail domain I previously published these two projects on YouTube and I'm going to use the same two videos because these are very high quality end to endend projects Below in the video description the link for the code is also available so let's start with the first project now today we will build an end to end llm project that covers a real life industry use case of equity research analysis we will build a news research tool where you can give a bunch of news article URLs and then when you ask a question it will retrieve the answer based on those news articles in terms of Technology we have used Lang chain open Ai and streamlet to make this project more interesting we have added some fun storytelling as well so let's take a look at that story first what if Rocky lived in the Chad GPT era how would he invest all his money would he use Chad GPT to find best investments no way he would hire someone for that Rocky B's recruitment team got Peter pande the equity research analyst Peter read lendy stock market articles for his research but Rocky by did not like it Rocky said Peter promised to create a chat boat like chat GPT for his investment Rocky B liked Peter's grit and he said fasten your seat melt so get ready folks we are going to create a chatbot for Rocky by perhaps the rocky boat equity research analysts such as Peter Pand in our Rocky by story do exist in real life let me give an example of a mutual fund you might know about about all these mutual funds where you can invest your money so all these three yellow color people are the Common People Like Us who are investing their money in the mutual fund and mutual fund will eventually invest in individual stocks now they need to pick right amount of stocks for which they might have a team of research analyst and the job of this team is to provide a research on these companies let's say tataa Motors Reliance how these companies are doing or what are going to be their profits next year how is their management is this a good stock to buy you know they do all this research and in this uh research team every individual person might have couple of stock let's say Peter P is working for HDFC mutual fund he might be looking at Tata Motors and Reliance and his job is to do research on these stocks okay so daily he comes to his job and he will read a bunch of Articles from money control Economic Times or maybe he has access to premium product such as Bloomberg terminal and he will do all his research based on the news articles the earning report the quarterly Financial reports and so on now you can understand reading news articles from these various websites is tedious task there are so many articles so much information to consume see here I'm showing a pnl of tataa Motors why don't we build a tool which looks like this where you can put bunch of news article on left hand side and I'm showing just three you can have n number of Articles and then when you ask a question okay so see I'm showing all these articles and these are like different articles on moneycontrol.com and when you post this question it will retrieve the answer 6.55 to 8.1 lakh that was the answer and it pulled that from this particular article see and the article link is in the below okay and you can also say okay give give me a summary I mean it's not it doesn't have to be the number the answer doesn't have to be only one number it can also summarize the entire article okay I know about all this because when I was working with Bloomberg for 12 years uh in Bloomberg terminal we used to get research reports from Jeff open Hammer all these different companies and we would process that data and show that data on the terminal all right so I hope you have some understanding of the industry use case this is a real industry use case this tool can be used by companies such as Jeffrey's open Hammer all right folks so this is not some toy toy project let's think about technical architecture now we need to go back to basics in order to build the technical architecture whenever you talk about building any llm app the first thing that comes to your mind is can I use chat GPD for this because chat GPD is free well actually you can you type your question uh in chat GB and you say answer this question based on below article do not make things up and then from that News website you copy paste the article here and what happens is jet GPT has a capability see EPS is 8.35 it can pull the answer from that given text so the question is then why do I need to build this tool why can't I use chat GPT for this purpose apparently there are three issues with this approach number one is copy pasting articles is tedious equity research analyst are busy folks they don't have time to you know go to website copy paste and then then uh get the answer they also need an agregate knowledge base because when they're asking question they don't know where the answer might be they might have this question how many uh let's say Tata Nano Tata motor sold in last quarter now the answer might be in any article so how do they know which article to pull and also some answers might be spread over three or four different articles okay so they need some kind of aggregate knowledge base and chat GPT can't give that and third issue is Chad GPT is word limit in chat GPD you can't copy paste a huge article it has a limit on number of words you can supply so we need to build some kind of tool where it can go to the news website which our equity research annalist is putting Trust on and it pulls all those articles into some kind of knowledge base so so here the database that I'm showing here is some kind of knowledge base and you can build a chat GPT like chatboard which can pull data from that knowledge base now let's think about this particular article on Nvidia let's say I have this particular question okay what was nvidia's operating margin compared to other companies in semiconductor industry and give me the answer based on following article and when I give that entire article it will give me the answer in a perfectly fine expected manner but let's think about it we are building a tool here we are not using chat GPT so obviously behind the scene we will be calling open AI API and whenever you call a open API there is a cost associated with it per thousand tokens or you know if you want to think about tokens in a simple Layman language you can say okay word maybe okay so amount of text that you supply to AI there will be cost associated with it so if you supply more tax there is more cost but read the question carefully folks this is very interesting the answer of this question is actually in the first paragraph we don't have to supply second paragraph it is not necessary because see 17.37% that's the answer and therefore you don't need to supply the second paragraph so is there a way we can smartly figure out that okay for this question I need to only give this much chunk if you do that you will save a lot of money on your open AI Bill okay so just think about this article as two different paragraphs and based on the question you can figure out which paragraph to supply in your prompt thinking about this in a generic way you might have bunch of Articles let's say on Nvidia and when you are asking a question what's the price of h00 GPU you want to figure out a relevant chunks so let's say the relevant chunks where h00 GPU priz is mentioned are chunk 4 and chunk two in this case when you are building a prompt you don't need to give all the chunks one to n into your prompt you can just give chunk two and chunk four chunk is just a block of text which is relevant where the answer might be present for your given question okay and when you do that it will give you the fin answer so the question now comes is how do I find relevant chunks you can't use direct keyword search I can't say Okay h100 GPU is like contrl f try to look into all the chunks and wherever h100 gpus is pris give me those chunks okay h100 GPU is probably simple example but look at this example when I go to Google and say calories in apple versus revenue of Apple it knows that the first one is a fruit and the second one is a company how does it know that well it uses a concept of semantic search it looks at the context you know we as a human when I say calorie I I kind of figure out it's a fruit and revenue is of Apple is company similarly in any NLP application if you're using semantic search it can figure out based on the context what is the meaning of this word apple is it a fruit or is it a company and we use something called word embedding and or sentence embedding and a vector database for this I have given a separate video for this so because this explanation might take more time so I don't want to uh spend time explaining all these Concepts if you know this already then fine you can move ahead otherwise the link of this video is in description below so you can pause the video and watch that one first but let's say just for Simplicity um embeddings and Vector databases allow you to figure out so embeddings will allow you to figure figure out a relevant chunk and Vector databases will allow you to kind of perform a faster search on that database and then you can give your prompt to open a and get the answer so now thinking about the technical architecture the first component will be some kind of document loader where you can get all your news articles and load it into that object and then the second one will be splitting that into multiple chunks and then storing that into a vector database so that when you have a given question let's say I have a given question what is the price of h00 GPU you can go to Vector database and retrieve relo and chunk which is chunk two and chunk four okay so Vector database allows you to perform that faster search because this Vector database might have millions and millions of Records okay so the way Vector databases are designed they help you do a faster search and once you have that question you give it to your prompt and you get your your answer in terms of Lang chain uh we will be using all these classes that I have shown in the orange color uh and we will be building our application now for a short term phase one we are building this particular tool in streamlet when you are doing this project in the industry let's say you working as a data scientist for Jeff you're not going to build the whole project in one go you will first build POC proof of concept where you will build this kind of tool in streamlet you know left side number of articles URLs and the right side you put a question gives answer so that way you get a confidence that this approach works and once we are happy with the result of this tool for long term the architecture may look something like this you need to First build a database inje system it will have two different system one first one is database inje system where you go to your trustworthy news article website you write a web scrapper and you have it implemented either in Native python or tool like bright data and then you run that on some kind of crown job schedule let's say this Crown job runs every 2 hours or every 1 hour it will pull the data and it will convert that text into embedding vectors using open a or llama or B whatever embedding uh you want to use then that goes into Vector database and for Vector database we can use pine code mil chroma these are like popular ones today we can use any of these Solutions and that will be your database injection system the second component will be chatboard where in react or some kind of UI framework you will build chatboard similar to chat GPD a person types in a question question gets converted into embedding once again open AI or llama whatever embedding you want to use and then from Vector database you pull relevant chunk so this green and orange are relevant chunks which matches with the question what was Q3 2023 EPS for T Motors and then based on those chunks you form your prompt you give it to a your llm and the answer uh you put it back into your uh UI for the chat board so again this is the overall architecture that you'll be working with uh remember that when you are working in Industry as a NLP engineer or data scientist you first do brainstorming with your team you come up with this kind of nice technical architecture and then you start uh doing some coding okay you don't want to go into a wrong direction all right so in the next section we'll be talking about tax loaders before we talk about tax loaders make sure you have watched this Lang chain crash course so you have a basic overview of Lang chain Library assuming that you have watched this course the next step for you will be to install Lang Chain by running pip install Lang chain this is the command that you run um so let me just show you so you will run pip install Lang chain to install Lang chain library and once that is installed you can launch a jupyter notebook and there you can import the class so you can say from Lang chain. document loaders import text loader okay so I have imported a simple tax loader there are multiple types of loaders that Lang chain offers and we will look into them uh one by one so text loader allows you to load data from a text file here I have a nvidia's news in one text file which is called nvda newor 1.txt so I will just load that here and and kind of show you how this thing works so nvda news I think 1.txt and I will call it loader and then you will do loader. load and then it returns a data object and if you print a data object looks something like this it has all the news content inside it and if you look at it it is actually an array okay an array zeroth element is that document which has page content as one one of its element so let me just show you here in a separate set so here if you do page content uh page content see it shows you the entire text content that it has the other element that this class has is metadata so metadata is your the name of your text file now if you Google let's say Lang chain text loader so let me do Lang chain actually Lang chain documentation and if you go to the documentation here you will find let's see you will find the documentation for various loader classes that you have now it is sometime hard to navigate this uh and it it can change based on at what time you're looking at this but see document loaders python guide will give you all these loaders so this is a text loader the second one they have is a CSV loader so let me talk about CSV loader real quick so I'll just copy paste this thing here and I have a CSV loader class here CSV loader obviously I need to pass a CSV file and luckily I have this movies. CSV file which has around nine records you can see nine records where you have movie title industry the revenue and so on I will provide all these files in video description below so make sure you check it code all the files everything will be provided to you here I will say movies. CSV that will give you a loader and loader. loadad that will give you data okay loader. load and if you look at length of data you will find nine records because in the CSV file we had nine records and if you look at the very first record once again you're getting that document class okay you can say type here and you will see the the document class from Lang chain library and that class has two elements right page content which is a page content uh so let's see what is what is inside page content so page content is entire record okay entire record in your CSV file which has movie ID title and so on separated by sln and if you look at metadata it has the m. CSV now one may argue that this metadata I want to have maybe movies name or movie ID as kind of the metadata and metadata is something that we will be using in our project so when remember in the preview we saw when you type in any questions and it will not provide you the answer but it will provide you a source link so how does it uh reference back to the source link the answer is it is through this matter data and and we'll look into it but for now here I will say that my source column here so my source column is let's say movie ID or title okay so what are the columns I have so movie ID it has title maybe I can keep it title as a source column so here when I keep it as title what you will see is in the metadata you get kgf2 for the first eord kgf2 right see for the first eord kgf2 second one Doctor Strange Dr Strange and so on so you can view all the records so here is one record and then here is the the metadata see metadata metadata and so on okay now let's talk about the unstructured URL loader because that is something we'll be using in our project in our project we will be going through some news article let's say this particular article on HDFC Bank and we want to load the text content from this article directly into jupyter notebook using some rade Lang chain class and that class is unstructured URL loader so this is how you import it and by the way you need to install couple of libraries before that and the way you install those libraries is by providing this particular command okay in the notebook if you run this it will install all these libraries other thing is you can copy paste this into your git bash or Windows command shell and install it okay it's just a usual thing folks uh you should know how to install librar so these are the all libraries that you need to use and by the way unstructured URL loader uses a library called unstructured so if you do python unstructured Library this is the library that it uses underneath to kind of go to that website look into the Dom object the HTML structure and pull all the information so let's create that class and the arguments is all the URLs that you want to supply and the two URL that I want to supply are basically I'll just Supply two different articles here okay and that will be your loader so my loader is this and our usual step is loader do load you get data as a return value and when you do length of data it will take some time but it will return two because you have two articles and you look at the first article and see again same thing page content and let's see what do we have in metadata see in metadata you have the source URL link so metadata is the source URL link and we will be using this in our news research tool after we load our documents through loader classes in Lang chain next step is to do Tex splitting and we have character TX splitter recursive tax splitter these kind of classes the reason we do this is because any and llm will have a token size limit that's why we need to reduce the big block of tax into smaller chunks so that it is within this particular limit and what may happen because of these classes that we're using in from Lang chain is individual chunks that that we get after we do split might not be very big or it might not be closer to the Token limit which is 4,097 let's say the first chunk is 3,000 second chunk is 1,000 it would make sense if I merge these two so that it is closer to the Limit and it kind of work more efficiently so we have to perform merge step so first you have a huge block of tax you divide things into smaller chunks chks and then you can perform merge so that each individual chunks that you're getting which is in this blue green orange color they're closer to that limit which which could be 497 2,000 depends on the llm that you're using we also want to do some overlapping so that when you are reading this orange paragraph you need some context from the blue paragraph which is which is you know one step ahead so you see part of this blue paragraph goes into orange also so that is chunk overlapping similarly part of this orange paragraph goes into this green chunk also you see this this orange thing at the top that is called overlapping the chunks all of this can be done using some simple apis in Lang chain so let's look at it here I have taken the Wikipedia article of Interstellar movie you might have seen that science fiction movie and we are going to perform Tex splitting on this one now when you think about text splitting let's say I have a limit of 200 tokens that I'm using in my llm how do you split this TX so that each chunk is is of size 200 well the obvious thing that comes to your mind is why don't we use Simple slice operator in Python and kind of divide things that way but when I do that you will notice that it might cut off the words in between see M what is M mad demon demon right so here it is kind of cutting that off and doesn't look that great you at least want to have a complete word so this simple slice operator is not going to work then you'll say oh what's a big deal I might write a for Loop this kind of for Loop where each of the chunks so if you look at these chunks each of these chunks is less than 200 okay you can do that uh but again this writing this kind of for Loops is little tedious and it can have other issues as well Lang chain provides a very simple API so that you don't have to do all this work manually and that API is given through various TX splitter classes so let's try the first simple one so from Lang chain. text spitter you can import let's say character text splitter okay and when you have this character text splitter class it will take separator as an argument separator as in through which character you want to separate things out here let's say we want to separate things out with new line character which is sln which means each line can be one chunk or multiple of those lines because it will do more steps also and my chunk size is let's say 200 as such it is like 4,000 something but just for Simplicity we are saying 200 and chunk overlap I will I will keep it zero just to keep things simple and this is my uh splitter okay and that splitter I can use to split the text so I will say split text and here is my text and what I'm getting as a return is the chunks and let's check the chunks length Okay chunks length is nine and if you look at it see it's like one this is first chunk second third and so on and if you look at the individual chunks length let's say for Chunk in Chunk we will print the length of the chunk you'll notice that while most of the chunks are less than 200 there are some which is more than 200 see the chunk size for these chunks is more than 200 so why did that happen well let's look at some of those chunks so the last two are kind of big so if you look at the last two ones you will notice that see this one is pretty big in that entire chunk there is no slash and it's like a one big or multiple sentences without sln so obviously uh it can exed the size maybe you can change this slash into dot so that you know after every sentence ends yeah it will just take that as one fragment but what if you have a bunch of questions so you might not have dot well you can use space but it no matter what you use you will always face one or the other issue for some cases character text splitter will work but we need something little more advanced with which can split things on multiple separators and maybe we can have some rules that first divide things by 2 sln then one sln then dot then space things like that and this is something that you could do using character text splitter which is of recursive nature and it is called recursive character text splitter okay so in recursive character text splitter the arguments are kind of going to be the same except that you can provide a list of separators see here you provide just one separator here you can provide list of separator so I can say okay my first separator is Ln second one is Ln third one could be dot or less is space and chunk size and chunk overlap I'll just keep it same and I will call this our splitter okay and we will split our text so let's say you are splitting your text you store it in chunks and you look at the length of the chunks okay so total 13 and I will also print the size of the individual chunks here and you can see that majority of them are now or actually all of them are less than 200 so let's understand how it works under the hood so what this will do internally see you'll just make one API call but internally what it is doing is this so first it will try the first separator which is this correct and it will split the things so now we had a big text blob it split that into three chunks see 1 2 3 okay that is what it did and once again if if you want to print the size I can print it here see all three of them are more than 200 size because we are splitting using sln so then uh let's think about the first chunk itself so first chunk is this I'll just call it first split just to kind of keep things simple so this is my first split and if you look at the length of the first split it is 439 when Lang chain detects that see first it will separate things out using SL and then it will check individual chunks so individual chunks are three okay this is 439 so when it says that this is more than the specified size which is 200 it will further split that using the second oper which is sln so here what it will do internally is it will say split this using sln and it will get again three more splits okay and if you look at the size of these three splits let's look at the first split so first split if you look at it or let me just say second split is equal to this and the second split if you look at the length of the first first one it is less than 200 so it is fine second one is 121 it is fine but if you look at this third one it is 210 so it is definitely more than 200 so what it will do is it will then go look at the third separator which is space and then it will separate things out and once it separate things out then it will again merge we looked at the merge step if you remember from here see it will also do merging because each individual chunk size might be too small so it will just kind of merge everything so obviously when you split things apart using pace so let's do that so when I split things apart using SP space obviously each chunk will be very less right this chunk is only three character this is two character one character we can't have chunk which is that small so it will then merge those things as I have shown you in the slide it will merge things such that it is kind of optimized so what it will do is this see let me just print for Chunk in second split print the length of the chunk first two it will keep it as it is the third one it will divide so what is our size 200 so it will say it will create one Chun using size 200 the other one using 10 roughly it could be plus or minus B because you need to keep your word intact you can't break the word apart so that that is a reason when we are using this API see you got 199 and 10 see 210 I mean there is a space character so one character is here and there but you see 105 120 so 106 121 okay so those two are same and then there was 19 9 and 10 so it split this 210 into 2 one you know 199 and 10 so it is like 209 in one space character so total 210 so that is how it is doing the splitting I hope you got some understanding of recursive text splitter this is something that we will be using in our news research tool now that you have some understanding of text splitter let's look into the next step which is Vector databases now for Vector databases there are lot of options Pine con milver chroma but we are not going to use them in our project we will use something called phase uh which is a kind of like a lightweight inmemory Vector database type of thing phase stands for Facebook AI similarity search it is actually a library that allows you to do faster search into set of vectors that you have uh but it can be also used as a vector database if your project is smaller and if your requirements are kind of lightweight requirements okay so you can read about phase but I will give you a very quick understanding so what will happen is once you have set of chunks that you have created using recursive tax splitter for our project we will convert those into embeddings see embedding conversion is a must step we can use either openi embedding hugging face embeddings word to there are so many embeddings out there in the world based on a problem statement we can use any of them and then we will store them into a vector database so if we were using pine con or Milas we would have stored these into that proper Vector databases but for our project we are just going to store them into phase Index this is like an inmemory structure which can uh do a faster search on your vectors so let's say if you have an input question called what is the price of h00 GPU we will again first convert that into a Vector using the same embedding technique and then we will give it to phase index and what phase index will do is let's say these vectors that we have created out of these chunks are let's say I have 1 million Vector phase will efficiently perform a search for a given vector and it will tell you out of those 1 million how many of those vectors are similar okay and this I have explained in detail in this particular video which I'm going to provide a link in a video descript deson so please watch it if you haven't seen that but let me just quickly show you how phase Works uh by using some uh simple code so you have to first uh install these two libraries okay and once these libraries are installed I'm going to import pandas um and I will just increase the pandas data frame column withd I I'll explain why I'm doing that later on but I'm loading a CSV file which has like eight records you know so different eight text and their category they are either Health fashion these type of categories so I'm loading that uh into a data frame here and my data frame shape looks something like this and my data frame looks something like this now I will convert this text okay these eight sentences into vectors and the way I'm going to do it is using the sentence Transformer Library so I will say from sentence Transformer import sentence Transformer okay and for this sentence Transformer I'm going to use a model uh or a Transformer entity called this L all mpet and if you want to read more about this you can just say a hugging face sentence Transformer uh you can do reading and you can kind of figure out how it works but in simple language all they're doing is is converting this text into a vector so how does it do that so I will just say encoder is this and then encoder do encode okay and that encode expects an array of text and array of text is DF do text see DF do text when you give it it gives you that that entire column and that you can store in the vectors and let me just print the vector shape it might take some time by the way if you're running for the first time just have some patience you can see that there are total eight vectors if you see this it's like a two- dimensional array okay so first one is this second one is this third one is this and so on um so meditation and yoga can improve mental health the vector corresponding to that is this one and you see dot dot dot so the total size of one vector is 768 which I'm going to store it in a in a variable so see so vectors if you do Vector do shape and one so this is the size of each vector and we have total such eight vectors I will store this into a parameter called Dimension because that Dimension I'm going to use later on and then I will import the phase Library okay so once phase is imported I will call index flat L2 so this is uh the index that uses a ukan distance or L2 distance okay so that's the index that we are using once again if you want to know more detail you can go to either phase. or go to their GitHub page you can do more reading but as such it is very simple it is just creating similar to database index it is just creating an index that allows you to do faster search later on okay so here I can supply dim actually and that will be my index so I'm creating an index of size 768 here and when you print that index you'll see nothing it just created some empty index now in that empty index I can add some vectors correct so when I added it now my Vector is kind of ready so going back to to that picture again we have total eight vectors right total eight vectors and the size of each Vector the size of this particular areay 768 we are just adding that into phase index now phase index will internally construct some kind of data structure what the data structure is that is out of the scope of this video but some data structure that allows you to do fast similarity search so for a given Vector we can find okay out of these eight Vector which two vectors or which three vectors are similar Okay so so here now uh once I have index I can do index. search and here I want to supply search Vector but what is the search Vector well we don't have search Vector ready so I will give some input search query and the search query is let's say I want to buy a polo dessert okay and that search query we have to of course encoder do encode if you look at that P picture we need to kind of convert this into a vector so that is what I'm doing here when I say encode encode my search query and I get the vector back if you look at the vector shape see it's a simple array 768 but this search Vector expects two dimensional array so I'm going to use numai and convert this Vector into two dimensional array so it's simple folks what I did is something similar to you know I I put let's say Vector into an empty array outside so it was one dimensional array 768 now it become two dimensional and if you print that see Vector was simple one dimensional array but if you look at this S V now it is same Vector but see there are just there is one outer array outside outside that and the reason is this particular function expects that format okay uh okay some argument is missing how many uh similar vectors do you want this is like K nearest neighbor so let's say I want two similar vectors and it gives me these two vectors okay so it returns a tle and the first one is the distances the second one is the index in our original data frame so in our original data frame locate the rows which has index three and two okay so which one is three and two so three and two both are articles related to fashion and you can see that I want to buy a polo t-shirt is kind of similar to Fashion okay so that's what it did uh I can store them in distances and I I can store them in tupple and if you look at I right three and two you can locate that uh using DF do log so if you do DF do location and do 3 and two it will give you those articles and if you want to be kind of like in a programmatic way you can do it similar thing so I of 0 is 3 and two only okay so that's what it is giving you now one thing you might have notic is in this text okay let me print the text here search query so in this text I want to buy Polo t-t see the exit word is not present here see so this is not like a keyword search this is a semantic search which means it is capturing the context or the meaning of this sentence and giving you the similar sentence here if you look at our entire data frame see it has meditation and yoga and all that but it it give you only fashion related articles you can change a sentence so let me say that an apple a day keeps the doctor away okay and I will go here and I will say run all the cells below this particular cells uh okay I think there is some problem okay let me run all the cells here you notice that when I say an apple a day keeps doctor away the search results were related to health once again you will see that in the similar vectors which are these two the exact words are not matching see an apple they keeps the doctor away it is not present in any of these sentences but if as a human you have to think which are the two similar sentences for an apple a doctor keeps a doctor away out of all these eight you would probably give these two because we're talking about health here and it gave me the health related articles okay uh you can try something else which is looking for a place so let's say looking for a place to visit holidays okay and go to sell run all and you notice once again it give you two articles which are related to travel so I'm going to provide all these individual notebooks by the way in the video description below so just check it just think about it uh and you will get an idea so uh this was just a quick demo of phase Library uh this is something that we will be using in our news research tool project let us now discuss the retrieval QA with sources chain once you have stored all your vectors in a vector database the next component will be asking a question and retrieving all the relevant chunks let's say my relevant chunk is chunk number two and chunk number four using these chunks I will form an llm prompt The Prompt will be something like I have s00 GPU what is the price of it give me the answer based on the below text which is Chun 2 and Chun 4 and then llm will give you the answer the benefit of this is you can tackle the problem of the token limit and also save some Bill on your open API calls so now when you think about combining this chunk see here what we did is whatever chunk you get you put all of them in one single prompt now as a result here I got chunk number two and four but actually I I might get more chunks let's say I got four chunks and combined size of this chunks is more than the llm token limit so then then that is the drawback of this method this method is called by the way stuff method so you're getting all the similar looking chunks from your vector database then you're forming a prompt and when you give all these chunks together it may cross that L llm limit token limit so that is a drawback of this method if you know that that chunks will not cross llm token limit then is fine the stuff method will still work it is the simplest of all but the better method especially when the combined chunk size is bigger is map reduce in map reduce method what we do is we make individual llm call per chunk so let's say I have these four similar chunks so for my question let's say what is my h00 GPU price give me the answer based on chunk one then again I'll ask a question what is the price of h00 GPU size give me is the answer based on chunk two chunk three chunk four so you are asking four different questions and each time you pass different context which is chunk one 2 3 and four obviously you get four answers here there is a type of by the way this is fc1 fc2 fc3 and 4 and so on so this is like a filter chunk or or an individual answer and then you make a fifth call and you combine all these answers together and you say to your llm that out of all these four answers just give me the best answer or just combine all these answers together and give me the final answer this way you will teer that that uh token size limit but the drawback here is you are making five llm calls see 1 2 3 4 and five in the previous method you made just one call so that is always a drawback so now let's do some coding and try to understand this thing uh in a little deeper fashion I have imported all these necessary libraries you need to give your open API key here okay if you create a free account they give you like $5 free credit so you can use that and after this account is created you get that key okay we have covered all of that in our Lang chain crash course so here I'll will create an llm object and then I will use un unstructured URL loader this is something folks we have already looked into it so I will not go in the in the detail here I'm just loading two different articles so first article is on Tesla okay Wall Street Rises Tesla whatever and the second article is on tataa motors which is India based automotive company and here we are loading both of these articles into our data loader and then we are using the same recursive text splitter which we have looked into before and creating this individual chunks so we created total 41 individual chunks I mean you can check the individual chunks here see this is the page content okay 0 1 2 3 4 whatever you can you can check it out right like nine and then once that is done you will create open API embedding so how do you do that well see we created this this particular class here so I will say embeddings is equal to open a embeddings and then uh you will use the phase class that we imported here and call a method called from documents now see from documents method in Phase will accept the documents or the chunks that you created here and then it will take another parameter which will be your embedding so here I'm using open API embedding so I'm giving that you can use hugging phas or any other embedding too and the resulting index will be this Vector index which we have and once you have that Vector index so I'm not running this code by the way because I already run this code and I have saved this Vector index into a file okay and the way I saved it is I can use this code see you have a vector index and then Vector index you can write it to a file called Vector index uh pickle file so previously before shooting this tutorial I already ran this code I saved Vector index into a file and let me show you that file on my disk see Vector index. P pickle file so this is sort of like a vector database it is a vector database which I have saved as a pickle file on my disk and now I can load that pickle file by running this code see now I can run this code and my Vector index is loaded into a memory so this Vector index now have knowledge of both of these articles now let's create a retrieval QA with sources chain uh class object the first argument that it expects is llm so wherever we created our llm which is here you know what just to okay I'll just put that in here and the another argument is retriever so retriever is basically how you're planning to retrieve that Vector database so Vector database you can give as an argument here and you can just say as retriever okay this is just a syntax that we're using and this thing we going to call chain okay here I need to save from llm and you will see that it has created this chain and in the chain you will see interesting prompt which might get you very exited which is this use the following portion of long document to see if any text is relevant to the answer of this question uh this is a thing we have discussed before now we are going to ask some sample question so my simple question is what is the price of Thiago icng and if you look at the article uh the Thiago icng price Thiago icng price is between this and that this so you you would want this particular answer uh from our code okay so that is my expectation I will I will enable the debugging in Lang Lang chain so that I can see what going underneath and then you will say chain in the chain the question that you want to ask is this and this is the argument that you give okay so now let's run this code and see what happens so this will kind of show you some internal debugging so when I said what is the price of Thiago icng first it retrieved the similar looking chunks from my Vector database so there are total four chunks 1 2 3 and four okay and the question is same for all of these so see the chunk is the company also said it introduced Diago see my actual answer is read in uh first chunk itself but it will still retrieve four similar looking most similar looking chunks now see this is also similar looking but my answer is not there exactly this is also similar looking and this is also similar looking okay all this code is available in the GitHub by the way you can run it and you can just go through it yourself so that is Step number one which is chunk one and then you combine a question and you ask four question individual questions to your llm so all this questions will go to the llm so my first prompt my first prompt is use the following portion of a long document to see if any of the text is relevant to the answer of this questions right return any relevant text ver ver team and this is the paragraph that you give okay similarly this is the the second question this is the third and this is the fourth question so these are the fourth question so you passed all those four questions to llm so there are four llm calls as a result you will get four answers correct so let's see the four answers now so the first answer is this the Thiago I price so you know this is the final answer but still it will give generate four answers so this is the first answer this is the second answer this this answer doesn't look good but it will still give you some answer this is the third answer and this is the fourth answer okay so fc1 fc2 3 4 these are the four answers that it give now you will combine those four answers in a summary chunk and give one more call to your llm so where is that question see these are the four four answers okay 1 2 3 4 now here is the combined summary what is the price of I Ang CNG so the summary is is content See four answers are combined so G given the exit prompt is given the following whatever summaries uh give me the final answer okay and then in the end it will give you a final answer and the final answer is the tago price is between this and this is the source reference okay so once again it is using this map and reduce method I have given this notebook with a lot of comments so maybe you can read it and get an idea but overall uh now what happened is we looked at all the individual components we started with taex loader splitter phase we talked about retrieval just now now we are going to combine all these pieces together and build our final project our final project is not going to take much time because all the individual pieces are ready we have also understood the fundamentals behind the scen see see learning only API is not important you need to understand the fundamentals how it works underneath then only you can become a great data scientist or anal engineer so far we have cleared all our fundamentals our individual pieces are ready we need to just assemble them and it's not going to take much time I'm super excited to move on to the next section now and this is the argument that you give okay so now let's run this code and see what happens now we are going to use all the components that we have built so far we will assemble them and build our entire project I have this this directory where I'll will be keeping my project in the notebooks folder you will see all the individual notebooks and here outside I will do main project coding right now you're seeing two files requirement. txt andv file if you look at requirement. txt it has all the libraries which we are using so you can run pip install minus r requirement. txt to install all the libraries in the EnV file I have open AP key so you will put your own key here which you have got by you know getting that $5 free credit on open or or if you have a paid account just use that now let me create a main.py file here so I will say main.py and here I'm going to import all the necessary libraries so since the list is pretty long uh I'm just going to copy paste it from somewhere and the very first thing I will do is uh load that open API key now so far we were using os. environment variable but that's a little clumsy way of doing it there is a better way which is using the dot EnV python module so if you look at this particular python module uh you have to install it first of all and then you can just use these two lines and I I'll tell you what what they do actually so here it will take all the Environ M variables from EnV file and it will load them into the environment okay so if you look at uh let's say EnV file EnV file is this so it will set domain environment variable as this root URL as this and so on okay so it's just one call and within one call we have loaded our API key and the API key is not visible in the code so it's kind of like a cleaner way it's standard practice that people are using nowadays uh for loading this now let me just uh write some basic UI so I will say st. title okay our title of the application is news research tool so that's what I'll say here and then I will create the sidebar so if if I to show you the UI of the uh our tool it will have on the left hand side it will have this kind of three URLs URL one Ur So This is URL 1 this is URL 2 this is URL 3 I will have process button below it and the title that I'm putting is here it will be visible here and here I will have on the right hand side the actual question and below that there will be an answer okay so in the sidebar I'm going to say Side by sidebar title news article URLs let's say that's my sidebar uh title and I will have three URLs so I will say for I in range let's say three okay and St do sidebar dot text input so I will take the URL from text input and I will use a format string here I will say URL I + 1 because it starts with zero so I'll say URL 1 URL 2 and URL 3 okay and below that there will be a button so the button will be called process URLs maybe okay and when you click that button that value I will get in process URL clicked and when I say process URL clicked here when I press that button the flow will go here okay let's just run whatever we have so far bar and see what happens okay so the way to run this is stimate run main.py uh when you run that it will show you this kind of uh UI see three URLs there's a news research tool here we'll add the question box but here you can add those URLs and when you hit this button process URL button actually the flow will go to to this if condition okay so let's write some code inside this if condition so here we will use the unstructured URL loaders and we will get all the URLs now here you need the URLs right so where are my URL so let's say I create this URL variable and those URL you can you can build that array here so whenever you enter that URL it will come to this array and that array is passed here okay and that is called loader and you do loader. load okay this should be very apparent to you so this first step is loading data after you load the data you all know next step is splitting the data and for that you will use recurso recursive character text splitter which has two arguments I guess separate and chunk size I'm not worrying about chunk chunk overlap that much although you can play with it so this is my teex splitter and I will say from text splitter split my documents and my documents will go here and I will get the individual chunks okay and those chunks I will then do embedding I'm going a little fast because we have already covered all these things before create embeddings and okay let's say create embeddings okay so these are the embeddings I have and then you will say phase from documents and the first argument is documents and the second argument is embeddings okay uh and save it to phase index and we'll call uh this particular variable let's say this okay and then we are going to save this in memory phase index that we have on this so we'll save it uh in a pickle format and here is how you save it the file path is going to be you can give any file name okay so I'm just going to give plus a phas store open a i Pi okay so that is that is stored here uh and just to show the progress uh when we are processing the URL so let me just show you the UR here when you enter bunch of URL here process URL here like below this news research tool here I want to show up progress bar so progress bar will say Okay loading the data splitting the data things like that and for that we have to use a main placeholder so I will say maincore placeholder is equal to st. Mt so this way you are creating an empty U kind of like UI element and when you are loading the data you can say dot text and you are saying okay data loading started okay so I'll just again copy paste here and I will uh put this kind of uh progress bar before I think split document and also here all right so so far I think my code looks good I can rerun this and see how it goes so I'm going to rerun so you can click on rerun or just Place R button so my code is Rerun now and I'm going to give these three articles here so this is my first article then this one is my second article so first second and the third one is this all three are articles related to data Motors and see it will say data loading started so now it's loading going to these three URLs using unru URL loader loading the data then it is splitting them into, uh token chunks then it is building embeddings using open API calls and then it is using phase to kind of build an index and save it to a disk so when you look at the folder that we have uh you will find this file phase store open a pickle this is like our Vector database so now our Vector database is ready so next step we will enter a question box here okay so let's do that coding now I'm going to say main placeholder do text input and the text input will have question and whatever query you are getting that question you will say if query so when you type in a question hit enter it will come here now what should be the first thing that you you'll be doing here well you will be loading the the vector database so you'll say say okay if the file exists there could be a reason where this particular file you know doesn't exist so you want to make sure you're doing that check if the file exist then you need to read that file so you'll say with open file path read it's a binary file ASF and pickle. load so you load that file and you call it Vector store Vector store okay we are using the Python's like pickle modu you can see here we imported that and then we are creating the retrieval QA chain okay so Vector store Vector store is here uh this QA chain expects llm as an input so I don't think we have created llm so let me just copy paste it here I I have the code ready here so that I can save time on the recording so I have created Creed llm object now my chain looks good and then in chain you will say uh what is the format so you supply a question here okay this particular argument you will supply as true and then you will get your result now result will be a dictionary which will have two elements okay so result will have it will look something like this so it will have um answer and it will also have whatever is the answer so it will have whatever is the answer and it will also have sources which will contain the URLs or whatever it can be an array so result answer contains answer we need to display that right so St dot our main placeholder dot you know what I will just uh use st. header here and and say here is my answer and then St do subheader and that subheader will have result answer okay so let's try this much so far so I'm going to bring that UI here click on rerun and it just rerun and I will ask now what is the price of Thiago icng okay so this is a question based on those three articles uh if you look at that article here I think it is here see I cng's price is 6.55 to 8.1 and see it is giving the answer uh properly I want to also see the source like from which URL it retrieve the article so we can do that real quick here just to save time on recording I'm not going to go too much in details because this is very minor so you are basically going to result and get the sources element from the dictionary because in that dictionary this might not be present Okay this may be present this might not be present that's why I'm calling get uh argument and if it is present then you are creating another subheader and providing sources list now why list because sometimes answer may come from multiple URLs so you need to handle that scenario so let's bring this code here now rerun this and ask the question again see Thiago C pric is this and here is the URL so if you look at this URL uh it will have the answers correct this now you can ask a summarization question as well for example I have this article K ches and I want to just summarize this article this is their recommendation on this talk so I will say can you please summarize this article okay and by the way this is like bold so let me just change this thing I'm just going to call it right because that way it is not bold okay so here you see it it already summarized the article and it also gave the source for that but but this is like little bigger font so I'm just changing it to smaller font but overall folks this this tool is ready now it's going to be very useful to my equity research analyst my peter Panda who is investing on aoki's behalf because now you don't have to read so many articles whatever question you have you can ask it to this news research tool it will not only give you the answer but it gives you the sources reference see this is very important folks and by the way with llm boom many clients are building this kind of tool how do I know it well my own company in ATC Technologies we have some us-based clients for whom we are building uh these llm projects and this is a real life use case that I'm showing you so this is not like a toy project it is based on a real things which are happening in the industry document summarization building chatboard similar to chat GPT on custom data so this is sort of like we build a chat board which can answer your questions on custom data which is my these three URLs so we just buil the basic proof of concept and it is already working a code everything is given in the video description below so please try it out longterm we have discussed this like longterm when you building this project in the industry you will build two components one is Data inje System where you will write some kind of web Scrapper that goes through all these um websites and it retrieves all those articles now for web scripting you can use native python or bright data uh even unstructured URL loader will work but you know it might start not working at some point because websites will detect the scrolling activity and they might block you uh that is a reason why people use tools like bride data which is a proxy network based tool then you will create embeddings it could be open AI hugging phase and store it in a vector database so if I'm doing this in the industry like a big project I will not use phase I will use Vector database I can still use phase for as a library but otherwise I'll use a vector database and once you have data in the vector database you can build UI in react or whatever tool and then call that Vector database to retrieve the similar looking chunks and post the answer back to chatboard data data data data lights me I can't Avo you in today's video we are going to build end to endend llm project where we are going to use all these Technologies at Le te is a store that sells t-shirts their data is stored in a mySQL database we will build a tool similar to chat GPD where you can ask a question in natural human language it will convert that question into SQL query and execute it on our database you will get a feeling as if you are talking to a database in a plain English language it's going to be a very interesting project let us discuss project requirements our atck te's t-shirt store sells four Brands mainly van Hussein leis Nike and Adidas and the mySQL database has first table which is called t-shirts where we maintain the inventory count so basically Levy's black color small size t-shirt have 15 uh stock quantity left okay so 1564 these are the stock quantities and this price is price per unit so one leis black Smalls size t-shirt will cost you $19 the second table I have is discounts so for example t-shirt ID one which is leev black small T-shirt has 10% discount in real life the database will have so many different tables to make things simple for learning I'm just going to use two tables the t-shirt store manager is Tony Sharma whenever he has questions related to stock quantity discounts and so on he uses a software which is built on top of this mySQL database if you look at retail domain overall they will have these softwares where you can use various UI options on the software to get answers of your questions and Tony is fine using this software but many times what happens is he has custom questions little complex questions and the software can't figure it out so then he will have to download the data in Excel do certain things manually whenever he's busy he goes to Loki who is a data analyst working for this company and Loki knows SQL so let's say if Tony asked this question that how many white color Nike T-shirts do we have in stock and he will just simply run the SQL query on that database and get the answer back to Tony Sharma but Loki is busy as well he's busy building power be dashboards and he doesn't have uh too much time for these ad hoc queries also L is the only data analyst working for this company and Tony sometimes have issues where you know Loki is out on leave and he's not available and then he has to do all this work manually because Tony himself doesn't know SQL so then he goes to uh data scientist who is working for the same company and you might guess what is the name of that data scientist well Peter P who looks somewhat like me and he says hey Peter buddy we are living in chat GPT era llm Lang chain all these cool Frameworks have come up why don't you build a tool similar to chat GPT where I can ask a question in a human language and it somehow converts that to a SQL query executes it on a database and gets me the answer that the answer is 3165 Peter likes this thought and he agrees to build this particular tool so let's look at the technical architecture of this tool whenever you have a question you need to convert that to a SQL query using some llm now we are going to use Google Palm here which will do this conversion and we will use Google pal from Lang chain framework within Lang chain framework you can use Google palm and other type of llms we will use a SQL database chain class within Lang chain framework this will work okay for simple queries but as the queries get little complex out of the box Google Palm model will fail sometimes it will give errors and we need to do some special handling we will use a concept of few short learning here few short learning means you need to prepare the training data set where you have a sample question and a corresponding SQL query here you will list down all those queries where out of the box box Google model is failing and you can prepare these queries with the help of your data analyst Mr lokal and you prepared this data set it is called few short learning because you don't need to prepare like thousand samples you know you can have some few samples here and then you will convert this training data set into embedding vectors if you don't have any idea on what is word embedding sentence embedding go to YouTube search for code Basics embedding or code Basics word embedding you'll find couple of videos where I have provided very simple intuitive explanation for these embeddings we will use hugging face Library once embeddings are created we will store them into a vector database when you think about Vector database there are a couple of options that you have pine cone MERS chrom rdb face Etc we are going to use chrom rdb it is open source and it will work perfectly okay for our project once the vector database is ready we will pair it up with uh Google Palm llm we'll use few short prom template to create the SQL database chain and the last piece will be building a UI in streamlet we will write just few lines of code five or six lines of code and our UI will be ready to continue further on this project obviously you need to have Lang chain Basics clear for which I have this particular video where I have covered all the basics in this one single video so make sure uh you have either watched it or you already know the Lang chain fundamentals so just the basics you also need to know what is Vector database in this six minute video I have given a very simple explanation of what is Vector database so if you have not seen it please uh see that now let's do a review of Google Palm there are three popular options when you talk about building llm application open AI gp4 model which is best in the market but it is paid the other two unpaid are meta's Alama and Google um I could have used meta's llama for this project but you have to download that llama model locally or less in your Google collab and it is very heavy like sometimes it's the the size is in gigabytes and it's kind of little bit hard to set up whereas Google spam is very easy to set up it works similar to open a API where you just make a query to their Google server and the Beautiful Thing here is it is all free okay so we going to use that as a next step we are going to set up API key for Google pal I have opened makers suit. google.com website where you can login using your Google account you need to go to gate API key and you can create API key in your existing Google Cloud project and if you don't have that just click on create API key in a new project so here I will use just any project and create an API key now this API key short of like a password so make sure you don't share it with others I'm showing you this apiq right now but I'm going to delete it after I use it in my project so I'll copy it save it at a safe place so that I can use it later on in my code talking about maker suit it gives you a taste pad where you can try different prompts for example text prompt let's go here and here you can write different prompts and it will use this text bison model Google p is architecture but the specific model that it is using is text bison the creativity parameter means if it is more closer to one then it will be more creative if it is more closer to zero it will be less creative you can try some sample prompt for example summarize a paragraph and when you run it it will summarize the uh paragraph you can try poem writing or write your own custom prompt behind the scene it is using the same API that we will be using in our project therefore if you want to quickly taste your API this taste pad allows you to do that very easily you can play with different prompts but as far as API key is concerned we are all set now we will set up our mySQL database now I have launched my SQL workbench by going here and typing MySQL workbench if you're not aware about my SQL don't worry you can go to YouTube type code basic SQL tutorial I have this 1 and a half hour tutorial where I have given a complete idea for any beginner uh so you can follow that and learn my SQL easily this tool is by the way free free you can download it easily by going to Google searching for my SQL workbench I will open this local instance and if you check video description below I have given you all the code files this will have a database directory you can go here and drag and drop this SQL file here this file is taking care of creating database and tables within it you can click on this execute icon and it will create the tables and data within it when you click on this refresh icon that's when you will see atck t-shirt database you can right click on it and set it as a default data set if you have not set it like that before so you just say set as a default schema and you will see that this font will convert into bold now table wise we have first table which is T-shirts if you click on this third icon you will see some sample record for example t-shirt ID one is when who SS red color T-shirt in small size price of one t-shirt is $15 we have total 70 t-shirts available in our store that's a stock quantity if you talk about discounts t-shirt id1 which is the same van H red color T-shirt has 10% discount what it means is the $15 is original price 10% of 15 is 1.5 so when I sell this one t-shirt I'm going to give 1 $1.5 discount to a customer so they'll get it for $13.5 these records by the way will be different when you execute this SQL script because we are using some random numbers here so don't worry if you don't see the same exit numbers in your case they are likely going to be different all right our database is set up now let's start coding in our jupyter notebook I ran python High M notebook to launch my jupyter notebook and here I have created a new python notebook I'm going to import Google pal model from Lang chain. llms okay now you can use uh open EI all kind of models Google Palm is free so let's create a an object for this llm and here I'm going to pass Google AP key which will be stored in a variable called API key and I will initialize that variable here and add my specific key here folks as I said before please use your key I'm going to delete my key later on so code will not work if you use my key once llm object is created you can ask some sample prompt for example write a PO PO on my love for DOA Dosa is a South Indian food I love that and you can you know print a poem on that and you see it is working good now folks before you run this code make sure all your libraries are installed so you can run this command pip install hyphen r with requirement. txt file and if you look at the requirement. txt file it has all these requirements Lang chain chroma DB all of that so I'm assuming you have installed all of that all right now let's create a an SQL database object and for that you can import this particular class and when you create SQL database object you will say from URI okay and here you are going to pass a URI or URI is like URL it specifies what is your database what is a host username password and so on so we'll store all this information in different variables my database is running locally therefore Local Host username password is root and atore t-shirt is a database name see you see it here okay now the way URI is formed is using this syntax I'll just copy paste to save time on recording you don't need to remember all these things anyway so this is the syntax of it all right and then the second parameter is sample rows in table and and I'll show you what this uh number three means so here the result that I got I will store it in this variable called DB and this will have a property called DB info which we can print when I do this uh I will get a confirmation that I'm able ble to connect to my SQL database from my jupyter notebook and see it is able to pull all this information which means my jupyter notebook is now connected to my database now we are ready to create our SQL database chain okay so in Lang chain there are all kind of chains like SQL database chain um and for different type of use cases you will have these different chains so this SQL database chain if you notice is imported from Lang chain experimental module now if you are seeing this video in the future it is possible you can import it directly from a linkchain module but as of right now it is part of the experimental module in the future if they make it available just just remove this thing you know use your common sense and you should be able to run it now let's create the chain object okay and this chain object will take first parameter will be M that we created second one is the DB object see this particular DB object and then you can store this into DB chain and now you can run a simple query before I do that I will pass one more parameter veros true so that I can see the school query that it is generating and I can see some internal details the first question I'm asking is this this okay let me just copy paste here how many Nike white color extra small size t- do I have and let's store it in this UNS variable control [Music] enter okay this is happening because here I need to use from llm see it pulled a right answer it is saying 59 and if you look at this query the query that it generated if you run it it it is actually the right query that it generated okay so here uh let me see here I can run that query see 59 if you just look at uh Nike T-shirts overall uh or let's say let me just do star here only Nike T-shirts see Nike t-shirts are this if you look at Nike white t-shirts this much and in that extra small size t-shirt quantity is 59 and the answer that it is giving is 59 if you do qns1 see it is giving 59 by the way it is giving a dictionary as an output if you want to get directly 59 here you can use run so when you do that q&s will have the direct answer now there are a couple of observations I have llm is actually doing pretty good job because I said extra small size and it is smart enough to figure out that extra small means excess and it is able to map that to size column when I say white color W is small but it is able to map it to capital W because it looked into our database and figured that our color starts with a capital letter so you see this is the power of LM now this was relatively simple query let me try a different query and the query is what is the price of the inventory for all SM small size t-shirts now while it executes this let me run that code here so we want to get all small size t-shirt okay so here I will say where small size okay let me get all small size t-shirts these are all small size t-shirt and what is the total price total price will be price into stock quantity so here I need to run sum I will say sum price into quantity okay and when I run that it is actually stock quantity so when I run that this is the price I get now let's see what we got in our notebook 215 wrong answer folks so why did that happen let's just think about it so here the problem is it said sum of price it did not say sum of price into stock quantity it forgot to multiply by quantity if you think a little bit you will actually find an obvious reason and llm is thinking that whatever t-shirts I have so let me show you these t-shirts it is thinking that the price column is for all the t-shirts so for Levy white color small size t-shirts I have total 51 t-shirts available and the price of total total price of all 51 t-shirt is 13 that is what llm is thinking because the column name is not price per unit it says price so price could be total price or it could be price per unit it is assuming it is the total price and if this was a total price then the answer that llm gave would be correct but in real life database column names are not going to be perfect so this is representing a real life scenario okay so the conclusion that we get is llms will make mistake and we need to tell it somehow that the price column is price per unit it is not the total price we can do this using few short learning we will do that after some time let me run some few more queries okay and meanwhile I will store the right q&s answer okay and the way you can do that is you can actually run the explicit query so the explicit query that we have is this okay so I'm going to run that query here so in DB chain. run you can actually run the explicit query and we got this right answer which is stored in my qns 2 all right so far it is looking good now I will run the third query which will be little bit complex so I'm saying if I sell all my lais t-shirt today with discounts how much revenue my store will generate now when you want to apply discounts you need to do some kind of join okay so you have all the Ley t-shirts okay so these are all the Ley t-shirts that you have you need to multiply price by stock quantity and then you need to sum all of this you will get total revenue then you need to go to Discount table and figure out on leis t-shirts how much discount you have for example one of the T-shirt ID for Ley is three Ley white extra small is three and for three we have 20% discount so on this price so 44 into 94 you need to apply 20% discount in that so let's see if llm can handle this kind of complex case no so it failed it failed because in the query now you see all these columns discount. start date discount. end dat usually whenever you have discount you will have start date and ended column because discounts can't run forever right there will be started and ended but in our database if you look at discount table we don't have start and end date so llm is using its general knowledge and it is assuming that there will be start date and end date in our table we need to tell it that hey buddy don't use your brain okay look at the table and if you find a column then only use it here start it doesn't exist how come you just use it you know so we need to tell that and again we will do that using few short learning after some time for now let me run that query uh explicitly okay so here this is the query uh to get the answer and we'll run it and by the way this is not a MySQL tutorial so I'm not going into detail uh but let me just just very quickly explain how this thing works so here if you look at this particular query it has this part is a subquery and if you execute this query it will pull all leis t-shirt it will multiply price by quantity and it will give you that so for t-shirt ID 17 if you sell all the T-shirt you will get this much revenue for 64 you'll get this you can sum them up to get a total revenue and then you need to um make a join of this table with discount table see this query by the way the result is stored in table called a and you are doing a left join with discount table and then you are applying the discount here so if you execute this this is actually the answer okay 24367 and if you look at this 24367 is the answer we got which we have stored in this qns 3 variable similarly let me run a few more queries so this is if I sell all Levis t-shirts you know how much revenue will I generate I will generate this much and another question I have is how many white color leis t-shirts I have now now let's go and figure that out so you want to know how many white color Le leis t-shirts So when you say total leis t-shirts it is this much and white color right so you will say and color is equal to White okay this much so total white color lais t-shirts are 94 + 51 + 29 15 and 95 but in our code what's happening is see 94 151 it it pulled all that numbers but it did not sum it up so if you look at the answer the answer is 94 why it did that because it is not able to figure out that it needs to do sum here okay so the right query here is sum of stock quanti so that will be 284 that's the right answer again it failed so what what what do we do now well we run the query explicitly so that later on I can use it in my few short learning okay so qns 5 query I will just copy paste whatever I wrote in my C workbench and qns 5 now is 284 so now we have all these answers and we have all these queries uh let's try few short learning uh so that our llm can improve uh on the errors that it is making in few short learning the first thing we need to do is provide the question and query pairs where llm was getting confused once we have those training example the Second Step would be to convert it into embeddings and we're going to use hugging pH for that so let's go to a notebook and start putting together those few short examples in a simple python list and each of these examples would be a dictionary and dictionary will have one element which will be uh your question okay so let's say my question is this and then the SQL query corresponding to that question would be this so we previously ran all this queries so I'm just copy pasting just to save time other than these two we need to have SQL result and answer as a parameter now why do we need this well just hold on we will uh see this later this is the syntax that the default Lang chain SQL prompt is using therefore we are using the same syntax I'll show you a little later and this first answer if you remember we stored that into qns1 Okay so one is nothing but it is this 59 okay so that's what we are having so we put we take all these samples and put them into this single array and once we have this thing ready the second thing is we use hugging phase for generating embedding and for that we'll use uh we'll import the hugging face embedding class and we are going to use this particular embedding now folks there are so many different ways to generate these embeddings I tried this particular embedding it was working fine so that's what why I'm using this you can even try open embedding if you're ready to pay the price and there are instructor embedding uh in the other project that we did for at Tech domain we used instructor embedding so you can use whatever embed can solve your need and this will be stored in this particular variable and let me just you know we can say embed query what embedding will do is you can type any query and it will generate an embedding which is which is just an array okay so let me save it here and the example you can use is okay let's say we generate embedding for this particular sentence so this e will be a list of size 384 and when you look at these numbers they don't actually make sense but they capture the meaning of this particular sentence in a right way so that if someone types A different query I mean the query is like similar to this but the words are different even then the embedding of that em and embedding of this query will be similar in terms of cosign similarity uh of course so I'm going to uh remove this and I'm going to now uh create a vector database and for that we need to create a blob of all these sentences okay so let's say I have this sentence what I need to do is I need to remove all these keys because they are really not needed okay so I remove all these keys and then I kind of merge these strings together so see I will merge all these strings together and I will generate a single big string with some space in between like this and this Q ands whatever that answer I think it was 51 59 whatever right so I want to generate this kind of blob and this text block I will vectorize and store it in my database now to do that I can use list comprehension I can say uh for example in few shorts so this is the array right like few shorts this is the array I can say um example do values so I'm interested only in the values these values not the keys so when I do that uh I will get this kind of uh a list and each of these elements are dict values I want to generate a string out of it and how do you generate a string from a list well if you know python you can do join of this and I will say two vectorizes this see it generated this uh list and if you look at the first element it is simply taking all these four values and and making one one big string out of it okay now let's create a vector database for which we are going to import chroma chroma is the vector database that we are using in this project and then from chroma we can say from text where you supply the text okay the the array of text and the second parameter is embedding okay so embeding is equal to embeddings and the last parameter is the metadata which is few short so the entire few short array that we have we are giving it to as a metadata you can go ahead and read the documentation but the essence of this statement is that this is how you generate a vector store so this Vector store is this Vector store it's already created and the job of vector store is to take an input question so let's say if I have an input question like this it will convert that into embedding and it will pull you the similar looking few short example so let's try that and uh to see how that thing works so for that similarity matching you need to import another class called semantic similarity example selector and in that you will pass two parameters so first first thing is obviously you need Vector store so you will say Vector store is equal to Vector store and then K is equal to 2 which means pull me two similar example K can be 1 2 3 I mean if you want three example just say three this I have stored in example selector variable and you can say select examples okay so you can give a a a sentence okay so you can give a sentence like this here and you can say can you pull me similar looking things from this and see the similar looking question is how many many t-shirt do we have left so just read this two statement okay this and this they look similar and the second based match is this this is not exactly matching but this is like a second based match that you can get okay so this mechanism that you give input sentence to Vector database and you can pull similar looking queries see if you can pull similar looking queries then my llm can look into those and from those queries it can learn and it can produce a good result all right now if you remember we already discussed giving a custom prompt to our llm because our LM is making mistakes such as discount table doesn't have a start date it is still using start date in my my SQL query so I want to have a custom MySQL prompt saying that only use database table columns right do not just make things up so I want to give some instructions so that it doesn't make a mistake now I have to write that SQL prompt on my own but the good news is that Lang chain already provides this prompt to you you can import that prompt by doing this and if you print that prompt let's see how it looks see you are my SQL expert given the question create my SQL query first never query for all columns I don't want to say select star I want to say select XY specific colums you must query Only The Columns that are needed to answer pay attention to use only the columns that you see in the tables below see this is important we are saying we are going to give you the table info and only use see table info is this folks this table info that you printed use the columns only from these tables okay that's what we are seeing also if you're talking about any date uh used current date for the today so we giving lot of useful instructions uh and then we are forming a query okay if you look at the prefix so let me print that one also I think suffix so suffix is like this okay So eventually what we'll do is see we'll take our prefix so prefix is this we'll take our suffix suffix is this and our actual query will come in between so here if you look at our prefix see prefix has this this format question SQL quer equal result answer and that is the format we have used here see look at these four elements that's exactly the format that we are using so now let's think about the query the the query in the middle okay will go in the middle for this we need to import a prompt template okay and then that prompt template folks just to save again time I'm just copy pasting things it will have question SQL query SQL result answer and the template will be something like this okay so what happens is actually when you actually type in a query um this this this query will have that format okay question will go here the actual question that you're generating the the SQL query and so on okay it's I think intuitive if you have seen my previous videos you will you will get some idea now comes the time to create our few short prompt template okay and in this F short prompt template we will pass bunch of parameters the first one is obviously the example selector that we have created see this is the example selector so if you do this you will establish the association between your LM and Vector database you will say hey llm if you're confused look into this Factor database okay so that is what we are doing here the second one is the example prompt that we have created and then the next two are the prefix and suffix so now using these three it will generate this kind of single prompt that you can pass to your Google Palm llm and the last parameter is the input variable okay so in the input variable you'll see things like table info so table info is this this is the table info okay and if you look at our query see that is a table info so here actually wherever you see this this bracket here you will actually put that table info okay so you will put all of this so your actual prompt will be a little bigger you will say this see now you're saying that um use the table info so here you I think you say somewhere right info info okay see pay attention to use only the column names you can see in the tables below so which tables see only use following tables this so this type of big prompt will be formed when you write this particular this particular few short prompt template we are going going to save it in a variable here okay and then we are creating the same chain see we created this this this chain before okay if you if you look at this code remember we created this object so it's exactly same okay we are doing that here but now we need to add one more parameter which is prompt this is the only additional thing we are passing so that whenever it is confused it uses that new information and now let's give those queries which were failing so how many white color lais t-shirts we have okay if you remember it was not using sum before now it is using sum so see it it worked second query is how much is the price of the inventory for all small size t-shirts and previously it was not multiplying it with uh this quantity so let me just show you this one so if you look at the previous query see it was saying 215 because it was not multiplying it with the stock quantity now it is doing that see it is producing the right answer and it's not like see you can give little similar query so I will say how much is the price of all the extra small size t-shirts this is little different and it will work uh it will see it will say size is excess that way so let's try the most difficult ones one that we had which was that Lees so instead of Lees I'm saying Nikes and I'm saying after discount how much revenue will it generate see it worked it said brand is Nike this is this you can you can change this a little bit you know when you change the language it is still doing that semantic search you know when we when we did this semantic uh similarity example selector it is doing semantic search therefore you're not passing the exact query which you passed in your few shot you can pass little different queries as well so folks try different queries and uh it is possible that for some queries it may not work in that case you will take that query and the right SQL query and then you will add it to your few short example right now I have five but if you want to make all kind of queries work you might have 40 or 50 different type of few short examples okay so wherever it is failing take a question take a write SQL query and add it to few short example and after that it will not make a mistake all right we are all set to put all these things together and write a streamlit UI which you will see is only few line of code so we are almost there folks please stay on you have come so far I know it requires a lot of patience but learning llm is amazing for your career now we will write the code for our project I have created at project folder here and here you will find two files requirement txt and t-shirt sales Jupiter notebook from here I'm going to launch pyam community Edition which is a free editor for Python and there you can open that particular folder so I will go here and in the C code directory I will open at Le T project folder like that and we'll not create any virtual environment so I'll just say cancel so there is no virtual environment and let me pull uh this window right here the first file we are going to create is main.py okay and in this file we are going to create our llm object so now now I will copy paste the code from our jup jupyter notebook to here we use jupyter notebook for all our experimentation and this is what data scientists do in uh when they're working for companies they will do some experimentation in the notebook and when they feel the code is ready they will try to productionize it and they will move it to a proper python file structure or a project structure so let's import all those uh libraries so I'm I'm just going to copy paste all the libraries that we imported in our jupyter notebook you can configure your python here and the first thing if you remember we were doing was creating a Google Palm object okay so here I'm creating Google Palm object and we need to give Google API key now in production code you don't hard code your key here the standard practice is to create environmental file so do EnV and you will keep your key here okay so this is the key I have kept here and how do I get this key from here to main. Pi well we use this special python module called environment from there we will import this method do load this one and when you execute this method it will specifically look for EnV file and it will read the content and it will set this as an environment variable so this key will be environment variable and this will be the value okay so after this line it has set the environment variable now how do I get the value of that environment well you need OS module so you will say OS do environment and in that the variable that we want is this particular thing okay and temperature is 0.1 I I will not keep creativity very high otherwise it will start bluffing okay and once llm object is created the next one is obviously the DB object and the third one is uh our embedding okay and what I'm thinking is I will create uh a function which will encapsulate all this code so let me put all this thing in a function here all right and here I'm going to copy paste all those things so if you remember we had embeddings we had um our Vector database and we have few short so few short was an array and I would like to put that array in a separate file so I will call it few shorts and this file will contain all of this now answer I have hardcoded folks because here we are giving this as an example to our L that's see this is how this is the format of your answer the exit answer it will get by executing this query so so that's something you need to keep in mind and you can import that thing here you can say from few shots import few shots okay so it will not give an error now after example selector I just copy paste all the code so you're creating a same exact same SQL database chain and returning it here and we will create a main F python function we'll say if it is main this is how you create a main function in Python by the way and you will get this chain and then you will run this chain okay you will say whatever is your query and you will put in the result okay so what is my query let's just give some sample query to test when you're doing this type of coding it makes sense that you write some code and then test it you write some code and test it so here I will just run this and see it gave the answer and if you look at the query the query seems to be right you can try different queries here uh but let's say this is working now we are ready to write our streamlit code for streamlit I would like to keep the UI code in main.py and I would move all the Lang chain code in a separate file I'll call it uh Lang chain helper maybe and let me just crl a xrl v you know contrl c contrl v is is the most powerful weapon for all programmers and then here I will import that method now let's do streamlit coding you'll say import streamlet as St st. title what is my title well my title is this and you'll have input box right so text input box you will type a question here and that question you will get here and then if question meaning if someone types in question and they hit enter the code flow will come here first let's uh teste this uh be skeleton so here in the terminal you can run streamlet run main.py and it will launch the UI in a browser so see my UI looks good when I type a question hit enter it will take that question in this question variable and the flow will come here so here what we need to do here we need to obviously first get a chain and then we'll say chain. run this is my question and you get the answer and then St do header so you will put another element on the UI and you will say St do write your answer the good thing about streamlet is that you don't have to reun it from here you can go here and just click on rerun now let's ask those questions so I will hit enter here see hooray 3083 let's ask a different question and by the way if you want to see a correspond query since we have set waros parameter to be true let's say you get 59 answer you're not sure if it's right or wrong you can either go to my SQL run the query or you can look at the query here see how many t-shirts do we have left for Nike and access and white see Nike access white sum of stock quantity this is perfect and then um yeah so folks you can ask uh different questions and get your answers now this tool will be very useful to our store manager Tony Sharma because he will be able to ask questions directly and get answers on most of the questions that's it folks we are done with this gen AI mini course we also finished two end to end projects which is something you can add in your resume if you like this video please give it a thumbs up and share it with your friends who wants to learn gen AI if you have any questions there is a comment box below [Music]