Transcript for:
Building Generative AI Applications using AWS Cloud

hello all my name is krishak and welcome to my YouTube channel so guys welcome to the crash course of building generative AI application using AWS cloud in this crash course we will be seeing various AWS services and how we can specifically use them in order to build amazing generative AI applications not only that we'll also go through AWS Bedrock Services which provides you various Foundation models in the form of apis without worrying about the inferencing part and we will try to build end to endend Genera application using Lang chain and we'll also be deploying them not only that we'll also be including hugging face and how we can actually use hugging face in AWS Sage maker and how we can deploy the llm models uh we will be seeing all these things uh just to make sure that uh the effort is sat effort is satisfied from my side I will keep a light Target of 2,000 for this particular video please make sure that you hit like and yes put down your amazing comments the comment Target will be 200 in this video I'm going to probably discuss about the Gen AI Project Life Cycle now since you already know that we are definitely going to develop a lot of applications specifically in cloud from the basic data injection till the deployment so it is very much necessary that you actually understand a generic workflow of the Gen AI Project Life Cycle so let me quickly go ahead and let me go ahead and write some amazing things for you in this particular uh notebook and I will be explaining you completely step by step how you can probably see see or follow a Project Life Cycle and uh as we go ahead there will be a lot many things that will be coming like llm Ops platform uh and uh we will be working specifically with Azure uh AI Studio AWS Sage maker studio and all so definitely both the clouds will get covered so before I go ahead please make sure that you keep the like Target of all these kind of videos till thousand that will definitely motivate me and I've been exploring many more things so that you get the right kind of guidance and knowledge so let me go ahead and let me start the Gen AI Project Life Cycle uh with respect to this gen AI Project Life Cycle I would like to make this entire life cycle into four to five steps okay the first step that is nothing but here I'm just going to write over here it is basically defining the use case okay so what kind of use case are you solving then this use cases can be a rag application can be a text summarization application can be a chatbot So based on different different use cases that actually depends on your requirements your company requirements so this is the first step you really need to define the use case that you are specifically doing now with respect to this particular use case we usually take this entire module into the scope part okay so this is basically my scope right if I basically use a generic term now once you define a use case you let's say that I am going to probably develop a rag application in that I'm going to definitely use Vector databases I may have a lot of PDF files I also need to probably convert that into a vectors and store it in some kind of vector store DB so some kind of use case you really need to Define and all the requirements that is required in that particular use case coming to the next step coming to the next step which is super important because this steps will will be involving two important things okay and that is nothing but choosing choosing the right model OKAY the right model when I say choosing the right model here they are two different things that you can probably split this into one whether you are using or you are probably using some kind of foundation models so here I'm going to probably write whether you are using Foundation model and solving a use case so this is the one category that I would like to divide this particular model into the other category is that whether you want to build your own custom llm right custom llm now see there are two things over here right when I say Foundation model Foundation models are already those larger models like opening AI llama 2 llama 3 right you have Google gini pro so these all are very huge Foundation models and first for most of the generic use cases you can directly use those kind of foundation models and you can solve the use case itself right now with respect to this Foundation models we can also further go ahead and do finetuning right so let's say I have a find uh I have a foundation model which I am specifically using to solve my business use cases on top of that if I really want to make this Foundation model behave well for my own custom data then what I can do on top of this Foundation model I can use Laura CLA techniques and I can probably fine-tune all these kind of models okay so this is one of the step the Second Step the Second Step that I have written over here is Custom llm Custom llm is nothing but it is building your llm from scratch building your llm from scratch from scratch right and obviously uh this one there's a lot of benefit also if a company is building an llm model completely from scratch for this specific use cases but lot of resources will definitely be required we have to really take care of model hallucination many things and all as we go ahead but yes uh I've also seen many many companies developing their own custom llm model okay so choosing the right model or what kind of models you're specifically us to solve this particular use cases that becomes the second important module with respect to this Genai Project Life Cycle and obviously I've spoken about Foundation models both in aswsu have and Google you have and Microsoft Azure you have currently Microsoft Azure AI Studio specifically have all the excess of open AI Services obviously it is investing a huge amount of money over there okay now once you probably select the right kind of model there are main three tasks that you probably do for going forward okay so main three task okay the first task is nothing but you can specifically use prompt engineering and solve a use case prompt engineering and solver use case the second task that you actually can do is nothing but fine-tuning right fine tuning fine tuning so with the help of fine tuning also you can probably develop your own custom llm model and on top of that you can basically do it let's say you're completely creating your llm model from scratch one more important mechanism that you have is nothing but uh aligning uh or you can probably say training with human feedback training with human feedback and this is one of the very important step that is actually used while you're training your llm models how a llm model is basically trained I've already created a video in my playlist uh with respect to the L chain and all generative AI playlist you can probably go ahead over there fine tuning how to specifically do fine tuning and all that also I've actually shown you the the reason why I'm showing you this gentic AI Project Life Cycle because tomorrow when I'm probably creating videos um let's say in the upcoming videos when I'm creating videos related to AWS series over there you'll be seeing all these particular steps going ahead okay now once you probably do all the steps uh the further step is something called as evaluation okay evaluation is basically seeing that how your model is performing by performing all these particular steps they also different different per performance metrics which we are probably going to follow okay this two steps I would like to combine and say something like this okay so I'll say probably adapt and align models okay so this will be the specific model that uh we specifically use for this purpose now over here your model will be ready everything is perfect uh you are able to solve the use cases let's say your performance metrix is increasing over here so your metrix is spef specifically increasing and it is saying that now your model is ready now it comes to the deployment part right so with respect to the deployment part I would uh definitely say deployment uh further you also need to do lot of integration with different different application so I will probably say application integration okay application integration and here uh what we do we specifically perform two major step one we optimize models okay um I'll just write optimize and deploy models optimize and deploy models okay and this deployment is specifically done for inferencing okay and here is where most of your Cloud platforms here is where your llm Ops LM Ops is used you know different different inferencing techniques are there one technique I've already covered with respect to a platform which is called as Gro right it uses a inferencing technique which is called as lpu so it is always good idea that you should definitely know multiple ways of inferencing see at the end of the day whatever models you specifically create unless and until the inferencing is not fast you definitely cannot use those things right so it is very much necessary that you know the idea of this model module extensively because tomorrow building all these things is very easy fine-tuning is very easy right you definitely have a template framework data set preparation and all and you can perform this particular step so that is the reason in this series of videos we'll be seeing that how much I will be focusing on this llm uh llm mops platform right and I will also show you multiple platforms like which can definitely make your inferencing very much good so this is the most important thing here uh definitely we'll be using AWS Azure right you can use all these things gcp and we'll see what all services they have specifically provided probably for the inferencing purpose again but initially our Focus will definitely be on AWS okay then the second step uh um after we do the deployment uh in the application inte integration what we do next is that we build U llm powered application llm powered application powered application because your integration is done your API is created now it's all how well you can actually build the solutions you can solve different different use cases and all so this overall gives a brief idea about the entire gen AI Project Life Cycle um since uh we have already started this generative a on cloud so this is necessary to know and uh you should probably follow all the steps and whenever I create any videos with respect to gen on AWS all the steps will be considered in mind and it'll be shown to you we're going to continue our generative AI on AWS Cloud series and in this specific video we are going to develop an amazing end to end generative AI application uh using various services that are presented in AWS Cloud I will give you a brief architecture what we are going to implement and then step by step I will be showing you how you can use various services that are present in AWS and Implement some amazing generative AI applications so what we are going to probably do in this specific use case uh this use case I'm just going to keep it as uh blog generation so here you can probably see blog generation is there uh initially what we will do is that our main aim over here will be to create an API okay so we will try to create an API so we are going to use Postman to hit this specific API with some specific body uh that body can include query okay query from the user like what blogs we need to probably generate and as you know in AWS uh in order to create apis we use Amazon API Gateway so we are going to also use this as soon as we trigger this particular API it is going to hit our Lambda function what exactly is a Lambda function we'll also understand about that this Lambda function in turn will be interacting with Amazon Bedrock okay now if you don't know about Amazon Bedrock here our all the foundation models will be available right foundation models will be available foundational models will be available like uh Amazon has lot of different different Foundation models like um llama 3 Lama 2 you know it also has paid from Cloudy anthropic specifically cloudy models and all are there we'll go ahead and see what are all these kind of find Ocean Models and how we can specifically use that and this Lambda function will be responsible taking up the query hitting this particular Foundation model and getting back the response once we get the response in this particular case will be our entire Blog the 200 words or 300 words blogs that we are probably going to create and finally what we are going to do is that we are going to take that specific block and save it in awss 3 in the form of text file or any kind of files okay we can also convert that into a PDF file and save it in Amazon S3 with the recent timestamp okay so this is the entire thing that we are going to implement and understand this kind of use cases is even available in AWS documentation uh and many many companies are implementing the generative AI application in this way as we go ahead with respect to the series in the future we will also be developing some amazing applications that are related to rag you know different document Q&A how we can actually use Vector databases uh I will be showing you all those things and we will try to use all the ecosystem that is provided by AWS Cloud one of the important um you can probably consider that in most of the companies they are specifically asking this in interviews they definitely want you to say that how you can probably implement anything in AWS cloud with respect to the generative VI application so uh so many different things uh we are going to go ahead and see so first of all uh let's go back to your AWS account so initially you really need to have an AWS account and make sure that you have it then only you'll be able to use all those Services now first of all what I'm actually going to do is that I'm going to go to AWS bedrock and just give you an idea like how does all the foundation model looks like and what all Foundation model is specifically provided by Amazon Bedrock okay so first I will go ahead and click this and here you can probably see that my Amazon Bedrock will get loaded once it is getting loaded you can probably go ahead and click on get started now with respect to this you can see over here Foundation models are specifically over there and you can actually do chat text image so uh it supports multiple models itself now which all companies model is basically there in this Foundation model here you have a121 Labs that's from Jurassic 2 Series uh you have iton by Amazon you have cloudy by anthropic you have command by cair you have Lama 3 you have Mistral you have stable diffusion so you can actually work in images also and you can generate your own image as you want right let's say if I really want to use Lama 3 so I will go ahead and see this here we can probably see all these things are there like Lama 2 13B Lama 2 chat 13B and even Lama 3 is also there I will also show you that you can also go ahead and check it out in the playground chat with it and probably get it now the first step uh before we start all these things right our first step is specifically to make sure that we have to take the model access understand with respect to different different region yeah you'll be having the access of different different models so you have to request for it okay here you can see that in my in my region that is I have selected Us East one you'll be able to see that I've granted access to all these models anthropic cloudy 3 models are not given because for this you really need to provide some use case uh details uh and then you have to submit it then the entropic will give you you the excess itself like what kind of use cases you are basically doing but according to mine I already you can probably see that I've got this excess granted in order to get this exess you just click on manage model access select which all M which all models you specifically want as access and just go ahead and click on Save changes okay I've already done that so here you can also see that uh the safe changes button is over here so here you can probably see safe changes is there you can go ahead and probably click on this what other models you specifically want just go ahead and check it out and then in some time you'll be getting the exess model OKAY the model exess will be granted so this is the first step that you really need to do okay now once you specifically do it uh then I will show you how you can go ahead and invoke all this C Foundation models and how you can go ahead okay so this was one of the services that we are going to use the next Services is specifically Lambda function what exactly is Lambda function we'll try to understand so here I will go ahead and search for AWS Lambda so here you can see what is Lambda console in AWS uh quickly to test an event it is a server list manage application in okay I will just go ahead and search what is AWS Lambda okay so if you probably go ahead and see the definition with respect to AWS Lambda um it's saying that run code without thinking about servers or clusters see instead of AWS Lambda I can also use a ec2 instance in AWS and then probably set up all the things install all my requirements over there set up all the libraries develop my entire application and then probably with the help of API Gateway I can probably interact with that and within that ec2 instance I will try to invoke my AWS Bedrock that also you can probably do and in the going upcoming examples I'll also show you that if you want but right now we are going to use AWS Lambda so AWS Lambda is a compute service that runs your code in response to an event so suppose if I hit an API this is going to get triggered and then it will go ahead and call the AWS Bedrock so the benefits of Lambda is that no need of managing servers it is automatically scaling and you have to probably pay as you go like based on the number of request and then performance optimization will also be taking place in the AWS Lambda itself uh many many companies specifically use that if you have a smaller application can definitely develop this um so let's go ahead and let's try to see that how we can go ahead and work with AWS Lambda so here you can probably see AWS Lambda as soon as I click on AWS Lambda this will get open here you can create multiple functions I've already created one function I will show you uh how to create this so let's go ahead and create the function over here so as soon as I probably create the function here you can see I can select author from scratch user blueprint container image whatever things you specifically want here what I'm actually going to do is that I'm going to give my function name and my function name what I can do over here is let's say I'll give my ews app okay with bedrock okay so this will basically be my AWS function uh function name uh Lambda function name uh runtime I will go ahead and use Python 3.12 uh just to make sure that I have the recent updated Python and along with that whatever by default is there you can go ahead and select that but here I'll just keep it since we are developing it for the first time and I'll go ahead and click on create function as soon as I probably create the function here you'll be able to see that my AWS bed app Bedrock uh Lambda function will get created and again uh this takes time some amount of time based on the internet speed and uh the kind of application that you're developing so here you can see that it has got created AWS Bedrock I will show you here are some options like layers add triggers add destination now see my main name is that as soon as I probably create an API Gateway right because I need to hit the API then only this uh function should get triggered right inside this function I will write my entire code how to invoke the Bedrock AWS Bedrock Foundation models and all right so if you go down here you'll be also able to see some amazing things like Code test monitor configuration allias version so if I go ahead and click on configuration here you can see there is some timeout configuration that you can probably give there are some environment variables if you want to probably give you can also give it over here directly environment variables I hope I've shown you in vs code we create a EnV file we create a key like open a API key and all here also you can probably write it down and you can call that open API key in this particular code so one General configuration what I will do is that I'll keep this time out to 3 minutes because just to keep it in the safer s so that it does not get um hampered over there and here I'm going to write my entire code okay now before I write my entire code uh whatever code I'm going to write I'll be writing over here but again this does not look very comfortable writing the code because here I will not be able to get much uh you know suggestion with respect to the code it is better that I write this entire code in the aw uh in my vs code and then I'll copy and paste it over here okay so what code I'm going to write it down over here I'll be showing you that specific code with respect to this okay so let me just go ahead and open my vs code over here and here is one of my I will just open my terminal okay oops let me just go ahead and open my terminal and I'm here I'm going to write my code completely okay uh command prompt so first of all I will just go ahead and create my environment so let me just go ahead and write cond create minus P VNV environment python as you know I have taken 3.12 okay so this is the environment that I'm going to create because I'm going to write my entire code over here with some requirement. txt and all so let me just create this I'll create one file which is called as requirements.txt so whatever is required uh for my project and that I have to actually put it in my Lambda function all I will be writing it over here okay so requirement. txt let's wait till this entire is getting created uh so here you can see a new version of this this and I will go just go ahead and click on yes and automatically my entire cond environment will get created so there are two important things that I'll do first uh let me go ahead and write the code for this what all things I actually require I'll be writing it let me create a file also app. P1 okay in requirement. txt the first Library I actually want is nothing but boto 3 so I'll keep this boto3 like this and whatever is the recent version of the boto 3 that is only supported by Lambda function okay not the previous one because uh the recent version of boto 3 has lot of lots of functionalities to invoke Foundation models from AWS Bedrock okay so you should know all these things okay so let me quickly go ahead and write my code now over here so first of all I will go ahead and write import boto 3 and I'm just going to write import boto core do config okay so this is two important libraries that we specifically use understand boto 3 is used to invoke the foundation model okay so here I will just go ahead and do one thing okay so let me just go ahead and see this is my VNV I will just go ahead and activate so cond AC activate VNV along with this what I'm going to do I'm going to just write pip install minus r requirement. txt so that my boto3 gets installed at any point of time okay so this installation will take place let it take place till then I'll go ahead start writing my code okay now what I'm actually going to do my main aim is to create a blog right I really need to write a blog I need to create a blog and for that uh uh what all things I will be doing I'll be showing you step by step uh let me just do one thing I will keep my notepad over here the reason why I'm keeping the notepad so that uh uh step by step uh I've from the documentation I've I've taken all the steps what is basically required and uh when we have to create what what services needs to be created and all I will be writing all those things right so uh I will create a function which will basically be called as definition blog uncore generate using Bedrock okay so this will basically be my function name and uh when I create this function I will also use this I will uh give my query or message or what blog topic I can also basically say blog topic block topic and this will basically be a string and uh this will also return a string okay this will also return a string right so this will basically be my function which will generate the blog here uh what kind of blog I'm going to use uh that I will be discussing sorry what kind of foundation model that I'll be using so first of all let me just go ahead and create all my prompts since I'm going to create a multi-line character so I will go ahead and write this and create a my blog okay over here okay so sorry my prompt I'll write my prompt and before I write my prompt uh let's go and see which Amazon Bedrock model I will be specifically using Lama 2 Chad 13B I can specifically go with this and here you can probably see this is my model name OKAY model ID okay you you you really need to remember this because these are all the things that I really need to give the content type will be application slution and body this is the most important thing what is the prompt that I am going to probably put over here what is the other information that you basically provide with respect to the body right so this all information definitely you really need to provide similarly if you go with Lama 2 chat uh uh LMA 2 chart 70 billion parameters so here you can see this is where you this is where you place your input text okay and similarly if you go ahead and explore Lama 213 billion you can also see other information with respect to so each and every model how the API request should be you should be able to get it from the AWS Bedrock itself okay so all the information are specifically over here and you can basically check it out right so whatever model you want to use okay you have to make sure that what what body what API request is specifically there based on that only you have to probably create everything okay now what I'm actually going to do is that I'm going to create my prompt okay now in this specific prompt I'm going to basically write an S character and let me just go ahead and provide like this s so the first thing is that I will be using this keyword inst as you all know in Lama 2 models we specifically use I'll go ahead and write human okay okay and uh human is basically giving this particular query okay I'll say write a 200 words blog on the topic on the topic and the topic will specifically be my blog topic over here which I will be giving from my input right whenever I hit that particular API okay and uh after this uh what I can actually do is that as soon as I give this particular instruction to as a human being over here uh the next thing is that I will have my assistant right and now I will go ahead and write about assistant and this will basically be closing the int this is the keyword that we specifically use for most of the Llama models right so very simple write a 200 word word blocks uh 200 word blogs on the topic block topic and this will basically be my assistant now as I said I have to worry about the body now my body has has what all things right you really need to go ahead and check it out from here my body has a prompt message it has a max gen message Max gen length message it has a temperature message and it has a topor b so these all parameters I really need to set it up so quickly I will go ahead and write it over here so first parameter will be nothing but prompt and this will basically be my prompt undor text okay so sorry uh have not created my prompt yet okay this will basically be my prompt okay now after I probably defined my prompt the next parameter that I have already written is nothing but maxcore genore length Okay and this will basically get assigned to 52 okay and then you can probably see this will basically be my temperature variable which will be my third parameter uh in my body that I will be sending and another one will be topor P which is another H uh another parameter that is specifically required in my body so this basically becomes my body now also what I am actually going to do is that uh with the help of Botto 3 I'm going to call this Foundation model so that is my uh metal Lama 2 13 million chat model and then further things I will try to go ahead and do it okay now before I call that particular model let me just go ahead and create a tri catch block so that it is always good that we really need to have a tri catch block so that whatever errors we specifically get so I will go ahead and write bedrock and let me go go ahead and write b3. client and here I'm going to basically call Bedrock runtime and if you have already seen my Bedrock videos I've already shown you how you can invoke the Bedrock uh we have to specifically create the runtime the region uh of all the foundation models that I'm actually going to call will be nothing but Us East one okay so this is the region that we are specifically going to use and the next thing that I'm actually going to give is my config so inside my config uh or let me write it like this so this will basically be my config uh inside my config I'm going to use my boto3 well sorry boto code. config dot config and here I have to set up some of the parameters right read timeout okay read timeout and here I'm going to probably take up 300 seconds and I'm also going to give one more parameter which is nothing but retries okay and this will basically be retries and this will be set to a variable which is called as Max attempt uh maxcore attempts will be nothing but it will be three okay so here I'm going to keep up all the values so uh this are the basic configuration that I am specifically giving uh with respect to my number of tries uh all the so in short this variable will be used to call my Foundation model okay now in order to call the foundation model I will write Bedrock do invoke uncore model and here I'm going to give my body the body is nothing but I'm just going to write json. dumps okay json. dumps and again I have not imported Json so let me just go ahead and do that so I'm going to write import Json okay so once I probably write import Json so here a json. dumps and here I'm going to probably give my entire body uh along with this I also need to call which model I'm basically calling so for that I have to mention my model ID and this time the model ID is nothing but it you have to refer it from here so this is nothing but meta 70 billion let's go with 30 billion because the request again I have to pay based on the number of request that is probably going right so model ID here I'm going to just paste it and I'm going calling the 13 billion chat V1 model okay so once uh you do this uh then what we can do we will get our response underscore content and let me just go ahead and write response. getet body so for that response uncore content uh response. getet and here we are going to basically write body do read okay do read so we are specifically going to read this specific response uh and one more thing uh that you can do right and understand one more thing guys very simple whatever response you are probably getting I'll just go ahead and import the response also because I will be using this okay so uh response doget body. read once I read that particular data what it will happen is that I will just go ahead and load this with respect to my response uncore content okay now here you should understand right when I invoke this particular moduel and when I give my body over here I'm basically calling this particular model over here itself that is my Lama 2 13B uh 13 million chart V1 and then with respect to this particular body whatever body we are specifically getting we'll read that particular response and let me do one thing let me just make this particular spelling correctly and as soon as I load this particular response content let me go ahead and store it in my my variable response uncore data okay now once I load this uh let me just go ahead and print it if I really want to see my response uncore data response uncore data okay if you want to probably see it um this response will be having a variable understand one thing that uh whatever response we are basically calling it is giving the response from our entire model the model that we are using the foundation model and now here I will go ahead and write this will basically be my blog details okay blog details and this blog details uh blog details will be available inside response undor data with respect to my uh key variable which is called as generation so this model whatever response it is basically giving you it will be saved in this particular key variable within your response data which has been converted into Json and when you write response. getet body. read in short you are basically getting the response from this okay and finally I'm going to return the blog uncore details so this function here you can see and let me just go ahead and write my accept block also accept Okay accept exception I'm just going to create my exception exception as e and we are just going to print the errors understand one thing guys whatever printing that we are specifically doing AWS lambdas make sure that all the logs will be logged in the cloud watch logs right so that is also very good advantages for you so that you'll be able to see that so errors generating the code uh okay error generating the blog and I can probably display this particular error which is basically over here and uh let me return it with a blank string okay so here we have basically done almost each and everything we have displayed the data this is the blog details that we are specifically getting and this is the function that is responsible in generating the blog okay now let me quickly go ahead and do one more thing let me go ahead and uh See by default if I go inside my Lambda function uh it has a function which is called as Lambda Handler okay so let me just go ahead and Define this Lambda Handler because because my API Gateway whenever it say sends any post event uh it will be captured from this particular trigger and it will hit this particular function first so we have to capture this event along with the context and then we can probably uh retrieve whatever query we are specifically giving see here we are giving the block topic right that block top topic will be captured in this specific event okay that is what is my plan okay so here what I'm actually going to do I'll just go ahead and create my event so event json. loads and I will take this event and let me do one thing let me keep a variable right which is basically called as body because whatever post request we are sending this will be available in the body key itself okay you'll be able to understand it and then this will basically be my block topic so I will let me go ahead and write my blog topic my blog topic will be nothing but I will just go ahead and write event off I can keep a variable called as message or I can also keep a key called as blog so you'll be able to understand inside this body I will be giving in a Json format where my blog topic will be in this particular variable name okay so blog underscore topic let's say here I go ahead and write blog uncore topic is equal to machine learning then I should be able to get this okay um then uh once I probably get the blog topic then I will go ahead and write generate uncore blog and now I will go ahead and call my uh generate blog using Bedrock okay so this will basically be my function and here I'm going to probably give my block topic name which will be nothing but it'll be your block topic that I have actually created so in short uh this function when I'm calling it it will be responsible in giving me the entire blog by calling this particular uh model that is Lama 2 okay so once I probably get this generate blog now I'm going to go ahead and write if if generate unor blog okay now as you know that I'm going to save this particular blog in my S3 bucket okay uh S3 bucket the reason why I'm saving it I'll try to save it in a txt file so that I will be able to access it whenever I really want so let me do one thing let me go ahead and import from date time I'm going to import date time okay so the reason why I'm importing this date time because I really want to create all the txt file in a way where I consider the current time okay so current uncore time is equal to date time dot now and here I'm going to use strf time in the form of hours months and sorry hours minutes and seconds okay and then I will go ahead and create my X3 key key name okay S3 bucket key name so S3 key how my file needs to be saved okay so I'm going to create an F string and let me just go ahead and go ahead and create something like this blog output folder so this will basically be my blog output folder and inside this I will go ahead and write current uncore time do txt file okay so this will basically be my txt file which will be saved in that particular S3 bucket now let me give my S3 bucket name so S3 bucket name and let me go ahead and write AWS Bedrock Bedrock course one okay so let me create this particular Bedrock name okay and now I've given all the information now I can call a function okay and let's say first of all let me just close the if Clause so here I'll say print no blog was generated right no blog was generated perfect now the next step what I'm actually going to do over here is that once I get my key and bucket I should be able to call another function I'll create another function and let me go ahead and write this particular function and here I'm going to write slave uh save blog details okay in S3 okay so this will basically be my function and uh with respect to three I I require two information one is my S3 key S3 key and the second information is my S3 bucket okay so in short I'm taking up all this particular information and uh here you can actually see that I'm I'm saving it in the S3 bucket now the further code I can probably write it over here again over here what I will do I will go ahead and write boto 3 dot client okay so this will basically be my boto 3 let me see I've imported both are free or not so this client will be nothing but my S3 bucket okay and again here I will go ahead and create my TR catch block and uh I will say that hey go ahead and create s3. put object is nothing but with respect to my S3 bucket uh S3 key uh body uh body also I need to save it and the body will be nothing but uh let me just see over here okay body will be nothing but uh whatever is when generate block that I'm actually getting okay so here I will go ahead and give my generate blog which will be assigned to this okay so in short I am putting I'm taking my entire information block that is generated in this particular bucket with this particular file name so here I'm going to probably call my this function that is nothing but save blog details and with respect to this particular save blog details as I said I require S3 _ key the second parameter that I require S3 bucket okay and the third parameter I require is nothing but my generated blog which will be in the specific name okay so done this is good enough everything is probably given over here but again understand with respect to Lambda Handler uh I should be returning something because once the execution is done just for my verification I will be returning something like a status code if there is no error so I will set up a status code which will be nothing but it will be equal to 200 and body will be equal to json. dumps and I'll say blog generation is completed okay just to get me a message okay so done this is good enough uh I've got each and everything over here uh it looks absolutely fine with respect to the functionalities whatever is required I have actually given it over here and this looks absolutely fine with respect to whatever information I see over here right and uh again uh understand uh okay this response variable I will save it over here okay so since I'm hitting this particular bedrock and this response I can remove it okay so that is those variables so let me quickly copy this and let me do one thing let me paste it completely over here right so here I'm just going to paste it and this will basically be my entire Lambda function with respect to all the details and whatever things I specifically require again to write it over here it would have been difficult but now there you could see that how easy it is right now we'll go ahead and click on deploy okay so once this is probably deployed uh I can go ahead and create my API Gateway and all okay one more important thing understand that uh this Lambda function requires the recent version of boto3 and if you probably go ahead and see what default boto3 version is basically present in this Lambda is the older one now in order to do create or have a new boto 3 Library installed what you really need to do okay so here you can see I'll be showing you one very important thing I've actually created a zip file okay so you have to probably follow this step okay and this will basically be helpful for you okay let me do one thing let me just remove this okay so here you can probably see uh I have created a folder which is called as boto 3or layer okay now understand one thing what I am actually doing over here this is important for you all to understand okay unless and until you don't understand this so I will go ahead and activate my V andv environment or okay V andv environment is not there okay what I'm actually going to do c over here okay cond cond deactivate okay now understand one thing as I said my Lambda function requires a new version of the boto 3 installed see whenever we open a Lambda function by default the older version of boto 3 is there now in the older version of boto 3 Because at that point of time uh you did not had a Amazon Bedrock all the foundation models right so what we need to do is that we need to update the packages with respect to the boto 3 now how do you do that okay how do you do that and this I would definitely explain you in a better way see what I have done is that I've created a folder which is called as python okay now inside this python what I have actually done see it is very much important now inside this python I have installed I have installed a library which is called as boto 3 so if you probably go ahead and see this right here you'll be able to see boto 3 boto core all the things information is specifically there right now this will be a task for you just tell me how can you install a library inside a folder okay just to give you pip install boto 3 minus t right and if you go ahead and write this right what it is basically going to do is that the entire boto 3 Library will get installed inside this particular folder and once this folder is created what I can do I can make sure that I will go to this particular folder and I will make a zip file you can compress this particular file and you make a zip file and you can name anything here I've given boto 3or layer. zip okay if you want to install any packages in Lambda first of all you really need to create a python folder install all the packages that you want let's say by default over here if I go ahead and write import pandas pandas may not be installed in Lambda so if you really want to install pandas what you really need to do inside this python folder you will go ahead and write pip install pandas minus t python folder slash automatically the installation will take place and then you convert into a zip file the zip file name can be anything but inside the zip file there should be a python folder with all the packages okay so I'm going to specifically use this okay so I've have created over here boto 3. layer. jip now I'm going to go down and uh let me do one thing uh over here you'll be able to see layers okay so I'm going to click click on layer add a layer now inside this particular layer this basically says that if you really want to add any kind of packages you can add it in the form of layer okay so here you can see AWS layer choose layer custom layers and all specify a layer so what I'm actually going to do I'll just go ahead and select my custom layer see over here you can also create your own layers also right you can create your own layer information so one of the layer you can probably see over here it is written as uh boto 3or bedrock and this is the layer that I have already created okay now in order to create the layers I can go ahead and click on this particular layer I'll go back I will click on create layer see what I've done so okay this is my Lambda function understand this thing okay uh if I probably show you my functions this is my function okay now I've already written the code I've deployed that particular code now what I need to do is that I need to create a layer right so that I update all the packages okay so in order to create the layer I will just go ahead and click on add a layer and over here what you do just go back to this particular layers and click on create a layer and here I will go ahead and write boto 3 updated layer okay and description I don't want I'll upload a zip file I hope everybody knows where the zip file is so this is my zip file so I will go ahead and click on upload okay and I will take this path over over here and I'll paste it okay I will go ahead and write boto 3or layer. zip okay and then uh there is also like which all version you really want to be compatible with so I'll go ahead and write python 3.12 python 3.1 python 3.10 okay so these all layers I really want to put that okay it is compatible with and now let's go ahead and Cate on create the layer okay so once this particular layer is basically added uh created then we can add this particular layer to my Lambda function and this is just to make sure that all your packages are up to date that is the main main reason why we are specifically doing it so it is going to take some amount of time again based on your internet speed and based on that we will try to see that what all things we can actually do okay so let this thing get created and let me have a look with respect to this once it is getting created okay now this looks that it is created now I'll go to my Lambda function okay uh create a function let me go back to my function this is my AWS bedrock and let me go ahead and now add a layer which I have already created it okay so now it should be my custom layer I will go ahead and select this boto 3 updated layer version I can probably give it as one and click on ADD okay so as I as soon as I probably give as ADD you will be able to see that over here my layer will get automatically added so one layer has got added so my code is ready my layer is ready now all I have to do is that I have to probably go ahead and create my API Gateway so that it will get triggered or integrated with AWS uh with my Lambda function and I will be able to trigger things okay so now I will go ahead and call my API Gateway so API Gateway is over here this is how my API Gateway looks like now let me do one thing let me quickly go ahead and integrate this API Gateway with my Lambda function okay so now my API Gateway is opened over here now quickly let's see this uh as soon as I probably get this particular API Gateway I will go ahead and integrate it and then whenever I trigger this particular API it should be hitting my Lambda function which is in turn interacting it so I will go ahead and create my API and here you can see I will be using an HTTP API um over here you'll be able to see can should we add an integration my API name and all are there uh let me just give my API name so here Bedrock demo API I'm just going to write it down review and create okay I will just go ahead and create it okay now once I create it after that I will start adding routes so here you can probably see I'm getting routes over here so let me just go ahead and create some of the routes so here I will go ahead and write post request and here I can go ahead and write blog generation will be my API and let me just go ahead and create over here let me select this post here there are two option one authoriz this protect your API against unauthorized request so if I really want to keep some kind of key for authorization purpose I can do it I can basically attach it over here but right now I really want to show you that how the post request will basically get handled so here I'm going to basically attach integration right now so attach integration I will go ahead and click on and uh with respect to this attach integration I am going to attach to my Lambda function now inside my Lambda function I will go ahead and choose my Lambda function so so here you can see a AWS app Bedrock this is the Lambda function that I've actually created and I let me just go ahead and click on create so once I probably create this here you'll be able to see that my Lambda function is done okay now the most important thing okay one thing is that here this particular API I can also deploy to multiple Cloud platforms and sorry in uh see whenever we do a bigger project there is Dev environment there is QA environment there is uat environment there is production environment so similar kind of environments also you can create so for that just go to the stages okay and by default this is basically the default stage but I will just go ahead and create my stage name so I'll write Dev environment and um I don't need to add the stage variable and all so I'll just go ahead and create okay now inside this particular Dev environment uh you'll be able to see that um here right uh okay let's see this so this will basically be my URL and along with this URL if I really want to call my API so this will basically be my URL so I'll just go ahead and deploy it so no stages found let's see okay Dev is there so I will just go ahead and deploy it okay so I'm deploy this entire URL and this is the API that I have to specifically use for interacting with my Lambda function okay this slash whatever will be my router name I can actually use it so this if I go ahead and I'll copy this also okay just a second I'll just copy this um okay it has gone to this okay no worries I'll go back to my API Gateway and here now you'll be able to see all the API gateways that I have specifically created I have also integrated with the Lambda function that is available over there okay and this is how the entire interaction will take place see it's more about writing a code understanding about each and every services and that is why all these things are amazing right so recent date is 9th so let me just click on this okay post over here I've already deployed this so let me do one thing so this is the/ blog generation if I probably go ahead and see with respect to stages if I go ahead and click on dev so this will basically be my API okay okay done each and everything is done this is integrated uh this is in turn integrated to my Lambda now one thing that I have to do is that I also have to create my S3 bucket still I have not created my S3 bucket because with respect to the code that I've actually given in my AWS Bedrock my S3 bucket name should be uh let's see what is the S3 bucket name that I have to write so it should be AWS Bedrock course one so let me quickly go ahead and create my AWS Bedrock AWS Bedrock cross one so I will just go ahead and create my bucket and let me just go ahead and write my bucket name okay and just let me go ahead and create the bucket okay so this is the new and understand one more thing guys this uh AWS Bedrock course one that I'm basically writing uh it should the name should be unique okay so this is the name that I'm actually selecting AWS Bedrock course one okay and let me just go ahead and create the bucket because throughout the region it will be it has to be unique okay now let me just go ahead and update in my name over here in my S3 bucket name with respect to this okay now just go ahead and deploy it so my AWS S3 bucket is also created the code is written where we really need to save all the files and all everything is ready now uh let's quickly go ahead and test it and I hope everything will work fine so I'll just go and open my Postman okay create my new file it should be a post request let's see I don't know whether it should work everything should work or not if I'm getting any errors definitely I'll let you know so let me do one more thing uh let me just go back back to this um so this is the URL I've already seen I've already deployed it in it uh let me do apis block demo apis so this should be/ blog generation okay so this will be my URL I'll go back to my Postman and you have to be patient guys understand you have to really be patient because they multiple things that you'll be able to see now let me just go ahead and set up my body my raw body will be with respect to one key and if you probably see the Lambda function what is the key name that I've actually created uh let me go ahead and see it I'll open the Lambda function and U once I open the Lambda function with respect to my function name a Bedrock okay and here down if you probably go ahead Come on load it quicker come on it's taking time to load I don't know why so code is getting loaded I've already deployed this code also you also have to deploy the code in order to use it okay so Lambda function is getting loaded let me see this is there this is there okay over here you can see this will basically be my block Topic in the body so let me just go ahead and write block topic block topic and let me just go ahead and write the block topic will be machine learning uh and generative AI okay okay so this will basically be my key name uh inside my this this entire thing will be a event it'll be in the event itself inside that event there will be a body variable and then we will be able to explore this blog uncore topic okay so let me just go ahead and click on send hope so it works fine so it is hitting hitting hitting till then uh message not found why it is saying message not found because blog generation is there let me see um blog blog generation blog generation let me see it blog generation let me go back to my API Gateway and for this also what I can actually do is that go to my configuration or sorry go to my monitor and inside that I can also see the logs and it will be available in the cloud watch so here you can click on view Cloud watch logs and here is my entire Cloud watch walk that you'll be able to see okay but right now it is not working let's see why it is not working uh we'll have the complete details that will be available over here now it goes to my log groups inside that groups specifically where we are actually working with again I'm not going to delete this entire part of the video because I really also want you all to see the logs okay to specify log does not exist in this particular account why okay the logs has been not created let me just hit it once again blog _ topic message not found status 404 I think so there is some problem with my API Gateway stages SL blog generation is it correct SL blog generation this looks fine my post request is this one uh the integration in the background save request let me just go ahead and see my stages my def stage is this one fph let me see whether I've deployed that or not deploy successfully created deployment the deployment is active for Dev this looks absolutely fine now let me just go ahead and hit it fbh this one message not found still message not found let me see that what is the problem in this so guys one simple mistake that I've actually done over here is that uh see when I pasted this entire thing right there was an extra space that was added and Postman does not resolve this particular space and it it keeps the space like that so I'll just remove the space now I'm going to search for the topic on generative AI so I will go ahead and write this 200 words block now let me just go ahead and click on send and here you can probably see blog generation is completed let's see whether everything has worked fine or not I'm just going to reload this log groups and obviously you know how to go to this particular log groups go to configuration okay sorry go to Monitor and just click on view Cloud watch logs so that you'll be able to see the logs itself right now let's see whether the logs uh we'll be able to see the buckets also on not I'll also go ahead and check the buckets so AWS Bedrock so right now nothing has got created so I think there should be some kind of error over there but we'll have a look uh whether there is an error and we'll try to fix it okay uh so all the things usually happens over here so here you can see AWS app bedrock and there is one log uh so with respect to this particular log let's see AWS Bedrock some no blog was generated uh generating the blog an error occurred while when calling the invoke model uh is authorized to provoke invoke model and resource this because no identity based policies allows the Bedrock invoke model action so guys from the error that you probably see over here it is not provided uh it is basically not providing you the EXs so definitely the role that we have actually created for a Lambda function right we really need to change over here right so what we can do see initially it was in code right now let's go back to the configuration and select the role name so if we select the role name over here okay let me just go ahead and select this and now within this particular role name we can provide further access to it okay so let's say over here only the a AWS Lambda function is basically there so what I can actually do is that I can provide more access more more permission policies so that I will also be able to access the AWS Bedrock so to just to see like what kind of policies we can probably provide it you know and over here right now what is basically provided only AWS Lambda so let's see that how we can still add more permissions so here what I'm going to basically write I'm going to basically select uh attach policy and again based on like how your IM console basically runs right uh I will basically provide it to administrator access again if you specifically working in the company you can provide different different access based on the services that you're using but just to give a brief idea about it so what I'm actually going to do is that I'm going to add this particular administrator access so that you'll be able to see that particular exess over here now once that is added uh let me just go back over here and let's see whether it is updated or not uh so I'm roll this is there okay perfect everything works fine so let me just reload this and now I think we should be able to work out and see that yes now we'll be able to get the access so here you can see all access is basically given now I'll go back to my post man and again hit the same request now let's see see if it is taking more time right that basically means my entire application is working fine and it see we are able to see that yes it is taking more amount of time now I think our blog will be generated 200 words blog will be generated and we we should be able to get or we should be able to be storing that entire txt file in S3 bucket so let's see I will go again back over here again reload my cloud watch so let's see whether my cloud watch has basically created or not yes new Cloud new log has been created so here is my entire block that has got generated generative AI the future of creativity and Innovation here you can see everything is working fine code save to S3 bucket I can also see in the logs now it's time that let's go into my S3 bucket and see whether I will be able to find it out or not so I will go back to my S3 bucket let's go ahead and hit this and then I will be able to see this AWS Bedrock course one and here is my blog output and here is my text file right if you want to probably see this particular txt file I'll just download it but usually when we are working with a Services we can probably see that and here let me open this and here here is my entire blog right and this specifically we are using the meta uh llama Lama 2 1 million parameters generative a is a revolutionary all the blogs has been generated so guys uh we are going to continue our generative AI series on AWS cloud and in this video I'm going to probably show you how you can actually deploy your hugging face model Let It Be An Open Source model or any llm models also specifically in AWS Sage maker that is Amazon Sage maker um Amazon Sage maker is one of the important services in AWS where you can probably complete the entire life cycle of a data science project or an AI project completely along with mlops till the deployment each and everything so in this particular video I will be showing you how we can take up any hugging face model in short and probably deploy it in the sage maker itself so please make sure that you watch this video till the end and one more very important thing guys uh uh just check out this particular video there will they may some there may be some amount of charges that may incur so please be careful with respect respect to this whenever you start creating and finally when you create that particular endpoint make sure that you delete all the endpoints itself and I will be providing you the entire guidance as we go ahead so first of all we are just going to go ahead and search for AWS Sage maker and uh to start with uh how to probably work with AWS Sage maker studio and all I will also be talking about that so first of all you can see uh you can just go ahead and click on getting started okay uh and with respect to this AWS Sage maker the documentation is pretty much amazing right so if you don't know about AWS Amazon Sage maker it provides machine learning capabilities that are purpose built for data scientist and developer to prepare build train and deploy high quality ml models efficiently so uh what we are going to basically do is that step by step I'm going to actually show you so I'll go to The Domain part and I will go ahead and create the domain so once I probably go ahead and create the domain so here U there are two options options one is set up for organization and the other one is set up for single user okay so as soon as uh and right now we are just going to test it for single user so what I'm going to do is that I'm going to select this here you will be able to get an IM role automatically it will be creating it with this Amazon sagemaker full access policy uh public internet access standard encryption Sage maker Studio you'll get the access to this you'll be getting sharable sagemaker studio notebooks Sage maker canvas and I am authentication so each and everything uh by inbuilt it will be providing you this entire environment where you can actually do the coding so just go ahead and click this so once you probably go ahead and click this you will be able to see uh something like uh okay the setup is going to take place and it is going to take some amount of time you know um again it depends on how much time it is probably going to take over here it will automatically create the users and everything you'll be able to get it right so here if I probably go ahead and just read load this you'll be able to see that my third domain is probably over here getting set up pending is this and it'll probably take some amount of time based on the usage but other than that that you can probably see that I've already created two so uh since this is already getting created I will just go ahead and show you some more examples with respect to this so let me just go ahead and quickly click on this quick setup domain you'll be able to see that by default it will be creating a user okay you can add any number of users based on the organization that you're specifically working and let's say you want to provide access to multiple users in the same uh domain itself you can also provide it okay so by default uh uh as as soon as you provide the user access also there is one very important thing that uh you will be able to see right and uh that is nothing but the user information over here along with this let me just hide my face here you'll be able to see that you'll be getting a launch button okay so through this you'll be able to access canvas I will be talking about canvas as we go ahead in the further videos tensor board I hope everybody knows about tensor board profiler but we are just interested in working on this particular studio right so once I probably click on this studio and uh uh you will be able to see that I will be able to get this entire Amazon Sage maker it'll be a studio completely which will actually provide you the entire ecosystem with respect to any kind of development that you really want to do okay so here you can probably see Jupiter lab is there if you want to quickly start deploy finetune and evaluate pre-train model specifically for llm you can also go ahead with this you know if you want to probably uh go ahead and do some kind of automl you can also see over here there's an option model evaluation so you'll find this entire ecosystem okay now what I'm actually going to do is that I'm just going to click on this Jupiter lab and right now you'll be able to see that nothing is running over here okay nothing is running specifically and uh what I'm actually going to do I'm going to create a create a Jupiter lab okay over here Jupiter lab space so that I will be able to work it okay so here I'm just going to write test demo sagemaker okay and this I'm going to show you with hugging space okay uh sorry hugging face models right so here uh it'll first of all ask you to probably select the instance right uh multiple instances you can probably select for my use cases again uh this instances if I go ahead and check over here Sage maker instances based on different different instances there is a different different price so you really need to be uh taking care of this because see if you're working in a company based on the requirement you can probably select the instances so here you can probably see on demand pricing so uh let's say if I select ml 33 medium 05 here it provides you CPU with two core 4GB memory but understand if you're specifically working with generative AI models right you definitely require a huge amount of data right uh sorry huge amount of space so uh if you probably go further there will be good good amazing system accelerating Computing which also provides you not only CPU cores but also gpus right so as you keep on using more and more you'll be able to see the charges will be going higher and higher but just uh in this specific video I really want to show you some demo uh but again if you just get this idea in any companies if you go you'll be able to work it out right so I'm going to take a small system and probably work with this right now which small system I'm going to specifically take so if I go up here is something called as ml. m52 large so if you go ahead and see over here uh MLF I 2 large so this is basically providing 8 core gpus and 32 GB memory the charges will be 461 price per hour for using this particular instance the reason why I'm doing this is that because I will just give you an example of one of the model how it is deployed and all okay and uh we'll see to that okay so here I'm just going to keep the storage to 10gb uh and now I will go ahead and click on run space okay so once the Run space is clicked on you'll be able to see that my entire environment will be ready and I should be able to work it out okay and uh with respect to this uh any kind of code that you run right in the my recent video you have also seen that I have actually uh created entire one Lambda function with the API Gateway I've shown you how to probably create the endpoint and all so all those things you can probably do with this also but here more amazing end points you'll be more amazing ecosystem and features you'll be able to get okay so this is going to take some amount of time so okay now it is created now I can just go ahead and click on open Jupiter lab Now understand with respect to hugging phase there are multiple ways to load a specific model okay there are multiple ways uh all those ways I'm going to talk about and this will be important for you because you really need to know have an idea like how we can actually work in AWS system right so once this is loaded uh the first step okay what you can do I will just go ahead and change my theme so I'll make it Jupiter dark okay so here you have options uh you have options for Notebook you have options for console terminal python anything you can specifically work it I will just go ahead and open Terminal uh sorry notebook and the first thing that I will do is that just go ahead and write pip install Sage maker okay which is the upgraded version just go ahead and install that automatically the installation will be taking place and all the recent updates that is basically there with respect to the sage maker that will get updated understand this is my entire environment the same jupyter notebook that we usually specifically work on at all right so here you'll be able to see this now uh to go ahead with what we really need to do initially so I will go ahead and import Sage maker okay and there are some steps that you definitely need to follow as we start right so I'm going to import boto 3 and I'll be providing you all the code in the description of this particular video so first of all we'll go ahead and create our session the session will be nothing but Sage maker. session okay sagemaker do session so by default whatever is the session we'll be able to get it uh initially uh we have not set up any uh session buckets or anything as such you can also create that particular session so I'm just going to write over here even with some comments so that it'll be beneficial for you so here you can see session uh s maker session bucket used for uploading data model locks if you want to probably upload data upload a custom data or download the hugging face model in this particular S3 bucket and probably retrieve from that or use from that you can actually do that s maker will automatically create this bucket if it does not exist okay so you can probably go ahead and write this particular condition and here you'll be able to see that if there is nothing as such um if it is none I'm just going to probably uh make if the session is not there I'm going to probably go ahead and create the default bucket okay so this is the code that we do initially uh and this bucket will specifically be used to upload any custom data our training data it can be models after the inferencing and multiple things now other than this we uh really need to focus on the role management over here because since we are using AWS stemer to execute any kind of code uh specifically by using the services of AWS stemer we really need to provide the role access and all so for this uh we can just go ahead and write some trash block so here I'm just going to go ahead and write try um try and let me just go ahead and do pass over here and let me go ahead and write accept block um with respect to this particular accept block I will just go ahead and raise some value error and the best thing over here is that you'll get all the suggestion right and that is pretty much amazing right once you're specifically using it let's say my IM user is not yet set so it will obviously give an exception so it'll go to this particular value error and here I will first of all I'll go ahead and set up my uh user so I'm I'm user in short so I'll go ahead and write boto 3. client and uh I'm also checking out the documentation page it is always good to probably check out the documentation page and perform in that particular way and then I'm going to basically set up my role um I'm going to use im. getor rooll now it is going to probably pick up those role that is configured with this particular jupyter notebook right so I'm just going to write role name okay is equal to Sage maker Sage maker uncore execution uncore roll okay and this is basically going to be set up with our role and from that we going to probably take our Arn number right that is how we specifically identify the unique role that we are probably trying to give okay so all this things is done over here let me just make some spaces that is actually required whenever we work with this okay so here you can probably see my role is also set up uh this is my exception now let me quickly go ahead and write in a try once the uh ex once the role is set up so I can basically go ahead and write sagemaker do get execution role okay this will be responsible in getting the execution role if by default nothing is set up then you can probably call it out uh based on the role that it has it has okay again one common thing that I've seen that we have to provide spaces over here so that is one of the thing over over here so please make sure that you keep on working with respect to that whenever you're writing any code uh after that I will just go ahead and write session is equal to S maker do session I'm just going to call my session and here I'm just going to use my default bucket uh so here one of the parameter that you can probably see is nothing but your default bucket which will be assigned to my default bucket or sorry not default bucket Sage maker session bucket that is my default parameter that is specifically required okay so this is the first step that you really need to do it okay and then finally I can go ahead and print my you you can see how it is going to pick up the Arn Ro okay s maker Ro Arn and here I'm going to basically set it up to ro okay so this is the role that is going to get print up and next print is equal to and here you can probably also go ahead and write your Sage maker session so say page make a session region if I want to display it okay it'll be nothing but session dot do boto region name right so this is also there so once I execute it uh let's see whether it'll be working fine so here you can see not applying htk defaults this is there not applying info okay some error I can probably see over okay let's go ahead and do that again okay now it's working fine you can probably see Sage maker role is basically this and this is the default role that uh the domain has basically created with respect to the user and then Us East one I'm actually working on okay so this is the first step now if I have the roles and all uh now I can probably go ahead and call any kind of models that I specifically want so uh first uh we will try to call one type of model and uh I will just go ahead and show you the code okay so here I'm I'm calling one model which is called as distal BT uncased distal Squad and this is specifically used for question answering so I am basically going to give the Hub configuration see hugging face Hub if you don't know about hugging face Hub then uh there you definitely have a lot of models which can probably use for multiple use cases so here you can see hugging Cas uh model ID I have to probably give whatever is my model ID the kind of task that I'm looking for I'll be giving in the form of uh key value pairs in this particular variable called as Hub okay then we go ahead and create the hugging face model right and for that we have already deployed or imported from sagemaker do hugging face model import hugging face right and then finally you'll be able to see that I'm using this environment Hub okay role I've obviously used Transformer version which I provided P version and Pi version 3.9 okay so this is how the parameters you really need to give with respect to this okay so uh I've already executed this uh and I don't want to again deploy this entire model in my another jupyter notebook because it is going to definitely take time so just to give you an idea I have already done that so let me just show you over here this is my another instance which I was actually working on so here uh is my second hugging face model which I have actually given third uh how to deploy it to the sagemaker inferences as you know I've selected ml. M5 x large okay so I'm here I'm writing predictor by using the same hugging face model deploy I'll be saying the instance count as one okay because I just required one instance and what is the instance type instance type is ml. m5. x large okay where I'm specifically deploying other than that I need to make sure that how should I give my input so I will be creating in this particular format because this model that is digital B based unced distal squar will be requiring the uh the input data in this particular format that is question what is the used for inferences then context my name is so and so I've just given some examples so that my more charges should not incur and then we'll go ahead and predict the data so here you'll be able to see if I go ahead and write predictor do predict okay with respect to this particular data because this data is required in that particular use case so here you can see score is this one start is this one end is this one answer is Sage maker right so what is used for inferences so here you can see from this particular question uh you have that Sage maker it is going to pick up understand when I execute this code it is going to take 5 to 10 minutes to deploy into this and you'll be able to get your end point okay just to give you an idea how does an endpoint look like um I'll just go over here just a second uh how does the endpoint look like if I if you really want to see so here I if I go back to my sage maker Studio I've created multiple endpoints so if you go to deployments you'll be able to see end points over here right these two end points are there let's say I want to go ahead and see this particular endpoint okay I can click on this particular endpoint and uh once we deploy it that endpoint will be created if I want to test the inferences you'll be able to see that that uh I'll be getting this application juston and whatever body I'm specifically giving like how I'm giving over here this is my entire body right and with respect to that particular body if I give over here also and just test it I should be able to get if I just click on send request I should be able to get my output right and there also multiple options like Auto scaling you can scale to whatever things you specifically want based on the charges again uh but here you'll be able to see that I am I have deployed this model in this particular instance and uh you'll be able to just use use that particular endpoint wherever you want in your code anywhere as such based on your requirement right so this was the entire thing right let's let's try some more thing okay I will again create some more data over here and uh um my name is crush and I teach data science I I'll just give this and I teach data science okay so what does Kish like okay I'm just going to execute this let's see I'm going to get the answer I'm predictor do predict so you have a predict method here you can see the answer is data science right what does Kish like uh what does Kish teach also you can basically do okay you'll be able to get the answer data science okay so in short this is just a simple model right I've probably deployed this particular model you'll be able to see that this is my entire Sage maker and here uh you know it took some amount of time I think this dot dots are like every 1 minute around 6 to 7 minutes to deploy this entire model in this particular instance which I have actually created now just to give you an idea like how does uh uh machine learning uh llm model be like so I'll be giving you multiple examples so first of all you have to probably go ahead and update sagemaker the same thing by keeping your role and uh region name then you are basically going to call the Sage maker image URI right since we are going to run it as a container so here you can see hugging face deep learning container in AWS sagemaker so the entire llm model will basically be put in a container then you can actually call that particular container over here so in order to call the container you will be using from sagemaker hugging face get hugging face llm image URI and then here is your version that you're specifically using and what kind of image you are actually coming up with okay just to show you if I probably give you an example I will go ahead and install all this let's see you'll be able to see the URI okay and then it will probably download it and upload it so there will be a lot of task in that and lot of money will be required because again at the end of the day you are see over here you have probably got this now let me just execute this again so here is your Sage maker role Arn then you can you can see that which Ur I'm actually calling right so if I probably go and see this is the URI that is basically required Right image URL so if you know about Dockers and containers so once you get the URI then you should be able to load the hugging face model now here also you can probably see here we are calling Falcon B 40b instruct number of gpus is how much you want and just imagine the instance type that we are going to use is MLG 52 into large I'll just show you the cost and you will be pretty much shocked to see the cost so here you can see internal storage GB is 1 into 3 3800 uh total GPU memory is 96 you know GPU 24 core bandwidth internet bandwidth is 40 and the GPU model is NVIDIA A10 right memory is nothing but 192 CPU cores is nothing but 48 just imagine how much the charges will be so that is the reason why I have actually taken a small model and probably shown it to you uh later on you can probably do with respect to to anything once you probably go through the company right so that is the reason we have selected all these parameters over here then we initialize the hugging face model then we deploy it that's it right and then after that you can probably use that same payload you can structure the payload over here uh with the prompt and then you'll be able to get the text okay so anyhow I will be giving you this entire examples in the description of this particular video but again yes I've told that I've been uploading videos on ew s maker and all but definitely they requires a lot of charges is involved but it is always good to have a knowledge about it that is the reason that is the main aim for this particular video so I hope you like this particular video this was it for my side uh again uh you can refer to multiple examples uh here I will also give you this GitHub link from the documentation of uh hugging face guys in this particular video we are going to discuss an end to endend llm project with the help of AWS bedrock and Lang chain this was also one of the most requested video from many people out there what we are going to do in this particular project is that we are going to develop a document Q&A application where specifically we'll be harnessing multiple models that are provided by a Bedrock like cloudy laru and you can also use Amazon Titan whatever models you specifically want that actually provides we can actually use them and we can implement this entire application so to go ahead with I will just go ahead and show you the quick demo over here so what is this specific entire application this is a rag right we are trying to create a RG system wherein we have multiple PDFs all those PDFs is stored in the form of vector Mings inside Vector store and whenever we ask any query we will be able to harness the power of AWS Bedrock which provides different different models like cloudy Lama 2 and uh based on this particular question we can retrieve that question from the PDF files so let's say I'm asking what are Transformers and if I go ahead and click on cloudy output that basically is going to hit the Cloudy API cloudy model and I'm going to get the entire response okay so currently this is running if you probably want Lama to Output I can also go ahead and click on Lama to and I can actually get that specific response right so whatever questions you you have with respect to this particular PDF documents what the set of PDF documents also I'll show you and you'll be able to get the entire response over here right so uh why it is taking some time because I still need to write that optimized version of the code but once it runs right for the first time I think then after that it works absolutely fine now let me just go ahead and write what is YOLO right so if I go ahead and click on Lama 2 output before see I showed you also llama 2 uh uh in my local system itself over there it was taking lot of time but this is directly coming from the API model itself so here is what is the response I'm specifically getting and I'll be able to get the answers now this is what I'm going to develop completely from scratch already if you remember guys in our previous session in our yesterday I had uploaded a video where I showed you the power of cloudy. py how to invoke the model and all uh then we also saw with respect to Lama 2 and all now let me just go ahead and create a new file over here so let me just go ahead and write this as app.py now inside this app.py I'm going to write my entire code now before I go ahead I really need to do some installation with respect to some of the requirements as said uh here we are going to use not only boto 3 and AWS CLI we are also going to use some more libraries like Pi PDF so let me just go ahead and write this over here so we require Pi PDF along with P PDF we also need Lang chain and then we will also be requiring streamlet right uh along with this we also require fir fire CPU right so this is what we going to use because from our local environment we're going to do the vector embeddings so for this I'm going to use the FB right so through this only we'll be able to create the vector iding into the vector store so these are some of the basic requirements that I actually require now uh aw CLI is also there because we really need to configure it now let me quickly go ahead and open the terminal let me clear the screen because yesterday I was creating the images over here the first step again you have to create a virtual environment do all this pip install and all right I hope I don't have to tell you all those things now because I've repeated many number of of time so let me go ahead and write pip install minus r requirement. txt if you are not able to understand it guys please make sure that I'll be providing you this entire playlist in the description of this particular video you can go ahead and watch it okay but uh you have to follow the playlist right again I'm not going to repeat everything from scratch otherwise it is just a waste of time right so till this installation is basically taking place let me just go ahead and talk about what all things we are going to basically develop right so this is the document Q&A search so in short my PDF will be stores in stored in the form of my entire PDFs we probably consider this my entire PDF will be stored in Vector store and from this Vector store we can specifically query any information that we want along with this we are going to harness the power of langin Lang chin along with the llm models from AWS Bedrock okay so this is what we are going to do in this project so please make sure that you prepare well uh understand the architecture what we are trying to do so our entire project deals with two important step one two and before this we are also going to do a very important step which is called as data inje right now in data inje what we are going to do is that we going to read from the entire folder how many PDFs are there all the PDFs we are going to read it okay so once I probably get this entire PDF then my next step actually starts over here I'm going to take all this documents the PDFs over there split this into chunks create the embeddings and here we going to use this firest database so here we're going to use this FES right the vector database and through that only we'll try to create this Vector store right so step by step I'll show you how to do this what embeddings also we are going to specifically use here see understand we are going to create the embeddings so for embeddings we are going to use today I will show you a new model which is basically called as Amazon Titan okay so that you understand you have multiple options you have you may have implemented multiple options with respect to that because tomorrow in companies if you go definitely they going to use this AWS Bedrock because it provides a lot of features right so for creating this embeddings I will specifically use Amazon Titan other than this if you don't want to use this you can also use openi iddings right it is up to you I've also shown with Google generative AI embeddings multiple embeddings techniques in my playlist with Google respect to Google Germany and all so after we create this Vector store now how we going to use the llm so in the Second Step whenever we ask any question first of all the similarity search will happen from the vector store whatever relevant documents and chunks we get we have to take this chunks give it to my llm model along with the prompt like let's say that I say that okay please summarize this entire information based on the query that I've asked in 250 word so this llm model along with this promt is going to take this particular data and is going to give the answer itself right so this both I'm going to develop completely from scratch step by step we'll try to see it okay so uh let's quickly go ahead and let's start our coding without wasting any time okay so here is my app.py I said that I'm going to do it completely from scratch so um there will be some installation there will be some uh errors that may be probably coming up so we will try to also import all the libraries based based on the pipeline that we have created based on the steps or architectures that we have already discussed so first of all quickly let's go ahead and import Json so I'm going to import Json I'm going to import OS uh I'm going to import uh CIS I think I'll not be using CIS but let it be these are some of the common libraries that we'll specifically use along with this we're going to use boto 3 and again guys in my previous video I show show you how to configure the AWS CLI please make sure that you watch the playlist otherwise you'll not be able to understand okay if you directly go into and jump in this particular video no you'll not be able to understand so in the description I will give you the playlist the first video only I've shown you how you can actually configure the AWS CLI okay so please make sure this okay the next thing is that as I said right we will be we will be using using Titan and embeddings mod model right for creating vectors or to generate embeddings okay so we will create this model the reason why I'm showing you because I've never used in any of my videos now this Titan embedding what I will do I will call from the Lang chain library or Lang chain framework Lang chain provides you multiple functionalities multiple fra multiple options to probably interact with bedrock right AWS Bedrock so as I said this framework is compulsory for you all to know one is Lang chain and Lama Index right so what I'm going to do I'm going to write from Lang chain dot embeddings I'm going to import Bedrock let's see bed Bedrock embeddings right so I'm going to specifically use this this will be small letter okay so guys I'm also verifying it from the documentation okay so there is a documentation that is given not very much clearly but I have implemented this entire project also right so from Lang chain. embeddings import Bedrock embeddings the next thing that I'm going to use from Lang chain do llms like we can also use Lang chain to call this llm models that are present inside Bedrock so that is also integrated right from llms do Bedrock import Bedrock okay I'm going to specifically use this two one is from lin. embeddings bedrock embeddings and from lin. lm. Bedrock um I'm going to import the Bedrock okay so this is specifically for the embedding part okay now there will also be some libraries that I need to import for data injection because I really need to load the data set right so over here you'll be able to see I will write import numai as NP okay uh this is numai I'm going to specifically use but I think by default numai will be available import numai as NP perfect so numai as NP then from Lang chain dot text splitter I'm going to use text splitter so that as soon as I load the document I need to probably you use this recursive character text splitter okay so this is what we specifically use and in my Lang chain series I've discussed everything about this then again I will go ahead and write from Lang chain Dot document uh document loaders import um we will specifically use this piy so let me just copy this Pi PDF directly doers so this is what we are specifically going to use P PDF directly loaders and it is actually present inside lin. doent loaders okay so this is specifically required for the data inje with this what I will do I've already created one folder over here which is called as data which has this two PDFs right I need to load all these PDFs from this particular folder and then perform all the vector embeddings that I required so in data injection first of all we will load it we will split the entire documents by using recursive character text splitter and then after this we will convert this into Vector embeddings so let me just go ahead and write vector embeddings and Vector store here I'm going to specifically use f index okay fire DB or chroma DB also you can use it is up to you again I've shown both the ways in my playlist so from Lang chain let me just see this okay so Bedrock embeddings is there something called as from L chenore Community it was right let's see okay from from Lang chain Community let's copy this is this correct or not we'll try to see yeah bedrock Bings was there uh let's copy this here also so Pi PDF directory loader is also there I think there will be some warnings that will be coming uh I think most of the some of the LI Lang chain Community okay so here what I'm actually going to do from langin let's see if you get an error again I'll revert it okay from lin. text spitter sorry not text spitter why I'm using text splitter already I've actually done it so I'm also going to use for for this Vector embedding specifically uh I have to use Vector uh fires right so I will write Vector stores import F okay so f is one Library I'll be using again this F will be present inside Community I guess okay so Vector langore Community uh do Vector stores import F let's see f is not accessible so I think it is inside Lang chain only okay otherwise you can see the documentation okay at the end of the day no worries then after doing this I will also write from Lang chain do indexes okay I'll not use this also let's just use f because we have already installed F CPU okay then um after doing this uh what we are specifically going to do is that we are now this is for the vector embedding now we need to go ahead with the llm models right so for llm models the langin already provides ways to load models from the AWS Bedrock okay so for here I'll write from L chain from Lang chain Community dot one I will specifically use prompts oops from Lang chin. prompts I'm going to import prompt template okay and then the other one is from L chain dot chains I'm going to specifically use one chain where I'll be using import retrieval QA right so since I'm going to create a Q&A chatbot with the documents uh Q&A in short so that is the reason I'm using this and this is basically to create my own prompt template now let's call the Bedrock client so that we get the excess of all the models okay so I will go ahead and set up the Bedrock clients over here so here let me go ahead and write Bedrock is equal to boto do client and here I'm going to give my service name is equal to better bed Bedrock D runtime okay so I've already shown you yesterday also uh in in my playlist of this Bedrock I've also shown you how you can actually call the client itself and access the models okay now as said as I as I already told that the embedding that we are going to specifically use is called as Bedrock embeddings right so let let me quickly go ahead and call this I will go ahead and write Bedrock embedding and let me show you how you can actually call this embedding from the AWS Bedrock so I will copy paste this let me give the model ID now this model ID you'll be able to see it over here right so if you probably go ahead in AWS itself let's say I will go ahead and write embedding okay so here on in Titan I think you will be able to find it out so Foundation model base models so if you go ahead and search inside this any of the embedding models you can specifically take but I'm going to use this Titan embedding model okay so if you probably click over here and see down you'll be getting the entire details like what is the embedding what is the model ID and all okay so I'm going to specifically use this so let me quickly go ahead and call this model ID so for this model ID I've already copied it from that Pro uh this particular website so here you can see I will have copied this entire thing and I will paste it over here on my model ID so you also have to do the same step okay so once we do this the next thing in this that we really need to give is the client and client you know that we have not yet called any client or what so we have called this Bedrock client so this will basically be my client over here right so once we give this Bedrock client that basically means it knows we are going to call this particular embedding model from this particular client that is bedrock itself in short we are going to use the AWS llm uh AWS Bedrock now let me quickly go ahead and this is what is all step by step we'll be going first we will create this data injection model see if this step is done we have created our client now we will go ahead and implement this data injection so quickly let's go ahead and implement this data inje and please make sure that you follow this so here I'm going to basically write write data injection okay and uh I will create a step which is called as data injection a function now inside this data inje what things you're specifically going to do right I have a folder so let me go ahead and write loader is equal to Pi PDF directly loader and here I'm going to give my folder which is called as data right from that data folder only I need to pick up all the PDF files right so this is the first step then I will go ahead and write documents is equal to loader. load right so from that this loader we are going to load this entire documents now in our testing uh we will specifically use something called as character text spit right so that is what we are we have already implemented right recursive character text plat now how do you implement this it is very much simple I will go ahead and write text splitter right and this with the Lang chain I've already shown you so it will be recursive text splitter and here first of parameter will be my chunk size chunk uncore size chunk size I will give it to 1,000 uh 10,000 right and along with this chunk size the second parameter I would like to give as chunk overlap please give this number a little bit more value so that you'll be able to understand it okay so this two are there text splitter recursive character text splitter and this right once we get this text splitter I will go ahead and write docs is equal to text spitter do from oh sorry do splitcore documents okay and then here I'm going to give my entire document so in short we going to split it right based on this recursive character text spitter once we do this then we will return the docs all the docs we're going to specifically return the docs so data injection is done right with ECT to this particular data injection here what we are doing we are reading all the PDFs from the data folder we are doing recursive character text splitter with this chunk overlapping and then we split all those documents right so this becomes this completes our data inje now the next thing is that we will go ahead with vector embeddings vector embedding and Vector store right and this is where we are going to specifically use that Titan embedding that we have imported along with that what we specifically going to do is that we going to use this F right so let me quickly go ahead and write it down so here I'm going to write my definition and quickly let's go ahead and write get Vector store so I'll create one more function now inside this Vector store I'm going to first of all give the documents whatever documents is coming from here because we take this documents and then we do the embedding techniques and we perform all the embedding techniques so in my next step what I will go and write I'll write Vector store underscore FES okay so this will basically be my variable and then I'm going to use fire start okay from underscore documents right and we are going to implement this right so in the first parameter inside F we give docs and the second parameter that we specifically give is my Bedrock embeddings right so this is the embedding specifically we are going to use okay one is the Bedrock embedding which we have initialized over here see over here in the Bedrock embedding we have initialized and that is what we are going to use it over here now after we get that Vector store F this is basically my Vector store I will go ahead and save it in my local disk so let me quickly write do savecore local and inside this we're going to basically write Fiore Index right so this will basically be saved in my uh folder over here itself right in my hard disk you can also see save this in any database as such if you want so this is my Vector embeding I've done it and this step is also completed now let's talk about the next step in the next step with based on the import we have to work with llm models now what we are going to do over here is that I'm going to create some llm models let's say I will go ahead and write first llm model that I'm going to work with is cloudy right so I will say cloudy llm and inside this cloudy we will create create the anthropic model because I should see already AWS Bedrock is giving you the power to harness multiple models to use the multiple models so we can specifically develop in this application itself it's just like having different different models in different different ways but here I'll try to make it much more generic so here we are going to write create the entropic model and I'm going to create my llm along with this we are going to use this Bedrock so Bedrock um before if I probably show you in my previous tutorial the way of invoking a model was something like bread ro. invoke model but now since we already using Lang chin we don't have to basically use invoke model so what we specifically do we just call this bedrock and you can probably see how where this Bedrock is present it is present inside Lang chain so they have actually created a wrapper and internally they will invoke that specific model but this is a way that how you can use this Frameworks like Lang chain in a generic way so now I'm going to use this model as Bedrock now with respect to this I will go ahead and write my model ID already I have searched my model ID so this is what is my model ID over here how you do you get this information again from this particular examples go and click on any model you will be able to let's say I want to generate Cloud right generate code so with respect to this this will be my model ID right so it is a generic way of finding the model ID so you don't have have to even worry about it right so I have my model ID which I did it in my previous tutorial also and then I will go ahead and write my client my client will be nothing but Bedrock bedrock and the next thing that I'm going to specifically give after this right one is model arguments right so model arguments with respect to this now this arguments this arguments that you'll be seeing right where do you find it again if you probably go over here inside the body and if you go at the last right there will be some arguments that will be added like this Max tokens to sample temperature this this you can also add it over here now with respect to this particular model that I am actually using this was the argument that was present inside that particular body so that is the reason I had created this right all this Json file separately so that I can refer it okay so I hope you're able to understand it if you're able to understand please make sure that you hit like till now and now let's go ahead and and here what we are specifically doing Whenever I Call this function that basically means my cloudy model has got loaded so I'm going to return the llm model now suppose similarly you want to probably go ahead and call the Llama 2 model so I will go ahead and paste it over here I'll say get llama to llm like this you can actually create any number of functions as you want okay so this will basically be my llama 2 now inside my Lama 2 let me just copy this again I'm going to see my so this is the Lama 2 model over here okay only the model ID change and whatever is the argument that will change so in the case of arguments uh with respect to this this is nothing but Max gen length right how I'm getting it let me show you again okay so let's say this is chain of thoughts let's say that I'm going to use this okay so here you can probably see this is my model ID okay and then inside my arguments this is from where we'll be able to get it right this API request is pretty much important with respect to this okay so return llm model and uh this is where is my LM model that I'm getting with respect to llama 2 so now you know how to create how to call any uh models itself right data Eng is done Vector store is done everything is done now let's go ahead and create my prompt template now this prompt template that I'm actually going to create I'm going to use something called as Lang chain so in my prompt template I've written see this okay so I've given a simple prom template which you can also use it so I've written human in this format use the following piece of context to provide a concise answer to the and at least summarize with 250 words with detail explanation if you don't know the answer just say that you don't know just don't try to make up the answer so context and question is there and based on this assistant whatever assistant output is there it will get appended over here now with respect to this particular thing we use the langin prompt template we write context and question and we use it okay now this is fine everything is good now let's focus on the response part so here I will go ahead and write getor response llm and here the first parameter is my llm the second parameter is my Vector store Vector store FES the third parameter that I'm going to give is my query right so this three parameters we are going to play with one is obviously if I want to get the response from a specific llm then I have to give this three information okay get response llm the llm model which I'm calling this is my f index and this is my query okay now once we get this we as you know that we have imported something called as retrial QA so retrieval Q&A here also we going to use it and this is how you basically call it right so I'm just going to copy it paste it over here again I'm seeing the Lang chain documentation so retrieval QA from chain type I'm giving my llm model chain type will be stuff I've already shown you what is text summarization different different text summarization techniques so stuff is there over here this is the most important thing retriever how from where the similarity search will basically happen so this is where we are doing right A F index basically has the entire index itself right and then we are trying to do the similarity search along with the argument top K3 top three prompts and then we also saying return Source document with respect to for this one and then I have my prompt over here right so all these things are basically done and then after this we will be able to get the response so now once we specifically write this chain type uh arguments and here we are specifically giving the prompt The Prompt is getting created by the prompt template over here and then in the next step we will go ahead and write answer I'll create a variable called as answer and we can use this QA variable and inside this we will give in the form of qu query colon and here we're going to give the query itself so this query is the query that is coming from the as an input okay and once we get the answer see when we when this QA is basically using right it is nothing but retrieval QA so it will retrieve the response and it will store it in this particular variable now inside this variable there will be something called as result as a key which will give you the output so if you print it right initially I printed this answer I've already implemented this so there there is a variable called as result which has the entire answer so this is my response llm where I'm getting the entire result now let's go ahead and create our streamlet app Now quickly there are two important things with respect to the streamlet app one place I have to make sure that whenever a document is updated you know that it should get converted into a vector embeddings so what I'm actually going to do quickly over here I'll create a main function and let me show you what I will be doing okay so quickly first of all I will import streamlit as St streamlit as St fine we are going to use streamlit now see this the thing that I've copied and pasted so I've written chat PDF chat with PDF using AWS Bedrock this is my user question like what kind of question I want from the PDF file then a sidebar is basically created and I will say update or create Vector store right that basically means inside this I will create a button saying that vectors update that basically mean once I click right then it should go ahead and call this data inje now data injection what it is going to do it is this data injection is going to read all the files from the data folder then it is going to take this load documents it is going to perform this recursive character text splitter and then after providing after doing this it is going to return this entire documents right after this we are going to call this Vector store so one by one we will be calling it so here you'll be able to see data injection and Vector store we are calling we get this docs now inside that Vector store what is happening is that we will save this in our local uh folder itself in the hard disk in the form of f syndex so as soon as I click this button that is Vector update which will be available in the sideways inside the sidebar sidebar on means in streamlet it is in the left hand side so the vectors will get created the vector stores will get created and it will also saved in a local folder itself right so over here a folder will come with this particular name that is called as F Index this is one step now the second step is that I can go ahead and create a cloudy output okay okay I will create a cloud a button which is called as cloudy output now see this this is important to understand okay so this is my button I've created another button when I say cloudy output that basically means I have to use the Cloudy model API so the first thing as soon as I click on the Cloudy output button my files should get loaded from the local so that is the reason I'm writing dot loore local with this F index and the Bedrock embedding the same embedding which I've actually used then I will go ahead and call this cloudy llm right I'm going to call this cloudy llm because I will get the llm itself now if I go inside this function if I go ahead and write F12 okay so here you'll be able to see where it is uh cloudy llm so cloudy llm it is going to call this particular model and return our llm over here right so here I'm going to get my llm with respect to this and this llm will specifically be used inside this function that is get response llm where I have given my llm Vector store FES and query right when I get this three information over here I'll get my response and I am writing this particular response over here very much simple very much easy right so let's execute this and let's see whether this is working fine or not okay so this is my main function so what I will do I will create this if uncore uncore name with respect to this mail now let's see whether we will get any error okay so I will clear my screen but in short we have executed each and everything right so I will go ahead and write streamlit run app.py let's see whether we'll get any errors first of all uh okay it's working fine I will close this okay now this on the left hand side I will get this Vector update update or create Vector stool and here I have chat PDF using AWS Bedrock okay now the first step as I said what I'm going to do here is is my data folder it has this PDF file attention. PDF yolo. PDF so what I'm going to do I'm going to probably click on Vector update as soon as I click on Vector update now what is going to happen see over here as soon as I click on Vector update what is going to happen so if this is clicked that basically means the data injection folder it will go it will read both this PDF and then it will convert that into vectors and we will save that Vector over here right now you cannot see any folder in the name of f index but now if I go ahead and click on Vector update now what is going to happen see as soon as this Vector update will finish now processing is basically happening and this will probably come up with like done okay done status unless until we don't get any errors let's see with respect to this right what will happen so it is now loading all the PDF it is performing it is converting those into vectors right and then it has now stored that as a fire index now if you probably go over here and see this folder my fire index has created it has index. F and index. pickle right now this is the first step now the second step is another button which is basically cloudy output now cloudy output is over here now if I give any prompt what is attention is all you need what is attention is all you need okay if I go ahead and click on cloudy output now it is going to call the Cloudy model right so if we probably go ahead and see the code again what is basically happening first we will load this F index from the local then we will call the Cloudy llm model then we will call this get response LM where we are giving all this three information the user question and now finally you'll be able to see I'm getting the output right now this is with respect to cloudy output right so similarly if I go ahead and write what is YOLO I will go ahead and click on cloudy output now again see follow this pattern inside it will go again it'll load this fast Index right and then we we will call this particular cloudy model now how the performance can be improved this FC index I'll not call each and every time so let me do one thing let me call this once okay let me do one thing for the first time I will just keep it outside this okay but again it'll give us an error but I will give you that as an assignment first of all let's work on this okay so here you can see my YOLO is also coming now see if I want to add llama 2 and all what I need to do I will go over here my llama 2 function is already created if you remember see over here my Lama 2 function is where get Lama 2 right so I will create another button I will copy it like this if my STD do button is llama 2 right so this will be my llama 2 output instead of calling get cloudy llm I will say get Lama 2 llm so now we will go ahead and check whether for Lama 2 it is working fine or not so I will save this quickly I will see now I'll click on Lama 2 now let's see whether we'll get the output or not again the same thing is going to happen first of all it is going to load the fire index after that it is going to take the llm model it is going to give all the information and then finally you'll be getting the answer similarly I can go ahead and write what is attention is all you need okay so if I go ahead and click on Lama 2 then again I will be able to get the response so this entirely everything is basically happening with the AWS Bedrock so step by step I have shown you almost everything the data injection step what all libraries are specifically getting used all this code I have written it in front of you the streamlet file we have actually created I would suggest go ahead and try it from your side and this is how you're able to get the response but the best thing is that all these models are available in AWS bedrock and it is already scalable it you can actually use it according to your wish so guys uh as we know that we are learning generative AI uh specifically in the AWS Cloud so uh today one amazing productivity tool that we can specifically use which is uh I think available for everyone and the name is Amazon code Whisperer so this is the page we'll talk more about it but uh if you just want to know what exactly Amazon code Whisperer does uh it's very much simple um I hope everybody has seen GitHub cop pilot where it provides General suggestion with respect to different kind of codes and all so similarly you can also use Amazon code Whisperer which is also now called as Amazon Q developers okay and this you can probably use to again get all the suggestion with respect to various codes and all I will show you in this particular video how you can enable it in your V vs code and trust me this is amazing productivity tool altogether to do your work very much faster and quicker uh at the end of this particular video since I used both GitHub co-pilot and Amazon code Whisperer that is nothing but Amazon Q developer so I will be making a brief comparison about it um you know most of the functionalities overlap but uh if I talk about Amazon Q developer it is specifically for some purpose so when you are actually developing and let's say you are probably using AWS account a lot you are doing all your works specifically to AWS then Amazon Q developer will be the better code suggest for you you okay so let me just go ahead and show you how you can probably enable it uh for enabling it it is very much simple what you really need to do is that just go to the extension right and just search for Amazon Q okay so Amazon Q over here if you're probably suggesting if you're searching you'll be able to see this right now I have already installed it and you can also install it uh let's see whether there is something called as Amazon C developer also I hope so we find it out if it is not there then we can specifically use Amazon Q so this is the extension that you really need to install and understand once you probably install you know in the left hand side there will be a box that will probably be opened wherein it will be asking you for the information uh regarding your AWS account where you really need to log in just and trust me uh no credit card will be also required you just go ahead and log in into this particular account with respect to your own personal email ID uh it is also called as AWS Builder uh which will specifically provide you the access to Amazon Q okay so there will be a page that will be probably coming up over here uh It'll ask you for the username password and it'll tell you to authenticate from the AWS account itself so once uh that step is specifically done then what you can do you can just go ahead and exit this particular notebook and uh exit the vs studio and restart it okay once you probably restart it now you see the power of this and I as I told you right I've been using both GitHub copilot and and U uh this one that is Amazon Q developers trust me both of them works amazingly well whenever you're specifically writing the code so just to give you a demo right now my extension is enabled uh and definitely use this for your coding purpose it will be very much helpful also so first of all I will go ahead and write down the comment let's say write me uh write me uh python code or write me a function to perform write me a p py write a python function to perform to perform binary search okay so I'm just asking on simple binary search okay okay now this is the comment that I have written now you can see Amazon Q is generating right now it does not show you something right over here and also it depends on internet speed both respect to kind of suggestion and how quickly it also gives you so if I go ahead and just write definition over here and you know that the function is going going to get created automatically just press Tab and all the entire suggestion will be in front of you right so say just understand like how good the productivity basically becomes right uh if let's say for the first time I I'm planning to write this particular function right obviously I'll do some amount of Google search um yeah I'm lazy I'll not remember each and everything and definitely unless and there is not an interview I think you you will also not remember it right so over here you can see just just by writing a comment and probably doing it or automatically all the suggestion is probably coming up right so this is really really amazing right um let me just go ahead and do one more thing okay some more complex thing related to generative AI okay provide me a streamlet app to call open a apis okay so let's say I am going and writing like this so the first thing is that I will uh specifically go ahead and import streamlet as St okay now let's see import open a I'm not doing it anything all the suggestion is probably coming from you know the Amazon Q developer over here right so same thing has got imported but I think that part you can probably fix it if it is not providing any suggestion that basically means it is telling you to give one line of code one line as an empty space and again now see not if it is not suggesting you something just press enter go to the next line and again it will start suggesting you right so it has probably created this message varable and now I think if generated oh session State also it is basically using which is quite amazing okay um okay past sessions everything over here it looks good again uh same thing I think no uh this is also making sure that I think it is also saving this prompt in the session State let's see if user not in session State how can I help you okay perfect I think think this should be working fine but again unless and until I don't execute it there will be some amount of errors that will specifically be there now see U now what this has basically done is that it is mostly hardcoding things over here um you know mostly hard coding things right with respect to my name is Chad GPD and all so I think it is not doing that amazing job but with with respect to GitHub copilot that I found right uh it was able to give me a good response and that is where I'm going to make a lot of comparison okay so let's do one thing let's remove this again I think it is it is in that hallucination mode okay let's go to the new line or let's come after this particular function let's see what what suggestion it will probably give us okay now finally it is probably given this entire generator response function uh what you can basically do is that I think I'll not use even this session because I just wanted a simple uh this one right uh so this is the function that I've actually created and you can also guide it just not based on the suggestion you should probably go and just press tab right so generate responses over here here it is specifically communicating with text T C 02 um let me do one thing let me also go ahead and uh make sure that create this open API key okay let's see open AI dot API underscore key and I will probably go ahead and set my API key over here uh after setting this particular API key let's see what kind of suggestion more it gives right so this is my messages return message okay so perfect I'm getting from this particular function my entire message this is my main session State again session State then again this message is append okay perfectly it is appending right it is appending with respect to the different roles and all so with some amount but see at the end of the day I'm not writing much code and uh again what we see and why it is not not that better when compared to the GitHub co-pilot because I'll tell you a specific reason why it is not okay okay let's try one more thing okay so here I'm going to write a comment now see how the performance of this particular code will become better by using Code whisper or Amazon Q developer write a streamlet python code to invoke AWS Bedrock AWS Bedrock Foundation models okay I'm just going to go go and write this right let's see import stream late as St boto 3 okay perfect it is coming up okay perfect st. tile again a stream late get the model name from the user perfect so enter the model name okay I will enter the model name what model name I specifically want uh enter the input data okay I'm splitting the data then input data will be this let's see what it is trying to do let's see input data finally I get over here print input data invoke the model okay perfect see boto 3. client is basically used to invoke sagemaker runtime or you can also use it for AWS Bedrock okay so model name body input what I feel it is good enough get the predicted output see with respect to the boto 3 we can basically just change this instead of writing sagemaker runtime I can write Bedrock runtime okay so predicted response body good enough and this is basically my predicted output I guess so it is right enter the model name enter the input data so some amount so here you can probably see the code is well structured and what I feel is that see when I probably write a code and if you have seen my AWS playlist where I'm specifically using Bedrock right U you can probably see it follows most of the similar structure so if I probably create a function if I try to write it much more in in a better way and I cannot just depend on the entire application on this right so here uh it has done a pretty good work right and that is where see why with respect to only AWS work it is doing well because this entire product is from Amazon Q developer is belonging to AWS right so uh just to make a brief comparison right or let's see I I'll just trite like this okay uh Define a function to invoke uh ews now see see this Bedrock models okay and that is where you'll be able to understand so I'll write definition okay and uh I will try to create a definition where I'll say call Bedrock model done see this automatically all the things are done only the thing that I really need to change Bedrock model endpoint Define the endpoint Sage maker runtime instead of writing Sage maker runtime I need to also give the model name so model input is over here I have to probably create this payload from my side which is nothing but the same model input which I will be providing to this particular function now just to make a brief comparison between GitHub co-pilot you should understand that GitHub co-pilot is specifically used for general purpose and I also have that specific exess if I probably consider Amazon Q developers um assistant that we have code assistant that we specifically have it is used to do or it provides you good suggestion with respect to any kind of AWS task that you are probably doing in AWS itself so just to make this brief comparison so always try to make sure that uh it is good that you should know this but again at the end of the day which is the most generic model uh GitHub co-pilot that is a kind of suggestion that I would like to give you to both are trained in huge amount of data but definitely if I talk about Amazon Q developer it is your generative AI powered assistant across specifically to most of the aw services