hi there data scamps and data champs this is Richie now before we get started I just want to say it's pretty amazing the uh comments here are started by someone called starter Joe that's felt very impressive to me uh so hi Joe hi to everyone else who's joining uh I'm very excited for this session uh and the reason for that is that AI agents are the hottest of Hot Topics right now and the idea is very simple every job has boring bits that you don't want to do so getting software to do those bits just saves you a lot of tedium uh the idea of automating stuff with software has been around for decades but now that we've got decent generative AI it means there are suddenly a lot more tasks that are more easy to automate and in particular I feel like report writing is no one's idea of a good time uh so that's what we're looking at building agents for today maybe that's a little bit harsh I know like a lot of people who business analysts they have like reporting as a fairly large component of their job but actually having some AI assistance there is going to take out a lot of the uh the grunt work and mean you can focus on more of the fun stuff uh so uh that's what we're doing today we're going to build some uh agents for financial reporting um one warning though so the agents we're going to build they're a little bit more sophisticated than the things that we usually do in these code alongs so I'm suest today is probably going to be more of a watch along for most people rather than actively typing code so as you're going through this I would say uh focus on like understanding like what's possible with AI agents and thinking about how you can maybe use them in your own job particularly uh if you write reports and then also um yeah I would think about like uh trying to understand the architecture like the the the code patterns used here you can always get into the details and figure out how the uh the individual lines of code Works uh later later on when you're playing around yourself so uh with that said uh let's introduce Our Guest uh so uh today's guest is J putat tunder uh she's an award-winning lead data scientist and director at the financial information company fit group and she uses generative Ai and uh NLP to create products for credit ratings so uh J was listed as one of the 100 most influential AI leaders in the USA uh in 2024 by AIM research she's also the New York City chapter lead for women in Ai and an ambassador for Stanford's women in data science group uh so uh highly credentialed uh guest today uh take please take it away Jita thank you very much Richie and thank you very much sh thank you for giving me the opportunity here and for the very kind introduction uh I think like you very correctly said and welcome welcome everybody whoever joined uh thank you for being here um I think AI agents are a topic that everybody wants to learn about but we want to start in a way that is observable or that that's implementable and it's not like too uh complicated infrastructure because of which we feel like oh it's too hard and maybe not for me so that's where that's why I'm here to build it uh step by step using open source dech um I don't want to like give an introduction again for myself Richie did a great job so let's move along we have one hour um and we have a lot to cover okay next slide please so building Financial agents right um we're going to talk through couple of slides here just to give a little bit of background because I feel if we start coding without like you know understanding uh why we are doing a few things the certain way or what the workflow looks like or what the problem we are trying to solve it sometimes leads to unexpected expectations because we all know uh with tech especially with generative AI uh expectation is through the roof that is just the magic pill that we need to swallow and everything is going to solve by itself so that's not right let's build the solution step by step in uh uh the right way and then try to solve for it so we're going to talk a little bit about AI agents versus agentic AI I'm sure you're being hearing this uh uh comparative terms back and forth so we'll understand a little bit about that and then we'll look at a couple of examples from the financial services domain that we feel where agents can definitely add value and then we I'll go through the code uh collab notebook to actually show you like four different kinds of agents that I have uh like Richie said I think for this one one I would really like you to take the time to listen in and without like worrying about the coding side of it because you are get going to get the collab notebook it's completely open- Source take so you can run it on your time and you can always feel free to like reach out to me if you have any questions but for today take the time to understand each of the step and then we're also going to talk about evaluation Frameworks because without that there's no way we're going to put uh any agent or any Services into production before I start uh I think for every Tech there's like some kind of hyp hype cycle that we go through uh with generative AI it was like peing then we kind of inflated and now we know exactly and which areas it can add value for us similar for agent AI everybody wants to build agents but let's identify use cases that actually adds value to the business rather than you know just trying to fit like an agent even when like where a traditional ml model can fix your solution so let's not over complicate and over engineer let's go problem first approach um so AI agent components right what are these are the three baseline or basic components that we're going to talk about today you as a task or a user uh have a request that you want to send to the agent uh and what the agent is doing is just helping you automate some of the processes or like you know sub layer of work that you were doing maybe sequentially or like parallell but you were orchestrating it yourself or I can say like manually orchestrating it now it is possible to kind of chain together some of these comp components so that the entire workflow runs a little bit in an autonomous way but sorry but I would use the word autonomous in a very careful way it's not that we are automating everything end to end and there's no human supervision that's completely not it because at each layer and at every uh evaluation process step and framework there needs to be uh human oversight and we're going to talk about that in the framework part of it so we're going to talk and like look at what tools can help us do uh a couple of examples being you can do external tool calls say you are reading a newspaper and you don't understand a particular word what do we do we look up the word in Google right so instead of like manually doing it for a organizational workflow for your personal use case please feel free to Google but for the organization workflow we're going to use uh like a tool calling function that would help us look up some of these Concepts write a summary Identify some links all within the code without you being uh like creating a separate workflow for that memory is again like you know the database the knowledge base uh the context that the a large language model has about your problem about your uh uh instructions or whatever you have given it beforehand or the chat history that you have been conversing on all that comes from the knowledge base and the memory base and then uh planning planning helps uh the agent break down complicated instructions so if you are saying that hey book a ticket that's just one goal right so how to go about 0er to one to actually go book a ticket that's the planning component of it breaking down step by step of the workflow and then kind of uh going one at a time asking uh or like utilizing tools or memory in whichever flow it's required and then the task is complete it's as simple as that there are a couple of things uh this this kind of shows like all the components you see on the right here interaction with the environment so the agent ability to kind of respond to external functions is nothing but like the tool calling that we just discussed there's of course data collection because you need to gather information from various sources it can be your knowledge database that you have at hand it can also be internet search like we talked about identify the latest uh you know 10 papers that were released like yesterday and this is actually a personal agent that I have because you know in the space every day there's like thousands of papers that coming up and I want to like only read about couple of uh uh Niche content uh genres and I have set up like a very small uh agent which kind of Pinks me every day by giving me that these are the 10 um top papers that were released in in the last uh week and then there's of course task performance uh we want to achieve a particular specific goals and the performance is measured by you know the the evaluation framework that we're going to talk about on the left what you see is basically the the characteristics of the agent uh is that we have a little bit of autonomy like I said we will independently uh operate at each step but there will be some kind of human intervention at the each layer to make sure the results are right or we are going the right way and uh uh you'll also see that in the code so and like we talked about planning as well as reasoning uh reasoning has become a big part of uh solving complex problems because we have seen that sometimes when you ask a very big question or there's like a very big prompt with like a lot of instructions there tends to be situations where there might miss some nuanced components so we need to make sure that when we are doing this planning part we are breaking down components in a way where the llm would be able to pick up reasoning and say that hey my step one planning was to for the ticket booking example uh go to a website I went there and after that my process is going to like look for the keywords so if you want to go to like I don't know like say maybe you want to travel to Canada you're going to look up like uh airline tickets or Canada and that's how the reasoning goes step by step in the in the planning process to set set us up for success one last point to highlight here AI agents versus agentic AI what we're going to build here today is the AI agents for very specific tasks so we are task oriented we have limited autonomy here and whatever the agent is learning is the Baseline learning that it's getting from the instructions as well as the knowledge base that it has access to it is not automating or it is not Auto optimizing at this point for the at the AI agent level uh its output or its performance that happens at the agentic AI where I'm sure you have come across uh you know the open AI that released swarm uh that is kind of a more goal oriented rather than a task oriented uh framework it's higher in the autonomy side and it's also advanced in terms of like how it kind of um uh relearns and then applies automatically uh and optimize its performance on the go uh that is not it so I've seen people like talking about AI agents and agent take AI as if they're the same concept so I wanted to make sure that you understand this of the bad um so how do you choose how do you choose which is your best approach and what we're going to do today is we're going to take a problem so we go problem first not fancy Tech first I know our uh human nature is to jump and get into shinier Tech I do that all the time but I think uh when you try to solve a specific problem from the business side you're very cognizant about how it's actually helping the business solve some issues rather than what Tak you're using so AI agents we're going to start looking at uh how we Define this is a very generic framework of how you start thinking about it let's not go into it but talk about the applications in the financial services right uh if you see the chart here it talks about some use cases that needs to have like very uh High real-time response like you know customer service uh and those kind of automation agents if we build but there are some like it might not need that you know Split Second responses maybe uh compliance monitoring it can take a while to identify all the sources all the knowledge and then kind of come up with the solution um so mapping your use case um uh and identifying um identifying processes to kind of uh see which works when uh really is helpful here um so let's talk about couple of these examples so streamlining operations it can like help you with automated repetitive tasks uh for customer engagements like if you want to have a very scored Financial advice specific to the customer and you have all the data that's sitting about the customer about their preferences likings in your database you can map that to kind of create that adaptive uh uh Robo advisor kind of a agent that caters to particular customers process um yep that's about it I think feel free to look at this uh when you have some time or go through it and let me know if you have any questions um let's talk about today's use case right this is our use case that we're going to try to solve for today what we're going to do is that we are enhancing some of these uh financial analysis that we would run on various PDFs complicated uh reports U and as you know like Finance in finance each company releases thousands of documents it can be an annual report a 10K um uh ESG report so how do you kind of make sure that you are creating a pipeline that can handle all these various uh uh document types and we're going to talk and we're going to utilize ASG report today because it's very dense um in in content in text and uh we're going to see how we can work through it um so this is the uh this is the agent workflow let's not worry about that too much it basically talks about the four sections or the four agents you're going to see in the code uh you have an input layer where you are inputting you know any of the financial documents that you need and then there's the user's query or the request that's coming in as like a PR um the the the this area of a rag agent that you see is what you're going to start with first because we want to optimize some uh some database uh that kind of holds all the chunks and embedded uh information from the PDF and like build this relevance engine to kind of understand or like uh process the user's query then we have a research agent where uh we will work through uh how do we search like do a web search for a particular query how do we create some context around it and then Aggregate and like create a report uh this report generation part also helps us kind of get at some level of understanding of what we found from our uh initial uh uh Searchers and then sets sets up for uh a process of yeah a process of like getting started with the the process so all the solution uh name that you see here is what we're going to use for our codebase uh they are completely open sourc there is no like uh we were saying in the beginning of the process there's no uh paid services so you don't have to worry about getting paid paid apis you need to create some apis and we'll go through that but there's nothing that's paid here so uh we'll talk about so let's start coding right we've talked enough so I'm just going to uh shift to my next screen um okay okay um can we make this bigger I think we can make this bigger okay that's about it I think I should have okay this is better okay so let's let's start um so the code base talks about the goal what we're going to do today we're going to build couple of AI agents that can handle various Financial tasks uh we are creating four agents here one for rag based query which is rag is retrieval augmented generation where you have a v Vector database you have a question the vector database holds all the information about the documents that you have passed and you can query that through to get your answers and we will see how we're also going to build another one like tool calling for web search the other one is uh for a a deep research I think this deep research we're going to do for a couple of stock uh processes and then we also have evaluation where we are going to do llm as a judge agent just to see how another llm agent kind of uh um scores a particular agent's answer and that is very crucial because if we want to scale sometimes manually reviewing there's always a human intervention but manually reviewing uh code or output can get tedds so we need like one layer of automated outputs that we can then uh review with our subject matter experts so very quickly I want to show you because we need two API Keys uh one for Gro which is the fast OS uh open source like LM inference it's it's absolutely free what you have to do is just go to gro.com uh you can create uh we you can create U what do you call like you can create an account uh once you create an account and login it will look like something like this uh you'll see that there's a get started overview with a lot of documentation lot of you know uh cookbooks and a lot of U you can also join the developer Community they're very helpful if you have any questions or you get stuck you can talk about you can look through there what you need to do is create your API key here I already created my API key here called Finance agent uh and you've seen I have a couple of calls there if you want to create a new one just say create new one enter your details and it's going to be here so we'll use this secret key uh in our notebook so that it can call um all the models that we would like to use and if you see models here these are all the production open source models that are hosted uh in Gro that you can directly utilize without having to set them up on your own system or own machine which you can still do it but this just gets you started with like an easy access right it reduces the stress a little bit when you're just starting to experiment um okay so we're going to use couple of models here uh we're going to use the Llama 370b as well as here the Lama 370b as well as uh the instant model for our evaluation agent uh but you can choose and test with like whichever model you want to use there's also quen deeps all mentioned here I think they're still preview uh they may not be like in production yet but start with something simple like the Llama 7B 8B or 70b whichever you're more comfortable with and then we are good to go the second framework where we need an API is uh agna they are a fantastic uh open source framework for creating or like you know chaining uh the the entire agentic workflow if you have not uh heard about them or look them up please check them out today I think they used to be called as fi Data before but they have rebranded as agno and I really enjoy like uh learning from all the cool things that they're building uh so what you're going to do is you're again going to create an account from from here and once you do uh I think once you do why am I not logged in oh I'm logged in so once you do you see that here's an API key that you can set uh this is my API key you can create your own or you can refresh uh and you're going to uh be able to access one good thing about is that you also have access to their playground so if you say don't want to run the code code that you have like from this session first but want to rather do a playground demo there you can also like check out their demo agent and like you know kind of uh uh go through it okay so there's also the doc sagn not entire documentation you can go to the like you know look at what their key features are what kind of Concepts like what kind of multimodal Agents you can build it's very uh very descriptive and you should be uh good to go on that aspect okay so let's come back to the code uh those are the two API keys in Google collab uh you know how you set environment variables through here and don't see mine I think it's already blocked out okay so you can uh set your API Keys here like this uh from the secrets tab add the value hide it and how you call into your system is like this um where you say that I was just testing with different keys so it's still there no worries so you call your environment variable um uh the groc API key and fi data that we just save in our secrets and you get the user dat.com and then I just printed it out saying that okay API skis have been set sometimes while running we see that uh you know it might um there might be like API error calls so another good way to check and I've just commented it out but feel free to run it is that to print like to actually make sure if the collab notebook was able to fetch the key from the environment you can also run this command and it will say that if it has founded or if not it says that gr API not not set that just for your uh peace of mind that okay we're going to be uh you're okay with all the API keys so let's talk about the first agent right what is the first agent we're going to run like a functional tool calling capability which is a web search and we are utilizing this solution called Doug Duo it's a toolkit uh uh it's like a private searching interface uh there's some other other uh other components as well and you can find everything uh regards to that in uh the AGR uh agnos documentation but let's start so what what are going to do we're initializing the first uh uh you know agent by calling our model that we talked about we're creating like an agent component we're calling our model that we want that's the Grog one uh and why did we put ID as Lama 70b because that's the exactly uh how it's mentioned here this is the ID model ID if you see so you choose whichever model ID you want from here and you kind of use it here uh for tools we're also using dgo uh tools as well as newspaper 4K tools and if you see here it's just like a descriptive name for the toolkit it just helps you uh uh look for news so uh what what um description I'm setting for this particular agent is that hey you are a um uh research analyst in the financial services domain and you can change it up you can say that hey you are a chef or you are a uh food blogger or you are whatever you want it to be so you whatever description and instruction options you set for the agent becomes its context and that's how it kind of runs uh and I've added a couple of you know layers of expertise that the particular agent should think about uh when it kind of answers so they should have like a deep deep investigative Financial research and Analysis knowledge why is this important because um I think uh I think the goal is you're setting you're setting like a standard or a bar uh that that at which level the agent should uh navigate or Implement their answers and why am I saying that because you will see in the evaluation framework we will use exactly these descriptions and instructions and checking the output based on that saying that did you meet the instruction requirements if I asked you to say an instruction is that if you have uh if you found any dollar value in a particular text translate it to Dollar like spell it out so that's you know uh an instruction and in one of the output we see that the L missed it or it was still showing up as like like a dollar sign and not as the uh not as the script that's going to be one of our evaluation uh uh checks and balances as well just to make sure that it's operating with the way that it should so make your descriptions and instructions in a way that's that's executable as well as uh we can evaluate on it so it has like data driven reporting and visualization expertise uh it can also you know understand Trends whatever you can right I added some of these just to make sure that it gets some kind of uh uh Baseline understanding so like ethical practices as well as give us like a balanced perspective not like you know sway to the two extremes but give us uh uh your balance perspective in the instructions what we are doing is actually breaking down the task that the agent needs to perform in a step-by-step basis like I was saying um you can you can like skip this or you can like give one big paragraph and it would still run potentially but when we try when we try to build like large scale solutions that has like you know I don't know like three or four pages of instructions running we cannot use that long of a context uh as is it needs to be broken down and and that's all the concept of prompt optimization right I think that's another huge topic that we can cover in another session but uh uh prompt optimization is very very important to make sure that we are being to the point we are not using um repeative words or we are giving instructions in a clear uh format and like a step by-step sequential task based uh uh framework rather than giving like a big chunk of information out there that we don't even like if a human reads it it's going to be hard to understand what's the flow of the workflow so we're we broke it down into research phase that okay based on the user's input or the question The Prompt that it comes you have to give five alterative sources there's going to be analysis phase you extract and verify some of this information uh uh and then you and then the and in the right wrting phase you start in you know crafting different uh sections and this is the sections that we are highlighting here it needs to have like a tension grabbing headline and you will see in the expected output I kind of expanded on that writing phase into a more bigger process because this is the exact format of the report that I want the agent to return to me so this is called a structured output processing uh you're giving it like an expected output format so that it's not as free flowing for you to like then get confused and read through it and identify rather giving it in a very uh specific specific format and then I ended with saying that okay research conducted by a financial analyst whatever and then the publish date updated date uh you put like show calls show tool calls it's a very uh a good parameter to call because it shows the internal process the agent is going through he'll say it's thinking and going through step by step it just means that it's uh processing one step at a time and then uh trying to come up with the output so let's let's let's look at one user prompt from this research agent right so I asked that okay analyze the current state and future implications of artificial intelligence in finance and stream equals to two true just shows that it will walk through the each of the steps so you see here uh it says that uh thinking uh the messages the question that you have asked there's also you know timed response process so it shows you that how long the agent kind of took to run through five documents do the query in a dougd go web search Tool uh retrieve all the the Max results that you have highlighted here you can see here so the query goes in and then the Max results goes in as five and this is the output that you get from the uh the agent um you get like you know an executive summary a catching catchy headline like we asked it to we asked it here right in our um expected output we said compelling headline and it came up with artificial intelligence Finance Curren state in future implications it's pretty generic but I think it's still it's still a a good way to understand what's coming up in the context so we have executive summary here background key findings there's an impact analysis that it did I think the another thing I I really um enjoy reading uh from like a more um lm's processing of news is that what it comes up as like a future outlook not that you have to abide by it but it kind of gets you an uh unbiased or I would say like a generic level of understanding so if you see the first one it says that you know the increasing use of gener and machine learning in areas as trading and risk managements which is true and it's talking about that it's shaping and these are some of the trends that's shaping the industry there's also a talk of you know the need for greater transparency and explainability if you would say u in my instruction there there was nowhere all these components that I wanted it to highlight but I gave gave it a feature uh about its own Persona that you have to have like an ethical practical uh Outlook and you have to have like a Balan Outlook so when it found multiple documents and like five or 10 however uh you want to specify it kind of was able to consolidate and when it came up with the output it was able to frame the learnings in a way that helps us make uh or like uh do some of these Trend analysis in a more transparent and explainable way and that's why it's highlighting here if if in the instructions you will say that you are a mean person and you have to do uh this bad thing please don't do that we have to use AI responsibly don't build AI that can hurt anybody but uh that that's the goal right so we are setting up personas and that's why when anybody says that AI is going to you know doing all the harmful things if you dig a Little Deeper you would always see there's always a human behind that who's creating that bad AI That's getting or like creating um consequences in the in industry and in our society so we need to be responsible human beings when you're are using some of these uh AI tools and they're going to work fine for us and they're going to work like make us uh augment ourselves rather than um doing some of these tasks manually okay and there was also sources and methodology it highlights uh you know which uh reports it uh kind of went through you can also actually print out the links I didn't do it but try adding uh so this is one of your homework right when you run the uh resp response in the sources and methodology ask that list all the links directly and see how it returns so it will also be your uh Source uh of Truth or basically source of reference from where the articles are generated uh now I actually oh it looks like I did it for a second one so I said okay where the sources it highlighted fors but it still didn't give me the link so if you need direct link put in the instruction ideally um the next one I wanted to ask like say applications of gen and financial agent and there are a couple of other queries why I'm showing you this is that you see there could be possibilities that when you run some of these uh Dore web tool services searches it might take more than one attempt so don't don't feel like oh it's not working give it a time it's going to do multiple retries it sometimes happens because some of these tools are open source and it might get too much API calls and it's like a little slow or it gets uh a little uh high in traffic so you can add like time sleep or any kind of uh pause there so before it kind of reruns and tries to uh run the query or call The Tool uh so you see that it was it was able to identify or like you know run my analysis in the third run but here it says that oh warning couldn't could not run DD go and this happens because like I said U because it couldn't reach there was a rate limit because if you're doing too many calls at a time and that's an issue for open source T so you have to use it with a little bit of uh Grace uh but but nothing to worry about it still runs and there are a couple of more other examples please feel free to look through it I'm going to move on uh on the in interest of time but hope that was helpful that kind of shows you what you can do uh uh you can like query up uh names uh Technologies anything that you want to learn about and kind of get a quick report out of it um and then uh look through the sources um and uh okay let's talk about the agent two right so agent two is going to be more of a knowledge based query capability which is rag uh retrieval augmented generation so all that you saw from the previous agent there was knowledge but that knowledge was not stored anywhere it was just coming from an internet search or like a um or like a uh on the-go search but think about if you are like uh somebody who works for a particular use case you have tons and tons of data uh already saved for that particular use case so and you want to utilize it so how would you do it you would do it through rag which is a very common I think technology right now everybody uses rag we're trying to make advanced rag now we we are doing agentic rag which is what this is and there's also causal rag which is also one of my favorite subject uh but uh but that's for later uh another thing you need to do and you don't have to do anything just run this uh it's very easy if you are running uh the docker in your system you can just you know open up the terminal and just run the docker image but in collab it takes a little bit of uh uh processing so there is this concept called UD Docker that runs that was created on top of Docker that to run in collab so you can just show code and then just uh just run this cell and you would see that Docker in cab 1.10 uh version would be installed um we're going to be using PG Vector um uh for our database today it's an open source framework uh one thing you would see in most of the uh tutorials is that even if they use open source framework there's always a call to open a because they're using open ai's embedding uh process to kind of process the document and embed its information but again that's doesn't serve our end to endend OS operation pipeline here so what I did is that instead uh use sentence Transformers from hugging hugging phas which is like an open source uh embedding model and we will see how that uh set up but before that we need to initialize our PG Vector uh it's very simple like I said from your local terminal if you're using a Mac or Windows just do it there and then like uh run maybe using a vs code instead of running the collab however you want to do but for cab this is it so you start with like installing the ud doer you allow root this is a key uh tool if you if you don't if you don't um if you're not following this doesn't matter don't worry about it just run the script and it will set or make sure that your collab doesn't have any pre-existing um directories uh make sure that you um and it'll just make sure that the database is set up so this is basically for running the PG uh Vector uh container this talks about the environment that you're going to set up uh the volume where you are actually loading uh the postgress SQL database and where it will publish so this is the database URL this this URL DB URL is uh just just copying this framework and this is given in the Agro docu Agro documents um we are just working around that to make sure that we are able to host U our datab Bas in that so you don't need to change anything this will work as is uh just make sure um that uh yeah your collab is working it's connected it's connected to a Python 3 compute like you see here before in in the below and then you can get started so it'll I've created like a Max Treet try of five like before I said sometimes it can get uh stuck and the port might be busy or it might be you know some kind of failure might happen so like always have like a connection attempt multiple times it'll try and then create through it uh like I mentioned here uh I applied a 30 second sleep time so it's waiting for the database to initialize this is just to make sure that you are not calling the test DB connection to see if the database was created before it's actually created so it's always good to have like a Time sleep whenever you are creating or calling any external tools or even like setting up uh database connections um um so here so in the engine once we test the database connection um uh when we like test if the database is ready for rag agent initialization you would see that uh it says it was able to connect in the first attempt and it was connect uh successfully connected to postr SQL I want to highlight here that this tool may or may not be the right um database to use for your specific use case why we are using this here today because it's open source it gets the task done even if it's not 100% accurate for uh retrieval but it gets you to the step where you can start your PC look through the documents see what the outputs look like and then once you want to actually put it into production you have to use some production grade uh Vector databases like you can use um um that's available to you right so I'm not saying um this is the this is the best but this will get you started so in the next process we are saying let's define a class called document oh my God I spelled document QA wrong anyways uh document QA so so what we are doing is we are initializing an open source sentence Transformer embedding model uh this is nothing but and for all these things that we have actually used everything is linked on the sorry I'm just going to go up just to quickly show you so that you know where to look for everything that we use it's mentioned here in the open stack text uh text T just click through the link and you can like read more about if you want to learn more about the Technologies okay so let's go back sorry about the back and forth but just wanted to highlight that okay so here since uh and and this entire class you can omit if you have access to openai API key you can just call the open IPI key and do the embedding but it's a paid service but instead of that what we did is that we created this class we created initialized our class by saying that hey this is our DB URL that you have to uh ping to this is our chat model that you are going to use which is the again the the 8 billion model that we're utilizing um and then we're creating this embed model from here right um again if all these class formations looks complicated don't worry about it what I'm trying to say here is that you can do it using any open any uh you know closed Source or directly calling open a but this is just to make sure that you can run through the process yourself so it's doing nothing but just calling the sentence Transformer the paraphrase mini LM model it's a very powerful embedding tool and I actually use it for a lot of my production grade uh tasks as well because um um uh what do you call because it's always like good to like you know save some dollars wherever you can if the task can be done with smaller uh uh and a little less powerful Transformer uh embedding models doesn't doesn't harm at all so we are creating the get embedding and the usage calling the embedding model and then we are saying that print like once it's initialized successfully so you would see once we run it it says here that the embedding model sentence Transformer was initialized the next part of the process just talks about how we are loading the PDF URL this is just the function where it would help the user input any PDF URL link the PDF that we're going to use today is okay let's finish through this and we'll look at the PDF document and how the structure looks like so we're going to uh upload the PDF URL knowledge base this is actually part uh the function is part of the agno database so please feel free to read through it to get more insights these are some of the parameters that you can the URLs itself the vector DV that you have set up uh and the embeddings that you have just created in the function above it we're initializing the agent here uh where the agent will call the knowledge the knowledge where it'll call the knowledge base that you just created and then the chat model from the model the gro model lva model that we created um so we loaded the knowledge base it says that okay the knowledge base was successfully created and if there's any error it's going to show through it so let's look what it looks like once we run that um when we initialize that class uh into rag QA uh parameter here it'll say okaying model was created um it's you'll see that it might ask that you have to have like a hugging face token it's actually not required but it's a good practice so that they also have the data that you know who's utilizing some of the resources but you can just like do as is uh you see that all the model configs files the sentence word config files everything is downloaded the tokenizer these are all internal model uh uh weights for the sentence Transformer model so you would need this to kind of run your embeddings uh when we do it here so now this is the link that or the PDF link that we'll be utilizing uh for our uh creating our knowledge uh database so let's look at the document right so this is nothing but a very public document you can look up it's like a ESG report released by um Apple uh the the goal of this here is that if you see there's like a lot of text right there is report highlights it talks about all the initiatives that it's running it talks about uh you know what are the goals that they have or like what are the progress they have made met you see achieved ongoing ongoing so maybe one of the question you can ask if you are using this document is that um list all the goals that are currently in an ongoing stage um that gives you a quick access without maybe reading through I don't know like 113 Pages um of the document so that's a plus right so yeah that is the document feel free to read through it or like you know use any other uh document and see how it works so once we run or like you know pass that in our function load PRL you see that it it kind of prints through that what it does sequentially step by step so it's creating the collection it's loading the knowledge base it's actually reading the PDF documentation and it inserted uh multiple documents that it found in the knowledge base and successfully noed the uh the document uh when you this is a very uh key Concept in Vector database when you see um when you see uh that there are they're saying that 131 documents it doesn't mean that there were 131 Pages here sometimes document means like a chunk that was processed from this big huge PDF uh because there is context issues there's like you know processing issues so if a page or if a paragraph say runs for like I don't know five pages if there are something like that it may or it it's possible that it needs to be broken down into junks and there's a way to kind of um um create custom chunking strategies that we do all the time in our ADV Advanced R rag processes but we will skip that for now um so if you can ask all these questions like key points in this report given five bullets so it found like here are key points from the report that it has established some kind of safety program it also requires Supply staly materials and you know believes its responsibility extends Beyond its operation to supply chain community and planet and all that so basically you get like a very quick five-pointer um analysis of what this document is talking about you can also maybe if you want to create executive summary in 100 words because we know ex summaries um needs to be precise and concise uh you can also do that and say that okay uh here is 100w summary and then it talks about a little bit so you can review the output ask more questions like I said maybe ask ask like uh what are some of the goals that's achieved and see how it works and if we need to make more updates uh maybe POS possibly that could be done the third one is very interesting right I don't encourage anybody to follow um and saying that very clearly uh that um uh we don't need to uh you know actually okay I think we're running pretty tight on time so let me cover one other aspect before we go into the uh the stock is that um sorry let me take this out from my screen okay here so let's let's talk through very quickly of what we're trying to do here we're trying to analyze couple of stocks and put it against each other and look at Market overview and do some key Matrix analysis please do not follow this at all for any of your investment strategy this is just for research purposes uh don't trust the AI Robo Parts without your due diligence uh just getting the disclaimer out there and then we are asking it to create a report saying that uh you have to create an executive summary you have to create a a table for the data presentation you have to have a clear Matrix uh of Industry averages and like we talked about before this is all the guideline for the report that the uh agent is going to handle now if you go here this is a very small and quick and very easy agent to run you say that okay add date time to instructions you call a show tools call and then you ask that okay stock uh like run the process and it says okay stock agent created ready to take users queries you put in the query that okay what's the latest news and financial performance of Apple uh you see that you see in the running process it talks through each stepbystep process of what went through internally in the agent framework to acrew all the information and create this report for you right so step by-step process uh get current stock prices so symbol is a PL um get company info uh get the historical stock stock prices get analyst recommendations and then the company news so it goes through four or I think five layers of components and uh sources that it's trying to Aggregate and then coming up with the the report in the way that we asked it to so it's creating um uh it's creating uh uh executive summary the current stock price you see it says as of February like 11th it's this much um please verify the data all the time don't like again I'm saying don't believe in what the agents are giving in the face value but it gives a good structure for you to get started with your research right and that's the point it's supposed to be there to augment your intelligence and not like not like take that away so keep that in mind always you all we all see that we have like a historical stock prices here uh that kind of Compares high value for your timeline uh let's look at one other query uh before uh we kind of wrap and go to questions is that you can also do comparative analysis and say that okay analyze the semiconductor market performance focusing on all these four companies because if you if you've been reading the news like the the the the competition as well as the semiconductor like business that's having an effect on AI as well as like the AI chip and the performance is like very very huge so that leads to a lot of U Market um noise as well as like the movement of stock prices uh and what the future Outlook in terms of AI growth could be because if you're are prohibited by you know uh like the chip chip chip processing Powers uh yeah I can only do so much so um so so again similarly you would see that okay we do Company by company comparative analysis and then you see a very structurally created uh table where you do a market position uh comparisons and talk about future outlook for each of these right and then you can run all the other quer and see how it works the last part um uh the llm as a judge uh doesn't is not like an agent in itself or doing some work but it's basically we are using another model like I said LMA 3 instant we're using a different model here and saying it that you are a rag system evaluator we are setting up um the in the description we are setting up uh uh that you are supposed to uh analyze the output from the rag input these are your instructions and this is very uh industry specific I think this is this is not industry specific Matrix but it's very generic and used industrywide like faithfulness context relevance that is the answer good enough for the context that it was provided did it cover all the important qu like processes did it miss any important concept so we giving the instruction to the uh agent to kind of frame its background and context and then um we also like I said expected output we frame it like this score the faithfulness out of five because that's the that's the industry standard and we are also asking for justification and examples of what it missed and how it missed so this will help us um kind of uh make sure that U uh if it's rating something as like three out of five or four out of five there's a way for us to go back and see why it was bad and try to fix that in our uh second iteration and then we also ask for overall score and some key recommendations so feel free to read through it I did not use uh the query that was there before uh I just ran like a sample use case saying that what are the key features of a transformer model gave a very generic answer and missed half of it because I just said that they're characterized by self attention and didn't give much of a context so please feel free to test it out with your data or like your questions and context and see how it works but this is how the rag evaluation report looks to get you started it'll talk about okay the faithfulness because if you see I didn't put much in the context itself so it didn't have much to compare against and it gave like almost like good scores here completeness I think it gave two because if you see here it talks about key features mentioned where only self attention but uh limitations were not talked about and give more comprehensive overview of uh Transformer models so this is just like a baseline evaluator you have to make it more industry specific you have to make it more um you have to make it more um custom to your use case because that's how we all do it like all evaluation process we built is custom to our use case but this is just to you know get you started uh before we go into questions one last appendix here these are I think one of my not one I would say my recent favorite papers that I really enjoyed uh reading so if you are interested to learn more about this space please feel free to go through it there are a couple of other free courses one starting uh I think it's already started started last week by hugging phas if you're interested to learn more about the agent space and how you can utilize feel feel free to read through this resources and Asar my friend is like a genius so she also has like a fantastic guide in GitHub uh feel free to check out her resources as well uh okay so that was all about it uh I'm going to pause here um should I stop I think I'm going to just go back to my slide here and if you have any questions talk uh wonderful that was very very cool stuff uh yeah uh amazing agents and uh yeah thank you for talking to them uh just uh since you mentioned uh awaria um she's actually doing a code along next week uh so uh next Thursday uh I'm gonna be joining in uh cheering her on from Cheer from the audience that's that's wonderful uh yeah so uh next week we've got um three webinar session on uh three webinar sessions on AI and the Enterprise uh so we're covering everything from like AI strategy to like actually building stuff putting it into production uh so yeah a is g to be one of the guests for that uh maybe if if Reon give us a uh QR code for uh joining future webinars all right with that uh we've got a few minutes for um questions uh load of questions for the a so actually uh so uh someone with no vals in the names uh Miss s something rather um talking about uh PG Vector uh so that was that's the the tool for rag it's a post SQL plug-in saying it's got bad performance for Effective similarity search which is true why do you use bad tools and how do you get good performance with bad tools I mean like I that's absolutely right I think I've never used PG Vector in my in my life but that's because there were other paid Vector services that I had access to but like I said please feel free to try out chroma DB and and another another reason why I use that is like if you see um in agno can I change my can you all see still see the screen or no oh okay sorry about that I think in the agno database uh if you go through all the where is it where is it Vector DBS that it okay here all the vector DBS is it can um uh here so it kind of handles chroma DB Lance DB I have used chroma DB extensively before Milas as well uh cudr as well and single store is of of course good as well as mongodb but this was easiest to implement so maybe that was the way but it's not as bad as you think it is if you you using like small uh like single or up to at least four or five documents it's still works pretty well but if you want to push I don't know like um 100 documents then it might like lose performance in there so that's why it start said it's just to get you started feel free to swap out this with anything experiment how it works on your days said uh and if you have like a um actually paid service where you can use like you see you need a pineco API here I think it's pretty standard pineco is gr for production grade uh Vector DB so use accordingly that's that's what I can say yeah so it's actually kind of interesting because uh data uh data C at least in data lab we do use PG Vector for some use cases where performance doesn't matter and yeah reasoning from the engineers was if you're storing user data then under like gdpr like the EU data regulations if you use like any data service you got to like register this new provider and it's register the technology it's like well we're using post SQL it's no extra paperwork to use PG Vector whereas you use um something high performance like pine cone mver it's like oh exactly so only if you're in production think about your performance but don't really doesn't really matter if you're getting started and testing out so don't worry about that too much uh yeah so uh tons of engineering decisions Beyond like pure like uh performance okay uh all right so uh oh you can take the screen away sorry yeah there was uh another another question so this one comes from pratic talking about uh when to use agno versus Lang chain or Lang graph um yeah talk us through the design I mean I mean all of us like all of us started using uh and I'm smiling because I think I know prati so hi prati uh but um but but my concept is like when I started experimenting or when the the didn't take all this this Frameworks were coming up of course Lang chain and langra even Lama index could be a good place to start but I think I think we have to think through or some based on some of my very personal experiences that we saw that I had an entire code base up on one version and then they tweaked something that was not updated and it kind of crashed I had to go back I feel um I feel agno or like f data has done a decent job in making sure that it's very comprehensive end to end it doesn't break as much uh and it's just been my like a developer experience that I really enjoyed it's my personal choice you can use any other Frameworks uh but I feel they're very good end to end they have like a playground holistic process where you can and you can also like raise their um feature requests I think you can do it with everybody but I feel they're very responsive and I kind of enjoyed the framework and that's why I chose it that's a very important thing is you enjoy programming with one particular thing then use that uh rather than using something that you don't particular like uh yeah and there's a certain amount of personal choice in technology there I think uh all right so super uh let's get R of that one and there is a nice question from starter Joe uh where can I learn deep data science so uh I presume that's deep learning data science yeah that was gonna be my answer as well unsurprisingly yeah so we just basically have uh an almost infinite number of courses on data science uh there's plent stuff on deep learning and we've actually just launched some tracks uh career tracks on um if you want to do uh AI engineering both like from a a data scientist point of view on from like a software developer point of view so yeah uh building a applications there's two different tracks there uh definitely worth looking into uh all right uh let me see dinking oh there was one more question oh actually no before we get to that I I have a question for you so um um we built four different agents there and some they're fairly sophisticated what's the sort of simplest agent you might want to get started with I think it would be the the stock analysis because it's nothing but just calling the yaho finance API I think even the the if you if you would like to even uh do the web search part first because that's nothing but you're just asking the question like you would do in Google but rather through like a more optimized and autonomous process go or you can and like I said like look through the agno database they uh like the documentation they also handle or they also support other web search tools so try customizing it building it directly um uh uh doing that but I think I think yeah start from the maybe the stock part or even like the Doug go yeah it's basically like it's like your you're doing your AI chat but then it's like oh we'll just throw in a like a web search tool or calling an API here and there and uh gives you a bit of extra stuff to play with okay nice and when you've been using these at work have you noticed any benefits like what what have you sort of got from this like how you how have ai agents helped you I mean like I said like um and I have to repeat it again like uh in all Financial Services uh there there's no way that we automate everything end to end because the consequences are too high if you get it wrong so there has to be a very specific evaluation Frameworks guardrails uh around it so but I think to get us started doing some of those initial um rag based document extraction so that like if say say a user has like five or 10 documents it needs to read to get to a judgment and each of those document runs 100 and 200 Pages it's not POS it's a timec consuming it's too much effort but if we can get them to like certain layer of information that's nicely formatted uh all the data is extracted and with each extracted data there's some kind of source and even with like all the chat Bots that you see now like open AI like CLA everything goes back to the source so if there's a way that we can review The Source make sure that it's trustworthy that's how you kind of also build client trust and say that hey it's not like absolutely wrong but give us more feedback how we can make it more custom and then you add more custom features that don't don't use uh one of the very generic example one of the custom process we have or I have built in some of my other site projects as well is that don't use quotations from a direct particular person into the report that's not big but this is big that is big for the client the end users that they don't want to like look at personal codes but rather look into the data so more quantitative focused versus opinion focused if that makes sense that's how you build your custom Nuance tricks into the agent workflow okay uh that's fascinating and I guess uh so one more follow out to that uh if the if the do summary you generate it says something weird you might be like well I don't whether it's true it might hallucinated is there a way to go back from the summary to the original report and say okay it it came from this line in on this page yes yes yes we can do that uh I think what we are trying to right now do is break it down into components saying that if the in the executive summary if we are and that's that's also the power of the extractive summarization part right sometimes we want extractive summarization what that means is that we want directly that line as is don't change don't you know rephrase it so that we get the um so that we get the quantitative mentions of particular Matrix that we need as is and that goes into our evaluation framework saying that uh did the summarization mention a particular thing about carbon footprint Etc uh and when the actually the rag process looks up into the database and sees that there's a matrix about carbon footprint there could be 500 but for in the first executive summary whatever match number can we do a you know entity match uh entity match to exact a numerical match it's pretty easy to do but if it's more abstractive then we come to more of like a similarity or a a context mapping that is it actually talking about the same thing so it kind of goes back so I thought that was GNA be like oh yeah it sounds fairly straightforward but now you've mentioned that it seems like there's actually quite a few different things you need to think about I evaluation framework is a make or break for your use case if you don't have a strong evaluation framework it's going to wreck havoc in your life okay all right so yeah use these agents with care uh brilliant uh before everyone dashes off uh the way we can keep these codal alongs free is uh we take payment in likes and shares do hit like YouTu video please do subscribe to dat Camp Channel please do share this with all your friends uh colleagues people on the street uh yeah uh excellent so uh thank you once again J that was like very very cool stuff I very much enjoyed that I felt like we could carried on for like an hour at least yeah me too I can talk about it endlessly as you can see but cool uh all right yes uh thank you to everyone who asked a question thank you to everyone who showed up today come back tomorrow we've got a very exciting session it's going to be uh deep seek R1 versus 01 uh versus 03 mini like all the latest fancy models we're going to see how they perform on reasoning tasks so come back for that and do come back for the AI in the Enterprise series next week all right see you all