that's right we're bringing calculator to iPad so finally they have done it Apple has pulled an apple and launched the greatest calculator ever math noes is a game changer turning handwriting into digital equations solving problems even graphing on the fly but hey wait it's only for iPad not everyone has that what about us Android and PC see folks don't we deserve a better calculator application as well well guess what we are not just bringing math notes to the web we are taking it to the next level basic arithmetics Child's Play solving equations easy peasy but that's not all our calculator can even calculate complex scenarios like the time it would take for your Porsche to crash into that tree or maybe the answer to the life and universe and everything it's like having a math superhero in your pocket but wait there's more our calculator is not just limited to math problems it can even detect abstract Concepts depicted in your drawings for example draw a chicken and an egg and watch our app grapple with the age-old question which came first and if that was still not enough our calculator can even find the deep meaning behind Captain America's shield don't worry my friends we are not playing around in this video we are creating this application and don't you guys worry if you don't know a single thing about python react or even calculators because we are going to learn everything today let's start with giving it everybody's favorite superhero Captain America some red white blue another red and a little white star and let's see what it gives us this is the drawing of Captain America and shield it is made up of red white blue and this is a shield is a symbol of freedom and Justice amazing let's try a Pythagoras Theorem make a tree there be Shadow uh we'll keep it easy not some really hard number 8 m of length and 6 M of Shadow or base length uh let's draw a little sun and what will be the hypotenuse it should be 10 yep that's correct should we try making a car crash into a tree uh that's a common mathematical problem right okay uh let's draw a tree V leaves green and the length should be what 100 m okay and it speed should be something like 160 km per hour boom Kabam pow let's run it I think that's the correct answer 0.225 okay let's try asking it a more abstract question underlying theme pardon my flag drawing skills really okay let's make some people salute our flag that's our team that's the error by night's team okay and that should let's see what it gives us patriotism equals patriotism okay it's get getting there another one to really demonstrate its capabilities I think let's draw a Cricut Wagon Wheel with some runs all over the places Let There Be sixes Let There Be fours uh let there be singles different colors so that are really recognizes and we can give it this to recognize what colors each of the lines represent and that should give us what six8 and uh yeah 15 that's correct okay one last one that I really want to test myself and check if AI has capabilities to detect folk tales or stories from real life I think you all might have guessed it it's Newton and apple falling on him let's see what RI gives us okay yep a person looking the story of Apple falling on Newton's head discovery of gravity all right that was a lot of demo and intro setup whole shanans so let's dive in now let's start your favorite Cod editor and open up the terminal we'll create our project directory I'm naming it math noes and we will be using vit in this project so let's start our so let's create a new project using vit uh react typescript let's open this project in our vs code and let's initialize our Tailwind files now we need to uh configure our TS config files the app and node we need to add a base URL and give an alias to the root directory we will do it like this that and same thing in the DS conf. Json we also need to add the compiler options here uh now next things we need to also edit our tailin file tailin config file let's just go to their website and copy paste this we also need to edit our index.css file just add the Tailwind prefixes there now we will be using Shaden component library in our project Shaden is a very beautiful and an amazing very customizable library to create and reuse components as you can see there are various components here cards buttons scen Changers text areas select menu radio button drop- down menus hover cards inputs labels and sheets let's initialize the Shaden component Library use neutral uh you can use the default settings that are given on their website now let's open up our config and we need to install node types uh so that we can easily use path function in our vid config next thing let's try adding a button using our Shaden component library and that's done finally we also need to edit our tailin config file get the link in description for a T.C config.js and copy paste that content in your file now for project structure I usually use a screens directory where all my website screens are in so let's create one for our home directory uh I'm sorry home screen here we will be using functional components there are two kinds of components in react functional and class components you should refer to react documentation if you want to learn more about them first thing we need to make is an HTML canvas where we can draw stuff here here we will be using ref for our canvas with reacts use ref hook this ref is used when you want to store a value in your component but don't want react to rerender the component when this value changes if you use a state it rerenders after every change to States value let's also return this canvas component from our function so that it is rendered on the screen we will give it a class name of absolute and top zero left zero full width full height so that it fills the whole screen next we need to know whether at the moment user is drawing or not so we will create a state called is drawing which will be false in the beginning to know when exactly user wants to draw we can react to the mouse click event because obviously to draw user will click and hold the left Mouse button so let's start with the function called start drawing uh which will take a mouse event as an argument on the canvas element and we have to initially configure our canvas by setting its background color and setting 2D context so that we can draw 2D drawings on our canvas begin path function will create a new path and delete older stored paths and move to will move our cursor to wherever we want in this case wherever the user clicked then we set our drawing state to true and finally we need to add an attribute to our canvas component telling it to call this function whenever Mouse down event happens similarly a simple function for stop drawing on canvas this will just set this is drawing state to false uh let's add it on our canvas as well now we need a function to implement the actual drawing method here we will take the mouse event as an argument and if is drawing state is true then only we need to draw we will set our brush color to white using stroke style stroke style attribute and then call line two function to follow our Mouse and finally a stroke function to draw wherever the line two function was called Also let's tell our canvas element to call this function wherever Mouse moves finally add a use effect Hook when our website starts to initialize the canvas elements this use hook will take an empty dependency array so that it does not run every time some State changes it will only run when we start up the application now we are going to install mantin which is another component Library like shat CN and we will also use react router Dom uh to manage our application routes let's install react router Dom and we will use mantin for its color swatch component it has a very beautiful color swatch component that Shad in lacks currently uh you need to just copy paste what I have uh copied in this post css. conf. CGS from maninee's official website now let's write our app. TSX file there are several routers in react router Dom like native router for your react native applications and browser router for your browser application there is also a memory router but we will be using browser router here uh head over to its documentation if you want to learn more about routers they are very useful when you create more complex applications so browser router will be uh is used to just make routes for your browser application it stores URL in basic URL format and this router provider is an abstract component that takes in any router and provides a common API for all of them paths will be the array that will store all our URL paths in our application and finally our home scen component that we created so pass this array to our router and let's return our app component with router provider which will use our browser router let's go test to this app now npm run let's go to our website it's working great we can draw whatever we want now but it's quite boring there is no color here with it's just white on black and we do not have any way to erase what we draw so let's try that okay then I will first add Colors Let's create a file to add colors oops TS RM do T open that and we'll write uh let's create this watchat for all our colors I'm just copy pasting these colors I have already created let's export our file watches now we have to import these colors to our homepage and we will use mantin color swatch to display them on our screen uh let's also create a state which will hold the selected color and just find and replace with contr HED all the places we used white ad with the color State now we need to add some buttons to our page so let's import our Shaden button and we will also need a XS library because we are going to be sending our image data to our backend server through XS first let's create a reset button for that we need a reset State whenever this is true we will have to clear our canvas and also clear other states so let's create use and use effect hook it will just call a reset canvas function we'll create a reset canvas function next and in its dependency array it will take the reset State because it has to run every time the reset State changes now let's write our reset canvas function this function will just uh clear our canvas we will use the canvas's clear re function for that for first we need to get our canvas using the canvas ref. current and we will just call the Clear rect function here we'll give our XY parameters there and the width and height of our canvas uh now to keep typescript happy we will create an interface for our response this will be the response which will send back from our backend server we will send back expression result and assign in a short while you will understand why we selected these three specific values we will need another interface for the result which we generate on our front end it will be it will hold expression and answer to that expression it could be like x + y = 7 or 2 + 5 = 7 whatever the case may be uh we will need another state to hold this result variable it will have it will use the generated result State type now we will need another state to hold our dictionary of variables this dictionary of variables variables will be specially useful whenever we want to assign some value to the variables like x = y x = 5 or Y = 8 now let's create our send data function this will send the canvas image to the backend uh for our back end to analyze this will take the canvas ref. current and if it exists we will send a post request through our xus to our backend URL which we have stored in our V API URL it will have the data image which will be our canvas image two data URL function will convert the canvas image to a image format like webp PNG for now we will just print the response we'll get back from our server using [Music] console.log now let's add some buttons for the these three functions we just created we will have three separate buttons or two buttons and one color swatch grid so let's create a div with grid three columns it will have the first button which will be our reset button it will just set the value of our reset state to True set to true it will be the we will use default shat scene variant and black color let's write reset here let's just copy this button for our calculate function as well it will have calculate and it will call the set send data function we created earlier now we need another div or another group for a mantin color swatch this will have we will just map the swatches array we created earlier map function takes a call back and it will return the color swatch component with the key color and the color it will hold is the color we created and on clicking that it will change the color State we created earlier to whatever color the user clicked on uh to keep typescript happy we have to write the type for our Swatch color it will be a string as our set color state is a string uh let's go check if our app is working nowm rev uh we have not installed ax so here that's why it's giving this error let's quickly install that now let's run this and our app is working great we can also select colors blue red orange and we can also reset our canas we just need to shift this to the top of the website so it doesn't interfere with our canvas okay we just need to go to our index.css and remove everything from root to this media and we can go back and check our canvas is working fine now our color palet and buttons are on top and everything is working fine okay let's add our environment. loc file with the vit API URL there and this will be the API URL Local Host on Port 8900 if you have to deploy it you will be writing your backend URL there okay then we are ready to move to our backend let's create our python backend project directory and also a python virtual environment to keep our dependency separate I use a jang style project structure for python backend as well we will create apps and a calculator app in that we also we use pantic schema so let's create that file and finally constraints and environment file to store all our constraints and our API Keys let's add our backend folder to the vs code workspace add folder to workspace projects we will be using fast API for our back end instead of Django in this project here we will need an asynchronous server so we create an asyn context manager we will also add course middleware so that we don't face course course origin errors and we will put our server URL and port in the constants file so that it is easy to change it later uh in the fast API live span we don't want to do anything so we will just y uh quickly want to explain what the fast repair LIF span is usually you'll have stuff like initializing databases or loading data from database or a remote server server maybe loading up your machine learning model it's basically the code that will be executed before your server starts up and code after the yield statement will run when you close your application that is after handling all the requests fast API will execute this code clean up memory or saving your system state or whatever you want to do when your application closes now let's add our middleware course middleware and we will allow all the methods headers and requests from all Originals that's why we put an Sisk there that's a wild card finally are out to check our server Health uh it will just return a message that server is running and now write a main blog for starting uon server now we have to make a constants file to store our URL and Port before testing our server we have to install the required libraries we will need fast API below uvon pantic and Google generative AI because we will be using Gemini flash API in this project we will also need python. EnV to create and use ourv file now let's test if our server is running yep so it's running let's go get our API key now let's go to Google Cloud console and click on this link here now we need to create a new project you click here and create on new project we will name this project something like calculator notes calc notes let's do it and we don't have any organization so just click on create now after creating our project we need to use this project to generate our Gemini flash API key so let's go to Google AI studio and here click on this link uh sign in if you haven't have not already you probably need to create 2 fa before doing so after sign in in you need to click on get API key and click create API key select the calculator notes project we just created earlier and click on generate key now our key is generated let's copy it and go back to our project now let's go write our Gemini API key in the EnV file uh just so you know a EnV file is used to store your environment variables that is usually either secret data or something specific to your environment that you are running your code in uh so it's so it's a file you don't want to expose on your git repositories uh let me show you let me pull up my folder of git repo for this as you can see here all the environment related files and directories should not be uploaded on G they are in my git ignore file now let's go back and write our API key let's import this API key in our constants file now this load. EMV function automatically reads ourv file in the root directory and we can access anything we write in that using our os. getv function now let's create our calculator analyze function in the u. Pi the utility. pi file today we are going to be using gini flash API but shortly we will be creating another video uh which will cover how to run this locally using AMA and llama models or mistal models so keep your eye out for that one now let's configure Gemini by providing it your API key we just generated and now we can create our analyze function which will take in an image and the dictionary we created earlier uh we will put this model initialization above this function because we don't want to run it every time the analyze image function is called right I serialized this dictionary of variables to a string using Json DS uh so that I can pass it in the prompt easily for now copy this prompt I have written from my GitHub reposit Ray but fear not because there will be another video in a couple of weeks which will explain you my whole methodology and prompt engineering behind this so make sure to subscribe the channel to not miss that [Music] one now as. literal eval uh it is a buil-in function to evaluate mathematical Expressions which we give it in a form of a string I added this error handling because I was getting some errors in est. literally eval while finetuning and editing my prompt you don't really have to add it but you should if you want to play around with the with the prompt I created and in case it doesn't work correctly uh then we'll just add the assign key to answer according and written the final array before creating our API route uh we will create a pantic schema so that we can tell our fast server to what data we should accept from specific routes we will just give a string for our image and a dictionary for our dict of variables now we need to create a fast out where we will send API requests to get back this result by importing this API router we can create fast API routes outside of the main. pi as well now we will create a post API route in this post API route we will explicitly tell fast API to only accept data if it is in the form of image data schema we formed earlier we will use base 64 to decode this image and then send these bytes to our analyze image function base 64 encoding was done by the canvas when we called the function two data URL there uh now let let's return this to our front end with a message and Status it's a good practice to add status success and Status failure so that you can easily check your logs later if your application fail or acts weirdly final change to our backend will be just adding this route we created in our main.py for our Unicon server to handle this will just have a prefix so that we know what will be the route for this application all the routes created in this application will have the prefix slash calculate uh whatever we write in the rout that will come after it and with just that our backend is completed now let's go check if it's working correctly okay let's open our terminals caly C backend run python main p on our front end oops run npm run Dev uh let's try drawing something here 2 + 7 equals should give us nine now no result uh let's try again again no let's check in our Network tab okay we are getting wrong URL here I think we misplaced our EnV file yep uh now it's giving us. EnV we have correct URL I think and we are also um using correct URL here let's try printing this conso [Music] log R URL import meta let's print that uh uh REM yes it's printing incorrect URL uh I think our EnV file is misplaced it should be in the root directory not in the SRC it should be in our caly let's move it there uh let's restart our server and front end let's open it again now let's try again 3 + [Music] 7 correct result R see image processed and the answer is also correct 3 + 7 = 10 in our back end we can also check that yep all right guys our calculator is working very beautifully now so we can finish up our project uh to finish up I want to add a a small little thing as well instead of boring plain text we will be rendering our mathematical Expressions to latex latex is a type setting system that produces very beautiful mathematical functions and expressions so we will be using that to render a mathematical Expressions to latex we will use a library called mathjax to install this Library we can go to this website cdnjs math Jack here you have to use version 2.7.9 because that's what I'm using uh now do some files for and click to show all files okay now you have to search text MML am C HTML JS we will be using this one just copy the URL let's go to our code open our homepage here here in our use effect which we will run only once when our application start because it has only an empty dependency array we will also want to load our math JX script that we just copied document do create element oops paste oh sorry write script here and script. SRC we will add our link here now write script. async true document. head we have to append our this child so our script is loaded in our page now when we load our script we need to create a window and config our config the latex configuration to help render our Expressions We'll add a call back function here with Windows do maj. hub. config and here we'll write text to Jack this will not be a capital letter and what's wrong here okay we'll check this afterwards inline math and this is the format of normal latex expression it ends with it starts in ends with the dollar sign and let's add this this is how latex expressions are written in a normal latex editor or your code editor we'll finally return a call back function here which will remove this script we just added here now whenever our result State changes we need to render it to latex uh because we are storing the result in result state for that we need a new state let's call it latest expression for storing our latest expression that will be rendered form of our result const latex expression and another function set latex expression here which will be use State and array of string and and to set the position where we have to render this latex expression we will make another state called latex position same thing L position use State array I will put some default value here instead of just number because it usually does not render in our visible screen if this number is not there this will not be an array I'm sorry it will just be used it like that uh now let's create a use effect for the same uh use effect I put result as our dependency array because we need to run this whenever the result State changes and if result state so that it does not run when result is undefined or null we will run a function that will render the latest the this latex expression on our canvas it will take result and result. answer let's create this function now this function will take our latex and convert it into a large expression and answer we use SL SL large in latex format here we take expression and in the format answer I think this should work now we'll set our latest supression to this new value we just created uh we are going to use a spread operator here to uh in case we had already some ltic expression stored in our state we will spread them out if there are many assigned operations like X = to y x = 5 y = 6 and we have to render all of them we need to store them in an array that's why we took array in our ltic expression type here now we need to take our go to our canvas and the boiler plate stuff we have written everywhere CX canvas get context oops and if CTX is there we'll just clear the clear whatever image we had drawn there so that the image is replaced it looks like that image is replaced by the latest expression that was rendered now and here we'll close our function done now whenever this latex expression changes whenever this function is run we should uh we have to create another use effect so it gets rendered into the required latex we want another use effect and as you can guess now we'll give latest expression array here and in the function we'll just write if latest expression if there is some latest expression in our state and our math JX loaded after some time out this is because uh this is an asynchronous operation Hub you set type set into mrix Hub after time of 0 seconds 0 milliseconds I'm sorry we should also update our reset function to clear all these new States we have created Also let's go to our reset used effect set latex expression to Mt array set our result undefined set our dict of variables to also an empty dictionary uh and I think that's enough now final thing uh we need to use our latex position to know where to render our canvas you have to put them in our can return fun return statement uh but before that we need to edit our s data function because here's we were just printing our response until now of we will just run through a loop uh on our response where whatever data we have will be of the format response of the type response we just created earlier and this will if there's if you have assigned some value for example x equal to 5 that's an assignment operation but 5 + 5 is not an assignment operation so if there is an assignment operation true we have to set our dict of variables with just whatever was already in that so spread operator there and a data expression result expression will be X and result will be whatever we set X2 so if you later use x into 5 it will give us 25 uh now we have to find the center point of uh wherever we drawn so that our latex is rendered at the exact center of where we drew whatever we drew for that I will just call our CTX here we'll just take our image data of our canvas and run through a loop since it's an x and x into y Matrix we have to run through a n Square Loop for I in range of our x coordinates and for J in range of our Y coordinates and then we'll just find our Max X and Max y point where we had drawn stuff and that will and that averaging that will give us our Center of X and center of Y so let's do that e now we can set our uh latex position to the center points Center X and Center Y and then just another go through our Loop set for each uh sorry rest. data for each data response and set time out because we set this asynchronously set result to our expression and the answer and this will take what 200 seconds milliseconds maybe I think it's looking great now there are no errors in our file let's check it once okay we have not set our latex position yet so let's go do that on our canvas and now here I found a really nice component to so that we can also move around our result or rendered result called draggable from react dragable I think I have installed it already if not you have to just open your terminal and run react drag and it will install the component beautiful close this terminal and let's go to our canvas let's write our latex expression here we have to only uh do this if there is some latex expression already stored in our state so we'll write a conditional map like this it will not run if latest expression is undefined or null it will never go to the second part of code but if it is true then it will go to our second part we'll need some index to for our [Music] map uh there it will be a parenthesis not dragable I it seems correct but I'll write it anyway key index there will be some default position which will be our latex position oh latex position and the and whenever we and then on stop function prop uh which will tell the dragable what to do when we stop dragging around with our Mouse this will take in the event and data and it will just set a new latex position to that new point we'll just close our dragable component and finally we need a div that will take a class name for absolute position where just keep it simple white text and this will another diff for a ladex Content it class name will be ladex content you can just write it on same line and it'll take our latex State beautiful and finally our project is completed let's go test this for the last time let's just start by some assigning operations x = 8 yep uh y = 10 yep and let's do 2x + y this should give us uh what 16 + 10 26 yep uh let's do another one basic integration d by DX for x² yep it has given us 2X so as you can see it's all working perfectly now our math is being matched and it's being rendered beautifully in latex our colors are also working fine our buttons are working as well uh but let's just try a really complex question now all right let me draw up a really complex mathematical expression here it's a real tough one I don't think this application stand the chance let's see subscribe to a channel on YouTube equals subscribe to error by night hey it even calculated this correctly all right guys we'll see you in the next one until then Happy coding