a couple of hours ago open AI released a new model called gp4 o the O stands for Omni so what this model allows you to do is to input text audio and images and generate uh audio text and image outputs in return and so the note here is that GPT 40 is especially better at vision and audio understanding compared to existing models I suggest you watch all the demos here there's a lot of great capabilities but I just wanted to make something quickly to see if I even had access to this and it turns out I do so what I wanted to do was test out either the audio or Vision capabilities now since yesterday I just posted a video on uh uh programming my brokerage account and saving a different charts from that I thought I would test the uh Vision functionality of GPT 40 and so if you didn't watch my video yesterday uh it looks like this so I can run this and what this does uh this is a Python program that interacts with the interactive brokers twws API and is able to pull data uh on stocks and so here you see I have a a chart for Taiwan semiconductor right here but this also scans and automatically pulls up the most popular stocks for today uh so you can see uh AMC and gme are very active so I click gme and of course roaring Kitty came back to Twitter and gme just spiked right there so what does this have to do with GPD 40 well I thought I would just throw on an AI feature to this little desktop app that I've created and so one thing I did in the uh previous video video is I added the screenshot functionality and so what I can do is when I when I pull up my chart so if I click on AMC here pulls up that chart if I click on gme pulls up that chart and then I can pull up a particular time frame like the 1 hour and so this is interacting with my broker's account and now what I want to do is take a screenshot so I'm going to click screenshot you'll see it saves a screenshot to my screenshot directory of how the chart looked and then you see right here what I'm doing is telling GPT 40 to actually do technical analysis on that chart by just looking at it and so you can see every time I take a screenshot now it gives me a complete markdown description of that chart so let's say I'm just saving charts uh throughout my day I automatically get a log uh of some analysis and text format of that chart and so I don't even necessarily have to look at it it just pretend you're saving some type of Journal it's automatic for you and so I click so I took a screenshot of gme and it says a short-term movement shows a significant spike in the price of gme and it shows um it peaked at 3820 and then it talks about the volume that's on that chart it talks about a support and resistance level and the indicators that I had on the chart so how does this actually work in Python code well if you followed my tutorial yesterday there is a function called uh take screenshot so when I click this button I'm automatically triggering and saving a screenshot to a particular directory so all I did was add on to this function so I had a chart file name and I was writing the image right there so all I need to do is is write a little call to gbd4 that actually analyzes this chart and so after I write the image to the file system I have this little analyze chart function so I created a new file here that I just called gp4 technical analyst just like with my other open AI tutorials you need to pass in an open AI key you can set it as an environment variable uh when you initialize the open AI client you need you can give it a default organization or in my case I'm part of many organizations so I need to pass in an organization ID to make sure I'm billing this to the right organization which is mine um so I have this analyzed chart function and all it does is accept the path to an image and so since I'm saving each screenshot and timestamping it uh to a screenshots directory right here I'm just as soon as I write that file name I'm passing the path over to analyze chart and to call the open AI uh GPT 40 model you just need to pass in a the new model name assuming you have access to it so I have access to gp40 and I just use the open AI uh client here to create a chat completion and I can just give it a user role here and just say analyze this chart include the symbol and discuss the price action you can make this more and more sophisticated if you want to focus on particular elements so let's say you want to act as a a trend follower or someone that's looking for mean reversion or you can you can really specify how you want the analysis to be done now the key to using this um if you've used gp4 vision is you need to actually base 64 or encode the image and so you can see in my chart path here I have this function called encode image um and so what you do is import uh you import base 64 here and you can base 64 encode that image and once you do that you can actually pass this as part of the message and so instead of a normal like URL you would request on the web you can pass it this special data URL with a Bas 64 encoded image and then you just return back the response and so it just returns a normal uh chat completion right there and then you can see right here um in my analyze chart I'm returning the analysis as a text here and printing it at out on the screen and then I also uh output all that text uh to a file usually just basic python open a new text file and I'm just saving it to a markdown file right here and so you'll notice um if I pull up one more chart here let's just do this one more time I can even uh probably make it analyze my drawings here and so yesterday I added this ability to draw a trend line for instance on here and so I can draw a little trend line here and you know see how it would interpret this in real time so I'm going to do screenshot right there and you see it's saving a screenshot of AMC exactly as it looked with my trend line right in place and you see right there I have the analysis from gp40 in real time displays price action for the symbol uh AMC it appears to be a multi-hour chart uh it's been on upward trajectory for most of the session uh and then yeah you see it proceeds to give some analysis and let's see if it mentions uh yeah a a strong upward Trend initially with a pullback and yeah and so it's interpreting the chart just as I've drawn it right there and let's see if it even mentions yeah it says after reaching 588 and so if you watched yesterday I calculated the uh the high of the day and had it automatically draw a horizontal line and it detects that as well so yeah that's my first experience with GPT 40 you might say you were able to analyze images with gp4 vision in the past but one thing I'm noticing is the analysis provided by gp40 is already much better so the quality of that analysis I just showed is far superior to um what I had done with gp4 gp4 uh Vision in the past oh and I forgot to show that um I've actually been testing this out in other contexts so I have like a heat map uh here so uh for sector analysis right here it can actually look at this heat map and describe it um this ETF uh chart from vvis and then also um another chart I was looking at as well and so just to show you the a heat map example yeah you can see how it's able to look at this very uh sophisticated heat map not that sophisticated but there's lots of details and many different symbols on this heat map it can do it a lot faster than I can as a human so you can see how I can pinpoint what's going on in energy uh Tesla Amazon all those things are very small on this chart and it's able to pick up all the individual percentage moves and so if I look on uh technology here you see apple up by 2.05% it read that off the chart Nvidia so it can actually see um what's going on so yeah I expect a lot of this stuff I know I may had to make a program for this but I expect all of it to be very integrated into your desktop they just released a a desktop application as well that can see your screen and so I can you can imagine what where we're headed here where you know the big tech company is going to be watching everything you're doing on your computer and analyzing in real time and so yeah um so is this all going to be obsolete do I need to teach anyone how to program when gp4 can listen to you and answer all of your questions uh yeah maybe we're all obsolete definitely all the uh technical analysts are kind of obsolete we don't need anyone to talk about uh charts and visualizations uh necessarily if this thing uh gets really really good over time uh so so yeah I'm excited about the possibilities but also it's kind of blows my mind uh some of the things that were shown in this demo so I'll post this code somewhere uh hopefully this was interesting uh be having some fun with this and hopefully we'll share more uh in the coming weeks and months all right take it easy bye