I want to create a Pokedex for real life with my goal being to be able to take a photo of anything and have it analyzed and identified this is a Pikachu plush toy it is very soft and is also one of the the most popular Pokémon with its type determined some statistics and stats as well as an entry that's stored in a database so you can check back on at any time and since I'm building this Pokédex for the real world it's going to classify more than just Pokémon you'll be able to Anze any type of animal whether it's a flying type like a bird a water type like a fish or even a grass type like any type of plant or flower when I finish building it I'm going to release it for free you might be wondering why want to build this idea let me explain Pokémon has always been part of my childhood all the memories I have of it I look back on fondly even though it's been 25 years since its initial release I've stayed up to date to almost every single episode ever released all the books that have been published as well as even every single game in the series so it should come as no surprise that I'm a bit of a Pokémon fan and that's why I'm kind of inspired to make this Pokedex idea happen having created a few startups in my time the first thing I thought I'd do is some research on Google to see what kind of Aesthetics the pokeddex has and if anyone else has created an idea like this the concept for a pokeddex is pretty interesting especially with the fact that this idea was created more than two decades ago a pokeddex takes in inputs text or images and then spits out and generates an output explaining what that object is if you think about it that's basically chat GPT these days there are a few other features such as having a database of all the entries you've scanned having an AI voice that actually explains the entries wait let's try that again having an AI voice explains the entries and for each entry to have proper statistics such as weight height speed attack Etc I finished my research and that no one created a pokeddex for real life though some people did try to 3D print examples using raspberry pies but they weren't very good the closest I could find to a success story is this R1 or rabbit one as they call it and it's a pocket companion that's Allin red takes photos and uses Ai and vision to spit out explanations or help you with daily tasks I think I know where they got a lot of their inspiration whenever taking on a huge project like this one here I like to break it down into simpler steps so that it doesn't seem overwhelming before writing a single line of code I'm going to sketch out the idea and unlike the rabbit I'm not going to make a physical device because why would I create a physical device when we all have phones with cameras in our pockets here's my pokeddex plan I'm going to scan images create descriptions using AI as well as characteristics like height and weight I'll have them verbally explained using an AI voice and then I'll store those entries and save them to a database this is just the highle plan the lowle plan will include the design figuring out whether I want a mobile app or an actual app on the app store and what each one of the pages will look like sounds so simple in theory let me see if I can actually make it so it's time for me to start designing the Prototype now to do this I'm going to jump on to figma it's one of my favorite places to start creating different mock-ups for a app or website design and for this Pokedex design I want to recreate the same aesthetic and feel that we have from the original Pokémon games I'm going to head over to Google as well as behance and dribble and start collecting a montage of different types of photos of pokedexes this can include pictures of the Pokedex from the official Pokémon website as well as concept art from Nintendo it can include the Pokedex from the actual TV show and what it looks like and then there's a number of websites that have tried to do their own take of what a pokedex should look like what I like about these is that everyone has their own idea of what a Pokédex can look like as an application or for a website how the user interface can look like with Pokemon previews as well as stats what kind of fonts they're using and what kind of colors and from what I can tell they're all quite vibrant well the fonts are very pixelated though I'm not too sure if that's the kind of look I want to go for at least it gives me an idea of what kind of other Concepts people had in mind when they think of a Pokédex and with this done I think I'm ready to start sketching out my own screens for my own version of a pokedex the way I'm going to do this is with my iPad I use an application called Concepts it's one of my favorites in order to just sketch out notes or ideas or even just brainstorm I want to turn this project which can seem complicated into a simple number of screens and I find visually writing down exactly what I'm trying to conceptualize in my mind always a much easier way to get things started what I'm thinking for the very first screen which will be the login screen is a nice header with three little circles so it feels like a Pokédex I'll have the word Pokédex at the top with a login and a demo button that people can select to get started the next screen will be one of the most important it'll be the camera screen you'll have a big camera that you can select with a button at the bottom that you can press to take a photo the third screen will be the Pokédex itself with photos and names of the Pokémon and a menu at the bottom where you can select each page you want to head to I'll also need a viewing page where you see a photo of the Pokémon their name description and all their stats and I think that should be mainly it that's the core concept of a PO X so far so good though I'm a bit hungry so I think it's time for a lunch break what I might do is grab my laptop and do some work on the road I find getting some fresh air and new scenery always helps me be a little bit more creative especially when I'm in the design phase of creating an application or a project at this point I don't want to build anything concrete I just want to sketch out some ideas I had in terms of how the overall appearance might look like for the application and this will just include little shapes or triangles or circles that I might use later on for things like the camera or the overall appearance for the application itself and it didn't take long before my food arrived so I'm going to refuel and get some energy and then head back to the studio because now I have a pretty clear image of exactly what I want to build and how I want to build it and its design and layout overall so time to actually start on that for the design mockup I'm going to pull in the sketches that I've created here on my iPad and put them into figma I want to create the styling of the Pokédex to look familiar and like Nintendo and I saw a video recently by hyperflexed which did just that I really enjoyed how he was able to recreate these background effects and he was able to do so using HTML CSS and a little bit of JavaScript so I'm going to try and do the same for the Pokédex mockup trying to get these professional colors and styling happening I've pulled in the SVG of this sketch that I created earlier and now I'm going to create out the color scheme and topography for the colors themselves I'm going to sketch out the primary secondary and tertiary colors and to do it I'm going to pull in the logo from Pokémon and select each one of those colors and then place them here as the ones that I'll be using as a theme for the topography that's a little bit more tricky I did a lot of searching on Google fonts to find a font that looks friendly and kind of like the Pokémon or Nintendo theme and it was really difficult to do I didn't really find anything straight away I did some browsing on Google to see what the actual text and formatting looks like and it does have these kind of wacky comic sand kind of text and characters and while not as popular they're easy to search on websites like the font and this is where I stumbled across poet sand one it has the right Style with these curved characters that almost look like comic Sands since I didn't want to spend 2 weeks looking for fonts I decided to select and download it and for my paragraph text I was just going to use the font inter since it's pretty popular and quite readable now the easiest part of the design I'm going to create the High Fidelity versions of each one of the screens and this will essentially be what the end result looks like I'm going to place a lot of time and Care into the header of the Pokédex design because this is kind of like the camera aesthetic sort of how the iPhone has its three camera design and that is a signature of what the iPhone looks like the Pokedex here will have this blue circle for the camera with these three dots and these are kind of like the loading bars to show that it's processing a picture and the picture screen will replicate this for the menu I want to replicate things I've seen in Instagram and Tik Tok which will be these four menu items people can select but in the middle I'll have the camera option so people can immediately take photos at any time after a photo is taken I'll have this preview screen it'll have a description as well as details like weight height category essentially the stats for the real life creature or item after this I want the Pokedex entries listed out like a database so you can preview all the entries and photos you've taken so here I'll actually list them out and I'm thinking to have different colors depending on the type they are so for example if it's a bird bird it'll be a flying type if it's a plant then it'll be a grass type if it's something like a mouse maybe it is a dark or a normal type it really depends but for the time being I'll fill them out with examples from Pokémon such as Charmander and Pikachu and I've finished my design I'm pretty happy with what I've come up with the very first screen will be the login screen I'll have a login button and a scan button so that people can immediately use it without logging in this will take you to the camera page where you take photos by selecting the snap button then you'll head to the preview page viewing the object you scanned and finally if you create an account this will be saved into your own Pokedex database and while I think I'll turn it into a web app it'll have a menu just like a normal phone app now it's time to start programming this is my favorite part of the video I'm going to show you exactly how I built this project and it starts with a coffee and a single line of code I'm installing nextjs 14 this just requires me to run npx create next app I'm ashamed to say I haven't learned typescript just yet so I'll be using the basic JavaScript configuration along with the app router I'll remove most of the base styling and text so I can start off with a clean page just calling it Pokedex RL now I need a database for this idea I'm going to be using Astra DB Astra DB is a database as a service that's entirely free up to 80 gigs and it's built on top of Apache Cassandra they were also kind enough to sponsor this video and so you can learn more about them in the link in the description let's take a look at them and how they make this Pokédex come to life if you haven't heard of them before they're made by data stacks and they're a company that specializes in Enterprise development working with big companies like Netflix their new AI tools let me create Vector databases in a single click and they have a new JavaScript API module which lets me interface directly with the database through nodejs I began by reading their documentation to get a better idea of how I would structure my Pokedex database since I'll be running this on my back end I'll need to install the client which works with both typescript and JavaScript then in my console I'll run npm install add data Stacks SL asrb DTS now I want to add my very first Pokémon into my database which will be a Bulbasaur the only issue I don't have a database yet I don't even have an account so let me sign up to a free one I'll use my Google credentials to log in and accept the terms and conditions I'll be taken to the dashboard here I'll select create database I'm going to call it the Pokedex and I'll just use Amazon web services I'll select create database and this will just take a few minutes to initialize once done I'm going to create my very first collection this will be a collection of Pokémon so let me select to create one here and I'm going to make it as a vector database I can use the open ai's text embeddings Ada 002 model which is pretty cheap and Astra DB configures all the rest for me I'll select create collection and that's done the dashboard also has connection details so I can immediately connect it to my nextjs application I'll need to install the library which I've done and copy over this connection client as well as generate a token which I can do just over here just make sure if you generate your own token keep it secret and cop it into your environmental keys I'm going to do that right now creating a EnV file and then I'm going to create a new attribute here called Astra unor token in nextjs I'll create a folder called API and in there I'll create a folder called ad I'll create a new file called route which will be the API route I'm going to use to add my first Pokémon here I've got a basic get request using nextjs I'm going to add in the data API client with its environmental key and URL and I'll first test to see if I can pull out my collections from my Pokedex database and here it is my Pokémon collection is available now that I'm connected to my database it's time to move on to the next part now I want to figure out how to use an AI model to be able to analyze photos and provide descriptions of them I'm going to head to Google and search up open AI I'm not just a fan but one of their most recent models that they've released is actually probably going to be the best solution for this project I'm referring to GPT 4 Omni or GPT 40 available over here in the documentation this model of Chad GPT actually allows you to add a URL and it can be an image which it can analyze and tell you exactly what's in that image but on top of that you can also upload images using base 64 encoding so that's what I want to kind of do as part of the Pokedex design design I also need to consider the size of the image no larger than 512 x 512 otherwise it's going to use more than 65 tokens and I'm going to be bled dry by the pricing and just as one double check I created my own prompt with an uploaded image of a Pokémon to make sure that Chad GPT could analyze the image and it could so I should be able to use it in this manner now I just have to build the API to send it images heading back to nextjs I created a basic landing page where I can collect statistics about a Pokémon or a picture that was captured things like weight height speed Etc I started with a simple form where I could upload a file and this file would just be an image I would have nextjs query the open AI model and I had this prompt which I generated it was just a very small simple prompt to let chat GPT know it was emulating a pokedex and success I was able to detect different things in an image like a Pikachu however I still didn't have any stats such as HP attack defense so I needed to do another query for that I want GPT to give me a response back as a Json object so I created another system prompt basically asking for Json objects as a response and success and now I have all my stats along with the description of whatever the object is next rather than uploading files I think it would be better if I take a photo directly from a camera whether it's a webcam or a camera on a phone I even created a specific design for the camera page where hopefully people using their own phones would see a preview of what they see through their camera lens to figure out how to do this on a web app I Googled a JavaScript camera API and searched up if I could natively create some commands to be able to do this the get user media method from the web apis actually does work however I kept running into permission issues and I wasn't going to create a Pokédex that required you to manually upload files to get details about them I wanted that authentic experience almost like you're playing the game Pokemon Snap and taking photos of real life I continued to debug for days and in the end I decided why not just use an existing Library like react webcam which soled all my problems it was a simple single component I could use and I could even select the front-facing camera now I ran into another problem the iPhone and any phone creates massive images and these are just too large to upload to open AI so back to Google I went Having learned my lesson last time trying to solve things manually I found an npn package called react image file resizer I could use this function right here to plug straight into my code and even have a 512x 512 image that's a base 64 with the functionality working I want to now implement the design and Aesthetics I set up a user store using mobex and implemented all my functions inside of it then I wrote a few lines of CSS a few thousand I'm not going to go through how I wrote each one of those but suffice to say I have an overall design now of the Pokédex some things like the header or the background I just imported as SVG objects while other Pages like the camera I built from scratch I had this cool idea to use the sound effect from Pokémon Snap whenever you press the button to take a photo and I managed to find this really cool react Library called use sound where I could simply import an MP3 like this sound effect right over here and place it straight into my react code and this was the result this means I can now take a photo of an object and within a few seconds I get this really well-designed response based on the figment design I put together however I'm not done yet I always remember this statistic graph whenever playing Pokémon and I wanted to recreate it I know chjs is a great library for graphs and I thought I could probably use one of their graph types to recreate the same kind of design going through their site they have this option for a radial graph and this was just about perfect I headed back to vs code and installed the react library then I added it to the preview page and I have these stats which are now visible inside of any entry that is made since I want the stats to actually mean something numbers like the height are measured in centimeters weight in kilos and speed in column per an hour this means that a bird would have a high speed of like 30 km per an hour while something like a mouse might only have 5 or 10 next I had this great idea to generate the original Pokedex voice from the TV show I headed to 11 Labs where I've had success recreating my own voice and even popular voices like that YouTuber called fireship to train a good AI model you need about 3 hours of footage and while I tried to create a model there just wasn't enough footage with most Pokedex entries being about 10 seconds and unfortunately this did not sound very good I did some Googling to look for other options and I came across this website called fake you they had already trained hundreds of models and I had to look if they had the Pokedex model available and to my luck they did so I gave it a try I gave it a simple prompt to see if it would sound like a pokedex at least this sounds a lot more like a pokedex I headed to the API and had a look at how I can implement it into my own njs app I created Creed a special API route called voice and this would request a voice of the Pokedex and then send it to the client now when I take pictures of things and analyze them I can generate the original Pokedex voice copy in a glass one it's able to explain the entire description though I'm not sure how appealing this voice model is with this complicated task now done I want to implement logins maybe authentication using Google or GitHub and I want a simple login without a username and password because there's a lot of Services out there since I'm using nextjs I thought the best type of authentication for me to use is next o they work quite simply with a provider for both the front end and back end so that I can immediately have tokens for users and I can authenticate them directly from whichever service they want to use I followed the setup guide which just required me to run mpm install next o and then I just needed to create a folder inside of app API and O with a route that allows you to create the login sessions now I can can load up my app and select login and I'll have an option here to login with Google or which is the one that I've set up so far this takes me to a profile page on the Pokédex and then I can also sign out if I want to log into other accounts now that I have a login I want users to be able to save the entries they make into their own Pokedex into a small database and on top of that I want to create a leaderboard where people can see who scan the most entries so I'll need to head back to estra DB and update my database here for my Pokédex I headed back to nextjs and started coding these functions I put together a route for the pokeddex and the leaderboard and I started storing and retrieving any entries users made this was the result now you've got a preview of all your entries which is great you can click through to them and view the entry as a whole including the description as well as the audio in my design I also added this option here to make these entries searchable the way I'll do this is with Vector embeddings a vector embedding is a numerical representation of data that captures its meaning in a multi-dimensional space you can use these vectors to find other items that are closely related to the item this actually looks like an array a very complicated array with lots of numbers like this vectors can have hundreds or even thousands of numbers representing their Dimensions this can then be plugged into a search not all databases can do this but the one from data Stacks can here in astrab I've got the vector for it that I generated earlier from that function and now I'm going to paste it here into the vector search ideally the very first entry should be the Dragonite with other dragon types being available as well so I'll select apply here and you can see how closely related they are with their similarity score as well as on the right hand side the object they are so Dragonite Dragon Plush lion and Charizard all fit together this means that in my search bar now I can type in dragon with a similarity cut off that will take me to my entry but it'll also find other similar types to this like other dragon types if I had captured them and so I'm finally here with a working Pokedex I was so excited I even bought a Pokédex phone cover I can log into my account view all the entries I've already taken photos of check out the leaderboard of who's made the most entries and of course take a photo of anything I want in this case I'll just take a photo of my bookshelf GPT Vision creates the description as well as all the stats I've got the photo saved and I've even this audio it's being generated in the background that I can play and Listen to If I want to just hear the description and now it's in my Pokedex entries I've also published this online so if you want to check it out and test it yourself just go to Pokedex arl.com I'm also going to open source this project and I'm going to hide a secret inside so if you find it you might be eligible for a prize and if you want to help me add any more features then any pool requests would be accepted finally a big thank you for everyone who watched this video I spent a good month or two on this project so I hope you all liked it