[Music] what is lobe ai yeah low bi is um this is smallest it's a product built by a small company that was acquired by microsoft and they're just doing their thing and it's a really beautiful thing to see they have a very beautiful app to create vision models so if you have any need in your website or your mobile app for image inference you know if you need to point your camera at something and have um and have some kind of model telling you what that thing is then this is the product for you it's free it's done locally you can uh you can train everything up locally and then save your models locally and it really works well and right now i think what they're doing is they're perfecting this experience of creating this nice uh image inference technique and then later on they're gonna you know expand what the product offers the next thing i'm going to guess would be something like um allowing bounding boxes with multiple object inference that's might be the next thing that they'll that they'll focus on i'm guessing but there's a lot of really great things they can do so if you visit lobe ai and why don't i just um i'll just minimize this and just share if you want to share my screen we can just walk through their their website so i'm going to guess um give me a verbal signal if you're seeing my screen there you go yep okay cool yeah so um they have on their website some very interesting use cases it's not just you know wandering around and pointing your phone at things and figuring out what they are it's actually training custom models so you can make a very specific use case like um in my garden right down there i have about five different very specific to new england type of plants you know maybe i want to just point my camera and figure out what exact cosma xenia uh i think there's some tomato and some catnip down there you know exactly what's in that garden um and you could and if you're a farmer you know you might have this extremely use case that you need to figure out exactly what what disease is impacting your soybeans or something like that you could go ahead and train up using your own images a model and then you would be able to give labels to your to your images that you collect and then this product will help train up a model that you can use within a web app so here's another example um if you want to you know look for emotions um i'm trying to think of a use case that would be interesting um i i've i've heard like for for emotion i've heard different use cases for you know like somebody who who maybe struggles picking up um you know facial cues or sometimes for somebody who's vision impaired and is you know trying to get a little bit more sense of the room so like a described audio there you go yeah yeah maybe a comedian could use it you know to figure out when when that laugh track needs to happen because people's dead panning for me it would be never which is why i'm not a comedian yeah exactly and then this is the kind of mask or no mask um visualization so that's that's uh that's an easy easy thing to understand and this is looking at plants so um obviously there they've trained a very specific model here's you know one two three um but it really goes through on this home page exactly how to go about training the training the model and then um it's very cool that it's you know free and private so it's a desktop app actually so you're able to just have this thing residing on your machine and then you use your own images and your own camera to collect any way you like so i'll walk through a little bit the actual products but i wanted to kind of look through and i think that that bears repeating like is you said it once before and you said it a second time but i i think it bears repeating like a third time is the fact that it's it's you said it's free and that you're going to be running it locally so you don't need um like a cloud provider that you get to keep all of the images like everything is just it's it's there on your system exactly so it had you know the security is is within your own computer you don't have to share a thing um it's an interesting architectural choice to create a desktop app um it's i think some folks have asked whether it can be used on a chromebook and i think this is something that you know they're working on because those things with those machines kind of tricky to download um this type of application so it's interesting choice to make it a desktop app as opposed to a web app but that's the choice that they made i think probably because of the architecture behind it so um it's just a really beautiful i always find it a very beautiful user interface very very easy to to uh to use so they have some project templates here and this kind of talks about their roadmap going forward so right now we have image classification so you can label your image based on its content and then the next thing indeed would be object detection so here on my desk i have you know some plants and i have you know a glass and here's my microphone and it would be able to create these bounding boxes around it to identify what's what's available and what's what's all the different objects within one image so that's really interesting very useful um and then data classification so here's where we get into textual data analysis and this becomes quite interesting for you know your business use cases um if you need to um do some financial data analysis or look at all of the um hotel reviews that are coming through for your hotel and just just check you know whether it's positive or negative you could do some some kind of classification around that i'm guessing so but this is really cool to see that this is on the roadmap because this is a product that i think i think will go far once they start offering a lot of different um opportunities and options i dig it and so the the image classification versus object detection i just want to make sure that that i've got this right so that that that image classification it's it's almost sort of like um like a a boolean value although it you know obviously could go beyond that but it's it's going to look for something that's going to be true about the entire image so like somebody's master they're not masked or this to this this plant is a tomato plant um or this plant is is a zucchini but it's not going to be able to go oh this is where the tomato is and this is where the zucchini is that's where we're going to need like that object detection to be able to go with that that next step yeah that's right here i have my you know my mouse pad and my mouse and it has to be able to to make a box around this is the mouse pad there's the mouse and neither the twin shall meet so yeah that makes sense that makes sense yeah um i've seen uh there's a product by google that uses um tensorflow what's it called the coral board and it has a whole camera associated to the board and it uses object detection to detect whether it sees in a video a car versus a truck coming down the highway so it runs really really fast and is able to watch a video so this is there's a lot of useful useful use cases for this kind of object detection okay very cool very cool yeah so um this is the way you go about using it you label train and use and it has this kind of um circle that you're going to go ahead and you know train up on a certain group of images and then that you labeled and then you're going to test those images you're going to test some new images against what you trained on and then you can correct the model and it'll learn as you go so i'll show a little bit of how that all looks in the product very nice user interface you can automatically train um just you know set set it to go and it'll go and then there's the opportunity to use your model and this is what i really like so i'm i'm a web developer but i really love applied machine learning so i love the fact that you can take a model and input it into your web or mobile app and even use it offline that's the beauty of this thing you can have that model those model files residing right within your web or mobile app and then you can do inference against it and use it offline and that that is fantastic so you're not querying you know any kind of api externally you can use it you know within your web app the trick is those are pretty big files so you have to get that right off that's the trick okay and so the the the model that that that um that would be running there is that going to sit on the server or is that going to sit on on the client it's on the client it's on the client okay now um we had um a couple weeks back we had gant um on whom if i remember you you you actually know him um but he um he was on talking about um tensorflow.js and how you could actually use tensorflow but use that in the browser for loeb is there would you be using tensorflow there as well or is there a specific api or a different um framework uh that that it uses yeah it's actually using tensorflow.js in the background actually so and we'll see how you can export it right here within that format oh my gosh the turkeys are literally walking it's it's a turkey trout right now that was just i just looked outside and there are giant birds raptors or something anyway yeah so it's using um you're able to use tensorflow.js there are other options too you can use uh tensorflow itself which is you know not the javascript port of it but the actual sdk you can use onyx as well which is really nice as another option so instead of using tensorflow maybe you want to use onyx which um which does the same thing with just a different way of doing it um so i'll show you how that all looks and here's some examples that people have built wildlife behavior elephant or not so that's the product