Transcript for:
Building a Cost-Effective Custom API

in this video I'm going to show you how to get rid of your monthly API services that you use in all of your automations we'll build out our own API that we can deploy to our own server on the Google Cloud platform and if that sounds complicated stick around cuz I'm going to show you how to do everything step by step and best of all all of these tools are free and you can use them in your business today so there's a lot of great tools out there like Leonardo or 11 Labs or Json de video or chat GPT that helps us build out our automations for our projects this here is an automation that I built for my faceless video generator but the problem is we end up using these apis very heavily and it becomes quite costly as we add up all of these different Services some of these apis are quite costly and we don't actually use all of their services we just need one part of it or their pricing structure just doesn't match how we actually use the Automation in this automation here we had to do a lot of merging of video and audio in these automations right here and so it started to become very costly to run this automation at the scale we wanted so instead we're going to build our own API with some of these common functions that we need a lot inside and then we'll deploy that on the Google Cloud platform which will allow us to run all of these services at a very low cost and then we can take the other API services that we've purchased that are helping us complete our automations and we can just get rid of them and over time as we identify other apis that we need to replicate we can isolate those and then also migrate those into our API over time and not only that but we can create our own make module here you can see I'm working in the no code Architects toolkit and this acts as a wrapper to to make working with the API much easier the same way we use other automations like Leonardo in our make automations so if we take a look here we have one module here that will allow us to convert media to MP3 we have another module here that will help us transcribe media this will replace chat GPT whisper and it will also allow us to process files that are larger than 25 megabytes and then we also have a function here that will combine videos and then over time as we isolate other apis that we want to replicate and bring into our API we'll simply Target those and develop them and release them as we go and we already have other functions that we're working on like audio mixing overlaying audio on video and also being able to upload large files to Google Drive which is quite difficult to do inside make natively and so you can see here over time we can identify additional functions that we need inside of our toolkit to help us reduce the costs in our automations and we can even work together as a group to identify the different functions that we need to add into our toolkit from our automations to save costs and then we can expand the toolkit as needed and if you're wondering how this works all an API is is a server with some code on it that when you send it a request it does some sort of processing and then sends that request back to you so what's happening here is there are some developers and companies developing this code and putting it into the API and then they're charging you a fee to communicate with the API so we can develop our own API and then we can simply put in our own code into that API and make it easy for you to deploy it on Google cloud and then you can simply call your own API and get that response back instead of having to pay for the custom apis now this doesn't mean we're going to try to replace every API out there that's really not practical but what are those commonly used functions that we use that are very costly how can we just Target those specific things build them out and then use them at a lower cost so in this video I'm going to show you how to deploy your own API toolkit on the Google Cloud I'll show you how to use that API in make and a cool little air table database that shows you how to track all of your requests to your own API and the response from API here we have videos and transcripts and SRT files all being processed by our own API and if you want beta access to these make modules and the base template to this air table database make sure to jump into the no code Architects it's a growing Community where you can get answers to all your questions including a make automation course and access to a bunch of templates like the video you're seeing today but everybody in the no code Architects will get access to these make modules and will be able to work with us closely as we work through the beta and we're selecting different features and apis to replace now one of the first things we need to do before we can call our API with our make.com module or just the typical HTTP module is we need to actually deploy the service and you can actually get a $300 credit from the Google Cloud platform just make sure to check out this link here and start free and you'll get a free $300 credit that lasts up to 90 days after you've created that account you're going to want to go to your Google Cloud I'm at console. cloud.google.com it's already pre-selected a project that I have already worked on if you're here you're going to going to want to create a new project I have to come up here to new project you want to follow the same path I'm going to go ahead and type NCA for the no code Architects toolkit and then I'll put demo and then I'll go ahead and click create looks like I can't have in the parentheses so I'll remove those create going to go ahead and start creating that project for me once it's ready I'm going to go ahead and select that project once you created it you should see your project here under your domain I can close the notifications and then the first thing we're going to do is to come up to the menu here and we're going to go to API Services we're going to go to library we're going to type in Google Cloud Storage we'll hit enter and then we're going to go ahead and install the cloud storage API make sure you're picking the same one that I am here you want to make sure that that API is enabled then we're going to come back and go back to the library again this time we'll type Google Drive API we hit enter then we're going to add the Google Drive API go ahead and click enable if it's not already now we're going to add one more we're going to head back to the library and we're going to type Google run API we'll hit enter and here we're going to add the Google run admin API go ahead and click enable so there we go the first part is done now we need to go and create a service account so again I'm going to come up to the menu then I'm going to come down to I am and admin come over to the submenu and I'm going to go to service accounts here I'm going to create a new service account for the name I'm going to use NCA toolkit service account then make sure to create and continue then we're going to select a role we're going to type storage admin grants full control of buckets and objects and then we're going to add another rle here we're going to type viewer we're going to select that role and we'll go ahead and click continue and then we'll click done so now you have this new user here you go ahead and click into it then we're going to come up to keys and we are going to add a new key go ahead and click create new key and you're going to pick Json go ahead and click create now notice that it's going to download a file to your downloads directory keep in mind that that is there notice in my downloads I have that Json file here it downloaded automatically and we'll use it in just a moment go ahead and close this here now we're going to come back up to the menu and we're going to go to cloud storage and then we're going to go to buckets we're going to create a new bucket so now we got to name our bucket I'll call it the NCA tool kit bucket go ahead and click continue for now I'll go ahead and select the defaults multi- region I'm really going to accept all of these defaults here and I'm just going to go ahead and create the bucket so by default when you create this bucket and this bucket is where we're going to store files that we create in our API it will not allow the public to view those files but indeed we do need to allow the public to access these files and you'll see why in a bit so I'm going to go ahead and turn this off enforce Public Access prevention on this bucket I'm going to go ahead and click confirm and in order to make things public we still need to update the permissions a bit so click on on permissions then we're going to scroll down here and Grant access for new principles we're going to type all users then we're going to select that here and then in the rolls here we are going to type storage object viewer and we're going to select this here we're going to go ahead and click save and now it's going to warn us that we're making all of our resources public and we're going to go ahead and say allow Public Access and so now that we're seeing that warning here we are good to go so now that we have this set up we're going to head back to our menu here and we're going to come down to Cloud run this this is where we can actually deploy that server we're going to come up here to deploy container and we're going to pick a service here we're going to leave this pre-selected for the container image you can simply copy what I'm pasting right here letter by letter make sure you copy it exactly then you're going to pick a service name I recommend picking the no code architect toolkit I'm going to add a dash demo because I already have one running with a similar name it's going to give you this endpoint here which is going to be the URL of our own internal API for authentication we're going to go ahead and click this allow unauthenticated for CPU allocation and pricing we're going to go ahead and click CPU always allocated for service autoscaling you can leave that at zero you can leave this as is on all then you're going to want to expand this section here container volumes networking and security you want to make sure it's on Port 880 you can go ahead and leave these blank for memory I'm going to go ahead and move it to 4 GB I'm sure you could use less but this is where I did all my testing so I'm going to start there over time I'm sure we can use fewer resources and make it even cheaper you can leave CPUs at one for maximum concurrent request per instance we're going to move that to one for execution environment we're going to leave that alone and for revision Auto scaling you can leave it at 0 to 100 if you want to limit the number of instances that you run at any given time you can always lower that startup CPU boost you can leave that on now the only thing we need to update now is the variables and secrets so go ahead and click on that going to go ahead and add a variable the first one we'll add is the API key this is the key you're going to use when you call your own API I'm going to just go with test 1 2 3 then we're going to add another VAR here here we're going to add the gcp bucket name and then here we're going to need to use that bucket name that we created if you don't remember if you don't remember what that is just go back to your menu go to the cloud storage and just use this bucket name here you can even just copy and paste it put that into the value we're going to add another variable storage path in this case here we're going to type gcp in upper case then let's go ahead and add another variable here we're going to add the gcp say credentials and now for the value here we need to go to the file that we downloaded when we created that service account so go ahead and open that file make sure you're using a text editor and then you can simply copy that entire section there come back to the browser and just simply paste that into this value here and so with that you should be able to come down and create your instance it's going to take a couple seconds for that to deploy and if you're curious this here is where the actual code for deploying your API is located it's on a platform called called Docker Hub and this URL points directly at the code to download for you to deploy and again what's cool about this is that it's completely free to use this package and deploy it you just need to pay for your server on Google Cloud but this will be a lot cheaper than having all of those apis the first time you deploy it it'll take a minute so just be patient all right so now that this is deployed we can go ahead and test this out so for testing I'm going to use this program called Postman it's a program that I recommend you download so that you can test your API calls it's a free app that you you can get at post man.com and so all I'm really doing here is simulating what's happening in one of these modules here to our new server it's really not that complex I created a new tab which is going to allow me to create a new request I'm going to move that request to a post and now I'm going to come back to Google cloud and I'm going to grab this new URL that I added come back to postman and I'm going to type in that URL and then I'm going to put a slash the first API call I'm going to test is the transcribe so I'm going to go ahead and put slash transcribe at the end of this new URL then I'm going to come to the headers here and I'm simply going to type in X API key and remember when we set up our environment variables for our new container for the API key we use test 1 23 so I'm going to put test 1 2 3 just like that and I'm going to come to the body section here and I'm going to move this to raw and I'm simply going to put in these curly braces here open curly brace and close curly brace now in order to use this transcrip describe endpoint we need to pass it a media URL so you'll want to use mediacore URL with a colon and I've got a bunch of Google Drive links that I've been using as a test so this is a link to a public file on the internet that can be downloaded so it's you got to make sure it's public and it's just a video that I can send to the API to have it transcribed so I'll add a quote then I'm going to add in the URL just like that then I'm going to put a comma and then we have to select what the output should be cuz we can do transcriptions or SRT files so I'm going to put output transcript just like that and I should be able to now run this API call I'm going to go ahead and click Send that's going to take a second to run but it is now calling this container that we built previously and there we have our output so we have message equals success and we can see the response which is the transcript from the video that I sent it now again remember this is a Google drive file that I made sure was sharable on the internet so you have to change the permissions on that video file also for testing make sure you use a small video file not a large L one large files will cause an issue anything over 100 megabytes is going to cause an issue for your testing so make sure it's 25 megabytes or lower just to make sure that this process happens quickly and your testing isn't overly complicated with large video files so we got that working I want to test one other API call which actually generates a file this one only generates text so now what I'm going to do is modify this request to Media to MP3 and what this one does is it actually takes Media or a video file and it converts it into an MP3 and so this one actually has to generate That mp3 and save it to the Google Cloud Storage if you remember when we set all of this up we created a Google Cloud Storage bucket and so when we create that MP3 it's going to be created here and then the API needs to send it back to us so we can actually access it so here I just changed the endo2 media to MP3 and then I can erase the output transcript because we're not creating a transcript so now I'll go ahead and run this request on the new endpoint media to MP3 with the video and we'll see what we get all right so here you can see we got a success and it's also returning a URL that contains the MP3 so if I were to Simply cut and paste this URL into my browser here we're going to see that we now have an MP3 that we can listen to we've had clients that triple the amount of content they're producing and that is the same audio that was in this video that we passed it here so now that we confirmed the transcribe endpoint is working and also the media to MP3 endpoints are working and the API is actually producing the MP3 and sending us back a URL that we can access and download we can continue on with the rest of the video all right so now that we've tested that our API Works using the postman app how can we actually call our remote API from make itself so again remember I did mention that I have these new make.com modules that you can use the no code Architects these are built-in make modules that you can use and access currently these modules are in beta and are only accessible to members in the no code Architects but you can still access access your own API without these modules it just takes a little bit more work so I'll show you how to do that now and then after that I'll show you how these modules work in replacement of those so if you want to make an API call to your own API just like we did here in Postman you can use the HTTP module so I'm going to go ahead and add the HTTP module here and we are going to make a request so I'll go ahead and click on that and now we just simply need to fill in the same information like we did here in Postman so we have our URL right here I'm just going to go ahead and grab that and then I'll add that to the URL here I'm going to change this to a post in the headers here I am going to add X API key and I'm going to type in the key that we used when we launched our instance test one two3 for the body type again I'm going to go ahead and click on Raw for content type we will use Json application Json and for the request content I'm simply going to go back here to what we used inside of Postman and I'll copy that here in the request content and then I'll go ahead and update parse response to yes I'll click okay and now if I run this module once it's going to do the same thing that Postman did it's going to call this with this media URL and it should return a response here's our response here's the data so we have a success and now we have that MP3 that we completed as a part of this request so again we took this video and converted it into an MP3 and so from this point on we could add additional modules to do whatever we need to do with this specific MP3 file so this HTTP module works just fine but it's actually quite a bit easier to use this new noode Architects toolkit API using the no code Architects toolkit make modules so I'm going to go ahead and delete this HTTP module and I'll show you how these work so what's nice about the no code Architects toolkit built-in make modules is that it works just like an actual module that you've used before plus it has all of the endpoints and the different functions that we use easily accessible and when you add one and you go into it it simply gives you a place to put in the different variables that you can send so so far we've only been using the media URL and I haven't shown you the web hook and this ID yet I'll show you that in just a second but before I do that I also want to show you how you make your own connection so I'll delete this module and then I'll create a new one and then I'm going to add a new connection so when we do this what it's going to do is it's going to ask us for the X API key and again remember we were using test 1 2 3 and then it's asking us for our specific toolkit URL so again remember we deployed our own server so I'm back here in the Google console and here was the server that we launched and again remember we have this URL here so we copy that we'll come back to make and we'll add our toolkit right here and then I'll name it and then if I click save then it's going to save that connection for the long term and then I can continue to use it so now instead of having to use an HTTP module I can simply pass in that media URL directly into the module itself I'll grab that right here here I'll add that to the media URL I'll go ahead and click okay and then I'll go ahead and run this module only so it's only running this module with the new connection and then we get that output message success with our response which is the new Mp3 again we could take this and copy and paste it directly into the browser and then we can go ahead and play it and listen to it just like we did before amount of content they're producing each day and so there you go there's a a test new module and then we've also got modules here for transcribing media just like we did over over here in Postman remember we transcribed media so we could do the same thing here and then we could also combine videos and again that will come in handy when we're doing things like faceless videos where we're generating lots of videos and merging them together with audio and then again we're going to create a lot of other endpoints here too like audio mixing and really anything that we want to develop as a group now I haven't talked about it yet but there is a really cool feature in the no code Architects toolkit API that is also available in the make.com modules which is a web hook and ID so sometimes when you're processing these files like processing a video to MP3 or combining multiple videos into one and also trying to transcribe a very large video file sometimes these processes can take a long time to process and in those cases you can use a web hook so that when the process is finished it will send you the information to your web hook later which was the same process I was using in my faceless video generator so what you can see here is I have a very basic air table database that is set up and we're tracking just a few different fields that come back from the API when we actually call it so if you want to go ahead and create your own aor table base go ahead and replicate this where you have a request ID you have a text field for an endpoint you have a text field for the ID you have a long text field for the response a text field for the code a long text field for the message and then just a created timestamp and then all I do is I have this automation here which is turned on and is looking for requests coming into this web hook so if I did this from scratch I'll come down here and create a new Automation and then I'm going to see all down here and then I'm going to look for when web hook received and then I'm going to copy this web hook into the request we make on the toolkit so let's just go ahead and create a new one real quick I'll go ahead and do the media to MP3 and again you'll create that connection if you haven't already done so and here I'm going to put the web hook URL and again remember we copied that from the new automation we are building here and then we need that media URL so I'm going to come back to postman real quick and grab that URL and then we'll drop that here in the media URL and then the final thing that we need to do is we need to add an ID any anytime we send the web hook URL we need to pass it an ID that we can reference later so here I'm just going to do demo 1 23 I'll go ahead and click okay and then I will run this module only and now notice that it didn't try to actually process and return that MP3 right away it just sent us a message back saying that it is now processing and now if I jump back over to air table we can see that something sent us a result to our web hook and if we take a look inside we're going to see the endpoints that we called which was media to MP3 this was the ID that we passed at demo 123 and this ID allows us to keep track of all of the different requests that we make to the endpoint and sort them out when the endpoint is actually calling our web hook and then we have the response the code and the message and so we could simply come here and add a new create record and update the record here in this table and then we can update the endpoint that came from the web hook the ID the response the code and the message so you would just add all these fields like this and then you would just simply map in the data from the body endpoint ID Response Code and the message and now if I were to run this just like that come back to the data we're going to see in our log for the API that we have the media to MP3 we have the ID that we sent it remember if we come here we sent demo one 2 3 and so that came back to us here it actually sent us the URL for our MP3 here we have the success code and the success message and the timestamp so then we could come back to make and let's go ahead and add the other options that we have I'm going to do the transcribe media for the media URL we'll use the same file that we've been using we'll go ahead and create a transcript you could also create an SRT file for the web hook we'll pass in what we sent the previous one just like that and here let's just put transcript demo click okay we'll go ahead and run this module only and again remember because we're passing the web hook it says message processing right away it doesn't actually process it in make it's going to process it and then send it to our air table database where we could then trigger further automations and then we can go ahead and add in one more endpoint here the one that we have is combine videos and here we can add in multiple URLs so I'll add in one video here and then I can add in another video and it's going to merge those two videos into one and then for the web hook I'll grab the web hook that we've been using drop that in here and then for the ID I'll type combine videos I'll click okay I'll go ahead and run this module once only notice it returns right away as well processing and then if we come back to our error table log we're going to see we have the transcript coming in that's the one we did right here with the transcribe so we can come in we see the ID that we passed it and we can also see the transcription of that video with the success codes and then we just need to wait for the combined videos to run depending on how large the videos are and how much resources you gave your server when you created it it'll take a few minutes for sure now looking at the log it looks like there was another transcript that I ran that just came through as well so now we're just waiting for that combined videos to come through it takes a bit longer than the others and there it just popped in there's the combined videos so if I take a look at that final file here we can see it's a one minute video and I happen to know that both Clips happen to be 30 seconds so if I jump over to to past 30 seconds we can see it's a totally different clip that we merged together two into one now again I do want to mention that we are not necessarily going to try to replace every single API out there that would just really be impossible but we are going to focus in on what are the API functions that we need to run our AI automations and reduce cost are there certain functions that we can pull out from apis that we use a lot and get those into our own API toolkit so we can launch This Server and access these make modules and get our work done at a much lower cost and again what I think is really great is that as a group we can isolate the functions that would be the most beneficial to the most people and we can build those out release them and then we can just keep iterating and do more and more so again if you want to get early access to the make.com noode Architects toolkit along with tech support to help you get this up and running in your automations make sure to jump into the no code architects in the beginning I'm going to be focusing on the members in this community to get this launched but again you don't have to use these modules you can always use the HTTP module like I showed you earlier in the video you can deploy everything and use it just like that and if you want more information on the API toolkit itself it is on GitHub you can download it for free there there are readms on how to use it and all the different endpoints that you can call we're just getting started here I think this is going to be an amazing tool for the AI automation space I hope you found this video valuable and I'll see you on the next one