What's going on guys it's Nick and in this video I'm going to cover basically the final important thing that you need to know in order to make make.com work for you and that's what to do when things go wrong. So error handling, rate limits, and the many common and sort of inevitable issues that you're going to face if you choose to develop in make.com in any sort of meaningful capacity. So if that sounds like something you're interested in improving stay tuned and let's get into the Okay, so first things first, if you guys have watched my videos up until now, you'll probably have noticed that I use a particular design pattern in production-ready flows.
And by production-ready, I mean flows that I am sending to a client. I mean flows that I'm using myself in my own company. Essentially flows that make or maintain money.
And if these flows do not work all the time, it's a big problem. And so what I'm going to do in this video is I'm going to show you two different ways that you can deal with that. The first is going to be a built-in make.com way.
and the second is going to be a third party that probably less than one percent of make devs know about that essentially allows us to put a gateway before anything reaches our make app or add a gate after um our make app like you know if there's like an outbound request or something like that and these are two ways that you know allow me to manage well over 75 to 80k a month just in revenue for my own companies just using these sorts of flows so very simple very easy and By the end of this video, you'll know everything that you need to know. I'm not going to harp on all of the error handling modules because I think most of them are bullshit, to be honest. We're only going to worry about the ones that actually matter.
Okay, cool. So yeah, this design pattern that you guys usually see is there's just a bunch of modules here. And any module that calls an API, I put a little break module on top. So get a task for ClickUp, that's an API, right? Search for projects, Google Drive, that's an API.
Google Docs over here, that's an API. So anytime I'm calling a third-party resource and there's like a 1% chance, that out of a thousand calls that resource will be down. I just need to provide a way to to handle that right because it's not these are third-party platforms we don't manage them it's not like I have a server in my basement to like deal with all these requests that I can personally guarantee you know uptime for.
These are services where there's a million and one other people that are trying to access it there are a million and one other things that could possibly go wrong. Sure they technically have what are called service level agreements, SLA's usually, but I find in practice those don't really mean shit. I mean Typeform was just down for like 14 hours or something like that. I think ClickUp was just down for a day. You know, these are services that make me a lot of money and every second that they are not are a big problem.
So yeah, you know, I'm gonna show you how to like build out the same sorts of flows. Now, what is a break module? Basically a break module says, hey, if this module, the one that the break is connected to, should error out, you know, if instead of that beautiful green little bubble we receive a red one, then I want you to hold all of the data.
that was sent to that module. Then I just want you to wait a little while and then try again. So this essentially allows us to build out a system that can hold data with, you know, assuming that there are any issues in the flow and then just retry on a regular schedule that we define until that flow works or, you know, until we run out of retry attempts.
And in this way, you can basically virtually guarantee that if it's just a simple rate limit issue like You just call ClickUp or Google Drive a million and one times or something like that and they're like, hey man, slow the hell down. You know, if it's just a rate limit issue, it's basically never going to be a problem because you're just going to like drip that out later. It just lets you build a little bit more flexibility into these things.
So if I click on this, what does this actually look like? It's called flow control. You'll see that there's automatically complete execution, which I believe is just always set to yes by default. But the important part is number of attempts and then interval between attempts.
And so if you read through this little. Info here, it says, hey, this is the maximum number of attempts allowed for the execution to be tried and completed automatically. If they fail, the execution will have to be completed manually.
And so manual completion means that you will literally go into the make.com logs and then you will click retry yourself. And so this is actually just a piece of functionality that the break module allows you to do. If you don't put the break module in and if you don't toggle a setting, which I'll get into in a second, you can't actually ever retry a scenario manually. This actually allows you to catch the data and then retry with that data, which is super, super useful, especially when you're testing.
And then down here is interval between attempts, so just the number of minutes that you want. If the interval between attempts is 10 minutes, guess how long it's going to take between attempts? Pretty self-explanatory.
So that's the break module setting. In order to access it, you need to go down to these more settings. Oh, I lied.
You need to go down to the scenario settings, little gear icon, and then you need to scroll down to allow storing of incomplete executions. And then you need to tap yes. Normally it's no by default.
But by doing yes, now we'll have, we'll basically provide make the ability to like, if there are any issues, it's going to break the scenario and it's just going to wait with that data until we reach those cutoff times that we just set up earlier. So you can read through all this stuff on your own, but yeah, pretty, pretty high value flow over there. I'll give you a quick example.
So this is just a little default that I just set up here. So I just created a new scenario and I'm just going to add a ClickUp module. I'm just going to get a task and then ClickUp allows you to like.
add an ID essentially to try and pull a particular task. I'm just going to write a random ID that doesn't exist. And we're going to see what happens.
I have a feeling we're going to 400. That's exactly what happened. We received a 401, team not authorized. And this is actually just a quirk of ClickUp.
For whatever reason, these guys do not, they don't tell you exactly what the error is. They have really shitty error handling. But what I'm going to do is I'm going to go to this little settings icon, click allow storing of incomplete executions.
And then I'm going to go to this blue or a green here. I'm colorblind or it's early. one of the two and then I'm going to add this break module.
I'm going to click on it. I'm going to look at these settings. Three attempts, 15 minutes interval between attempts.
That looks pretty good. I'm actually just going to put it on top because I think that's more representative of what's happening. And then I'm going to run again. There's going to be an error, but now you'll notice that the break module has actually gone through. And you'll also notice that there was another message down here that says the operation was completed with a warning.
401-0-027, team not authorized. This is different from what we had before. So, okay, I think I'm just clicking the wrong hotkey here.
Yeah, my bad. But anyway, if I go back now, you'll see that there's a new tab that we've added to the right-hand side of our scenario editor, and that's the incomplete executions tab, which is pretty neat, right? So if I give this a quick little click, you'll see now that I'm basically being taken to this specific run that had the problem and that was caught by the break.
And so if you click on this red dot over here, it'll tell you what the issue is. Hey, incomplete execution. And then if you click on this get a task module, you can actually go through and then you can just run it again.
And so let's say I'm actually trying to run this on a real record in ClickUp. It's going to go down to the content pipeline and then get this task here. The IDs in ClickUp are always just stored up at the very, up in the URL. Then I'm just going to do this and then I'm going to click run once.
Well now you'll see it has actually gone through. We initially queried it with a task that didn't exist, a fake one. We obviously got the error, but then it held that and then It allowed us to click into that specific run and then run with our own data, whatever new data that we wanted or you know we could rerun it or what have you. Now what I just did is basically what the break module would do. The only difference is the break module would be running again and again with the same id but you guys can kind of catch the drift right.
Like usually there is some piece of data that is feeding into a function like the ClickUpGetAtask module and you know if that thing screws up then this id is going to be wrong. But with the break module you can catch that and just run it again. And, you know, usually if you're doing three times with 15 minute intervals in between, usually the second or third time will work if it's an API issue.
So I just do this or I say this really to just encourage you guys to, you know, in practice, this is what you do. This is like the step one, two, three. You go down to the scenario settings module or a button, you click allow storing of incomplete executions.
And then just anytime you're calling a resource, just drag and drop a break module on top of it. It'll fix probably like over. 50% of issues just up front like that.
Yeah, so that's probably like the simplest way to implement error handling. There are, of course, a number of other flow control options here. There is repeater, which I never use. Iterators, which I have shown you guys in a previous video, I do not like to use. Aggregators, which I've shown you guys in previous videos, I also do not like to use.
You can get far with make.com without using any of these advanced flow control functions. Writers, which I don't like to use, but I sometimes will use. And then these are... really like the error handling.
There's like rollback, break, resume, commit, ignore. Just for the purposes of like being complete, I'm just going to run through them, but you will basically never use these in practice, except I lied. I apparently did use this here in practice once, but there must've just been a very particular issue which resulted in me doing that.
Okay. So there's break, rollback, there's resume, then there's commit, and then there's ignore. And I probably have to connect these, don't I? I do. That's sort of annoying.
Let's go over here all the way to the end and just connect these one by one to my harvest module. Okay, I was just doing actually a little bit of documentation reading because I had never used the rollback module in my whole life and for good reason. It doesn't really do much, but over the course of the next minute I'm just going to run through these regardless.
So the rollback module essentially just allows you to stop the flow immediately and then mark it as an error. So you can imagine how I don't really see much utility in that and I'm never really going to use that in practice because if there's an issue with a module and that module has an error, it'll stop the scenario immediately and then just mark it as an error. So I don't really need this rollback module and that's one of the reasons why I've just never used it in practice.
The rollback module is sort of the opposite of the commit module. What the commit module will do is the commit module will, if there's an error, it will instead mark the scenario run as a success or as completed and then it'll stop the flow. So again, I don't really see any real reason why we need to use the commit module unless you wanted to intentionally have some type of incorrect information.
I wager that these are mostly just hand-me-downs from IntegraMAT from way back in the day before they really determined the best way to route stuff and error control and that sort of thing. They just haven't really changed that. The ignore module, this just does exactly what you think.
If it runs into an error, it's just going to ignore that error and it's going to pretend that that error did not happen or did not exist. I believe it continues the rest of the flow. And, you know, there are some instances in which you might want to use this, as you see over here. I think that this API endpoint that I was using for ClickUp at the time was pretty buggy. And like every time I'd call it, it would do what I wanted it to do, but then it would return an error.
And I just didn't really want the errors to come in. And if you get a certain number of errors, it'll eventually turn off your flow. So pretty sure we just drag and drop that ignore module there just to deal with that.
So I could see a little bit of utility in the ignore, but really, I don't really do that ever. And then the resume module. This is interesting and maybe a little bit more technical. Basically, if there's an error, hypothetically, at the apply budget to project module here, we can take in the status code, maybe that's returned, like the 400 status code, and we can just turn it into something else. We can basically just map over that and say, no, that isn't a 400 status code anymore because I waved my magic wand.
Now it's a 200 status code. And then if you have some other functionality later on, some filters that depend on that, you'll use the output of the resume module instead of the output of that. So, yeah. In practice, I basically never use these. And I imagine if you guys are following along for the first time, or you guys are just interested in making a large amount of money with this platform, not necessarily just being the master, I was about to say the master baiter, the master composer of flows, but to be fair, masturbation is, yeah, that's what a lot of people are doing on Make.
I think instead of making money, then, you know, maybe you'll disagree with me, but hey, c'est la vie. So that's the break module. That's one of the ways that, you know, you can use error handling in your flows practically. And again, in the real world, all I would recommend is just go to the scenario settings, click allow storing of incomplete executions, and then just drag and drop a break module on top of every API call.
The second thing that I want to show you is I want to show you a third party platform that I find extremely useful in dealing with higher demand rate limit stuff. So like really high volume applications, or maybe scenarios running every minute or every few seconds. And then this one also just has a bunch of really cool functionality, which I think you guys would be interested in. So I'm not affiliated with this platform in the slightest, but it's called HookDeck. And basically the way that it works is, as I mentioned earlier, it's like a gate where like, if you're sending a request to some scenario, let's say you're catching a webhook.
If you send, you can send it to web, or sorry, you can send it to HookDeck first. HookDeck then basically just like adds, like adds it to a queue, creates a bunch of queue functionality for you. And then you can like transform every one of those records. You can do whatever the heck you want to that. And then after a predetermined amount of time, it can then pass that to your make scenario.
So basically, it's just a way to transform data before it comes in or change the frequency of data. And you can imagine how it's extremely, extremely valuable if you're doing some sort of high volume flow. And so I'm not actually going to go through the rigmarole of signing up and stuff like that. But rest assured, it's quite easy.
Just go to hookdeck.com. And then there should be a Get Started button up there. You should be able to just jump on.
It's free. And then I'm just going to show you how to practically use it. So go over to connection, create a new source, and I'll just say example. Let's just say March 10th video. That'll be easier for me to keep in mind.
And then we'll call this source. And then the destination will just be March 10th video destination. And then you'll see here there are a couple of settings I can add. One of them is endpoint URL. And basically what's happening is we'll set up basically this is like a webhook.
We're going to get a webhook from this and then we can use that webhook and send things to it. And then it'll catch this webhook and then wherever we put the destination, the endpoint URL is where we're going to send it after. And so let me actually go to a scenario called example rate limiter, which I set up for this purpose. And then I'm just going to add another one here. I'll go back to this example break module actually.
Yeah, we'll delete that. I'm going to create an HTTP request module. I'm just going to call this manually so you guys can see.
So make a request. Okay, great. And then I'm going to go back to HookDuck, finish setting it up. So we need an endpoint URL.
So I'm going to go grab this webhook URL. I'm going to copy this address to a clipboard. This is just a random rate limiter that I just set up.
So sorry, a random webhook I just set up. I just clicked on this module and then I clicked a custom webhook. And then I'm going to go back here, paste that in. There's some advanced configuration options.
I never use those. I just use the default HTTP, and then I paste in whatever the webhook that's going to be receiving the request and the receiving end is. You'll see down here there are a bunch of connection rules, and this is really the meat and bones, the meat and potatoes of this. You can transform data as it comes in. So you can apply some type of mapping rule where, I don't know, if the data is in tabular array format, you can turn every one of those array objects into a JSON object or something instead.
My terminology is a little bit off there, but yeah, you can turn. every item in an array maybe into its own object, or you can perform some type of filter where, you know, if a certain key is equal to two, you just don't forward that over to Make. You see how this significantly expands your toolset and basically just allows you to do things that are probably a lot more operationally efficient. You can delay, and so you can set up a rule where every time something comes in, I want you to hold on to that for exactly 60 seconds. You know, just keep that in your server.
Don't actually send that to our Make scenario yet. I use this sometimes. I don't really use this that often because Make allows you to just usually sleep.
But if you know about sleep, you'll know that it only allows you to do it for, I think, five minutes or something like that. HookDeck allows you to do it for days. So you could theoretically get a response, just wait like three days, and then send it three days later.
And that might be useful for you guys if you're using, I don't know, let's say somebody is finishing a project. And after you finish a project, you want to send a survey to a client. And you want to say, hey, how did we do on this project? If you want to solve this natively in Make, a lot of the time you need to use like a database or...
You need to add like a new column to your project manager. But with HookDuck, you could just say delay for three days, and then three days after you finish your project, you know, a type form or something would go. So very modular, very customizable. It's pretty cool to be able to add that to your Makeflow.
And then really like the cool part of this, or the coolest part, I should say, because I've covered a couple of cool parts already, is this retry function. And you'll see here that this is basically just the break module, just in a third-party platform. But there are a couple of additions.
And the most important one is this retry linearly versus retry exponentially. Retrying linearly usually just means, hey, I want you to retry every X amount of time. So if I were retrying three times and the limit and the interval was 15 minutes and I was doing it linearly, it'd be 15 minutes, 30 minutes, 45 minutes. But if I'm doing exponential retries, basically what this will do is, and I don't know the exact math, but the first try will be pretty early on.
So the first try might be like after 10 minutes. But the second try will be after like 60 minutes and the third try will be after like 290 minutes or something like that. Like the distance between each interval will go up exponentially.
And the reason for that is because a lot of APIs, the way that they work sort of under the hood, is they use exponential functions to define when is too much and when is too... when you're sending too many requests and when you're sending like an appropriate number of requests. So if you just retry linearly, what can happen sometimes, you know, APIs are getting better at this and they're not as hokey anymore.
What can happen sometimes is you'll just never actually be able to make it past the rate limit because the time window that you're calling linearly is below whatever the exponential function results to when they count up the number of times that you've requested in an hour. So by retrying exponentially you can usually get away from that and it sort of follows logically. We're going to try once after 15 minutes just to make sure that they aren't really easy on their rate limits and then if that's denied then we'll be like okay okay these guys probably take it seriously so we'll wait another hour or something before we try again.
If that's denied, we'll be like, okay, okay, these guys probably take it really, really seriously. We'll wait like seven hours before we try again. That's sort of the logic there.
And you can just set that up naturally in HookDeck, which is really cool. So, yeah, pretty neat. I'm actually not going to apply any filters here just because I want to be able to test it really quickly. And then you can set a connection name down there. I don't really.
Maybe I'll just do March 10th video. And then I'm going to click create. Now, it's going to give me a webhook. that I can send requests to. And this is what I was alluding to earlier.
So I'm going to go over to example break module, paste this in, and I'm just going to run this puppy. You'll see that we sent the requests over to HookDeck. There was a response. It says 405 just because it was the very first time that we did it.
But if we go back to HookDeck, you'll see that we received that that call status was unsupported method, probably because I'm just doing a get request and maybe they want like, yeah, they probably want a different Let me just see. Okay, yeah, it only supports receiving webhooks over HTTP post, put, or patch. I'm just going to do post. I'm going to run this again.
I'm going to go back to hook deck here. And this is sort of just a queue while you're testing that'll populate with all the requests that are coming in. And so you see this one was 200 just because I'm using the method that they're comfortable with.
So yeah, pretty neat. If I click through this, then there'll be a bunch more information on where the request came from, you know, how much time it took to process, that sort of deal. And then what we want to do is now we want to send that automatically to another source.
So if I go to example rate limiter, if you remember we added this as the destination, so I'm just going to wait for new data here. And what I'm going to do, actually why don't we set a rule on this first, just to show you guys that this is potentially a little more valuable than just... How the heck do I do this? I have to like click edit or something.
Somewhere around here. No, it's probably here. Oh, transformations maybe?
I forget exactly how to... maybe open connection? I basically just want to edit and then add something.
Okay, it looks like there's a way to do it here. Okay, okay. So let's add a delay and then why don't we just delay for exactly five seconds.
Let's do that. Okay, great. So now what I'm going to do is I'm going to send a request at the front end to that gate and I'm going to wait and this is going to take five seconds because, you know, we just set a rule to keep it for five seconds. So voila.
I didn't add any request options in there. I didn't add any information. So the bundle is going to be empty, but yeah, we just sent and received a response.
If I wanted to verify that this worked with my own data, maybe I'd set... the body type to raw the content type application json and then i'd say like example test and then i'd say i don't know let's do example key example value and then let's say time sent and then i'll just do now and then let's run that puppy one more time So I just sent it. We have like a time sent variable in here now. And that time sent variable should be the same on both ends.
So March 10th 2024 6 46 a.m. This should be March 10th 2024 6 06 a.m. or 6 46 a.m.
But this is like in the date time format so it's just being rendered a little bit differently. But same deal you know we verify that the data is common and gone. Yeah so again like in practice what you're probably going to be doing is you're going to be using the delay feature or you're going to be using the retry feature.
And if you catch yourself you know having issues with apis where the apis are just constantly rate limiting you or maybe you just know they're a little more sensitive just go retry exponentially and then i just use the the one hour interval you can set basically however many you want here so you could have this be like 10 or 15 or 20. i think the issue with this is the exponential function means that the 10th time will technically be more than one week in the future so i'm not entirely sure what the function is under the hood but you guys can read the docs there if you want okay great so yeah in practice I'm only ever really going to use the break module. The way I'm going to do that is by changing the scenario setting to allow storing of incomplete executions, then just dragging a break module on top of every API call. And then I'm going to be using HookDeck for high volume applications or applications where I want to delay something for, you know, maybe more than five minutes.
Maybe I want to delay it for 24 hours or 48 hours, something like that. And to do that, you just sign up to HookDeck, add a source. That source is going to be whatever the that source is going to be the URL that you use when you call a resource from Make and then at a destination that destination is going to be the web hook that you are again going to catch and make so yeah voila that's how you do it on your own i hope you guys found that helpful if you guys are gonna get questions or anything like that feel free to let me know otherwise leave like comment subscribe do all that fun youtube stuff i'll catch you on the next one bye