Transcript for:
n8n Datenfluss und Schleifen

in this video I'll be explaining the flow of data and looping within NM In your workflows you will often be dealing with many items at once These can be multiple rows of data For example they can be multiple items that we receive back from an API In a very simple example here we have one Google sheet We're simply getting all of the rows from this particular sheet So it's going to get the first three rows When we press test workflow here it flows through as one item when it calls this Google Sheets node it got back three separate rows of data and these are represented as individual data items and these are all processed separately in NN in the background So there's already some internal looping going on behind the scenes where you do not need to use splitting or looping in the background at all So in this very basic example we're reading that data from a specific sheet which is this task list sheet But then after that we're passing this to another sheet So we want to update each individual row of this sheet as well When we go into this Google Sheets append note you can toggle between schema table and JSON If you look at table you see there's three separate items there But we can map those to our relevant field here So in this example here I've dragged in this JSON title and that will go through every single one of those one by one and upload all of those to Google Sheets But it happens very quickly behind the scenes A lot of times NN will handle all of this behind the scenes and you do not really need to worry about it But there are plenty of times where you will Let's take another example I'm going to press this plus icon Go to add another trigger and trigger manually I'll double click on that and then click the pencil icon up here And then I'm going to paste in some sample data I'll explain this in a second I'll press save and from there I'll click test workflow The result that we're seeing here is a task list So let's just try to map this to Google Sheets in the same way as we did above I'm going to rightclick duplicate that and then link those together Now I'll click into this Google Sheets node here Now we see on the left hand side that the data is in a very different format than what we had previously Previously N8N had broken out our data into separate items that it was individually processing behind the scenes Whereas I've just mocked up a very common scenario where we have an array of items and we can't just map those directly into Google Sheets And if you try you see that we're mapping an individual array If I try and press test step here responded with successful However when we go into that Google sheet we see it's just dumped all of the JSON for that entire array into this row So that's a super common scenario What you can do here is add in a split out node And from there drag in the overall array list that you want to separate out into separate items So I'll just drag this list in here and leave all of the other fields blank Then when I press test step you see we've now transformed on the left hand side this array over to the right hand side which are separate items And now we have that in the right format which is in the same format as we had earlier on From there we can now map this to our Google sheet So I'll go to the Google sheet instead of selecting this task title I'll then map the title in from this split out node up here Then I'll press save Press test workflow again And now we see it split that out into three separate items and it's updated the rows within Google Sheets And there we go We have all of the results there A really important thing to note here is that NAN processes all of the items in that specific node before it moves on to the next one This works differently to make.com which goes through the entire rest of the flow for each separate item You may be surprised that we were able to do all of this without even touching this loop over items node But there are cases where you will want to use this node Before we add this loop over items node I'm going to expand this workflow slightly which will just make things a little bit easier to understand once we add in that node After instead of just sending this data to Google Sheets let's say that we want to also add in AI into the mix So we'll add in a basic LLM chain And from there we're going to define a simple prompt So the prompt here is categorize this task Respond with either video blog post or email This is a kind of a contrived simple example but I am trying to keep things as simple as possible to explain these concepts So from there I'll just say here is the task title Then I'm going to drag in the title from this split out node Okay that should be okay I'll keep the default models as they are there And from there it's calling all of those separately one by one So we see it responded with three separate items It made three separate calls to entropic and it did all of this before this part of the flow actually lit up and turned green Let's link these parts of the flow together And I'm going to drag in the result from the LLM chain into this Google sheet Actually what I'll do is I'll drag it down here to the category And for the task title we want to drag in the title from the split out node earlier on Press save Okay we have a blank sheet here I'm going to click test workflow again I probably could have used a cheaper faster model for this but I was just going with the default Okay Yeah it's done all that And when it had all of these items ready here it uploaded all of these in one go Oh sorry for category JSON text I realized that I added this in as fixed whereas I should have selected expression because it just sent this JSON.ext category So let's try that again I'll press test workflow again So again it's going to go through all of those items before it proceeds with uploading those rows Okay there we go It's uploaded all the rows In this case this works completely fine You do not necessarily need to use a loop over items node There are some definite use cases where you're going to want to tackle each row at a time A really common use for the split over items node is for rate limiting If you have a ton of items here you could start really bombarding these APIs and you might get back errors Whereas if I select all these and move them over here I'll add in the split or this loop over items node I'm going to select a batch size of one So it's only going to take one item at a time then it's going to loop through each of those So I'm going to press this garbage icon here and press that and drag the loop part from here into the LLM chain I'm going to connect up this Google sheet back to here I'm going to delete this replace me node So now we have this lm chain linking to Google Sheets back to loop over items Before we add anything else to this let's just execute this and see how it works in practice and how it compares to the default looping that happens behind the scenes if we do not use one of these nodes So I'll click loop over items Then I'm going to click test step so we at least get the sample data that we're working off So once we've done that I can click this basic lm chain and this use this JSON.title So that has mapped correctly So that's all good Then I'm going to go to Google Sheets 2 This was linked from the split out item which is not what we're going for So we could execute previous nodes but I'm just going to type loop over items and then that should automatically map to the correct value So to this title from the first item over here So I'll click save Now I'm going to delete the data in there And I'm going to click test workflow So as we've done this we see that it has looped through the first item Then it's updated the Google sheet Then going back over here it's hit answer cloud and updated the Google sheet So it did all of those one by one and then it looped back to the very end And then when it was done it went through this flow In many cases you can just leave this as it is and connected to nothing But if you wanted to connect it to something you go to set edit fields and let's just have a look at the data that we're getting from that Within that we've received all of the items that we had uploaded to Google Sheets And then we could further process those if we want But within this looping one key thing that we can add in is a wait node So I click the weight node and we could wait 1 second for example before it proceeds with the next loop So let's delete that again We'll have a look at how that works in practice with test workflow It's going through the LM chain It's hit the Google Sheets but then after that it's going to update that row and then wait 1 second before going back again It's going to Google Sheets again It's going to wait and there we go We're nearly done Then after it it has gone through this done part of the flow as well So in this case it was not explicitly required to actually use this loop over items node but it is very handy if you want to add in wait time in between processing or if you want to handle items one by one and by the way it's not necessarily just one item at a time You can set up a batch size here So let's say you are scraping external data and you only want to scrape 10 URLs at a time for example You can set this as 10 It could quickly call an external API quickly 10 times in succession to get the results back You can then hit a wait node here to wait a few seconds before going back over loop over items to get the next batch That's a key reason why this loop over items node is also called split in batches node But there are some cases where an NAN node actually is not able to handle multiple items coming into it And that's where a loop over items node might be required beforehand If so you can just use this pattern where you have the loop you have the flow going back to the start of the loop over items then when it's done it goes through this done flow If you think there's a potential for errors to happen throughout your looping what you can do is go into it and go to settings and then the on error you can select continue or continue using error output If we click this continue using error output then we now have a new flow at the end and then you could add in a separate part of your workflow for that and then you could handle that error whatever way you want Another important setting is that if there's a potential that your loop will respond with no data then it could stop your workflow So select this always output data field here and this is used to prevent the workflow finishing on this node If this setting is unchecked then that could easily be a culprit for your workflow stopping prematurely There's also a very nice template on the NAN website where it explains the behavior of some different types of scenarios when attempting to loop In this case we just need to add in this array of URLs So I'm going to click this starting trigger then click the pencil icon up here and then paste in these URLs So this is just a URL and we have a list of URLs within an array That's the same type of data structure to this previous task list that we were working on Now let's go back to this scenario I'll press test workflow and then we'll go through each of these scenarios In the first flow let's have a look We have this code node If I click into it what's happening here is we have one item with an array of URLs If I try to add in an extra parameter to that item what happens is we have a list of URLs but then at the very end of that it just chucks in this other parameter Whereas you might be expecting that it would add in a parameter for each separate URL In the next flow we're using this loop over items node So let's go into that And again as you see here we're only dealing with one item So when we send one item into the loop over items again we're only dealing with one item And even if we keep batch size as one again it does not really make much of a difference We're only dealing with one item cuz we have not previously split it So when we go into this within the looping section of this loop over items click into it If we try to add in a parameter to this JSON again it's added the parameter to the end So after this URLs list again perhaps not what somebody would expect Whereas when we go down through this part of the flow we're finally using a split node We're dragging in this URL and then the destination field is URL So now we have five individual items And within the flow we're now branching that out to three separate flows So each of these flows will process each of those items So let's go into the first one We have five items coming out of this split out When we go through the loop over items five items individually went through this They're adding in this weight timer just for show Then after that it's going into this code node But instead of adding this parameter to the list as we had previously now we have URL and parameter now within its own individual item Then it's gone through this five times Then when it's amalgamated that together at the end of this we look at the result and now we have five separate items with this parameter added to each separate item But if you were following the earlier part of the video you might have thought do we even need a loop over items node for this and the answer is no You could just pass that straight into the code node So we have five items going into the code node Pretty important we select run once for each item When we're doing this we're adding this parameter And on the right hand side you see we have five separate items And again we have the same result as we had previously So again the loop over items is kind of redundant here and the result of each of these should be exactly the same On the right hand side we have URL and pram Then for this one again we have URL Finally we have a case here where we have all of these five items going down to this code node But afterwards it's amalgamated all of those into one item And you might be wondering how it's done that So within this node you can select run once for all items or run once for each item And in that case in the case of this node it just responded with the first item So it only responded with this Reddit URL and added the parameter to that And the result of that is as you see over here So if you were dealing with a node that's only going to take the first item then in that case you actually would need to use a loop over items node and pass it into that But in the case of this code node it can be run in either of these modes Either to take all the items or to only use one item So there's a lot of concepts around looping and data items here I'd highly recommend that you open up some Nitn scenarios and just start playing around with these just so you can kind of learn by doing because you can easily get tripped up by some of these and once you understand these concepts you can be a lot more efficient in building out your automations If you want to get way ahead then check out the link in the description to our community where you'll get access to all of our automation templates and our courses You can join hundreds of fellow automators in our active discussion boards and via our live workshops We'd love to see you there Thanks for watching