n8n Data Flow and Looping

Overview

This lecture explains how data flows and looping work in n8n workflows, especially when dealing with multiple data items and integrating with nodes like Google Sheets and APIs.

Data Flow in n8n

• n8n processes multiple items (e.g., rows or API results) as individual items behind the scenes.
• Simple data extraction (like reading a Google Sheet) automatically splits rows into items that are processed separately.
• Mapping fields from nodes is straightforward if data is already split into individual items.

Handling Arrays and Splitting Data

• If data comes as an array within a single item, it cannot be mapped directly to destinations like Google Sheets.
• Use a "Split Out" node to convert an array into separate items n8n can process individually.
• After splitting, mapping each field in nodes like Google Sheets works as expected.

Looping and Batch Processing

• By default, n8n processes all items for a node before moving to the next node.
• The "Loop Over Items" node is used when you need to process items one-by-one or in batches (useful for rate limiting).
• You can set a batch size in "Loop Over Items" to process a specific number of items at once.
• Adding a "Wait" node inside the loop can delay between processing items or batches.

Advanced Looping Scenarios

• Some nodes (e.g., certain APIs) can't handle multiple items as input; use "Loop Over Items" to process each item individually.
• Always check the settings in loop nodes for options like "on error" (to continue on errors) and "always output data" (to prevent workflow stops).
• The execution mode in nodes (e.g., "Run Once for Each Item" vs. "Run Once for All Items") affects how inputs are merged and processed.

Common Pitfalls & Tips

• If a node receives an array instead of separate items, split it first to avoid unexpected results.
• Use "Loop Over Items" for fine-grained control, especially for sequential operations or error handling.
• If dealing with nodes that only process the first item, split your data or adjust node settings as needed.

Key Terms & Definitions

• n8n — Automation tool for creating workflows connecting apps and data.
• Node — A step or function within a workflow (e.g., Google Sheets, HTTP request).
• Item — An individual data unit processed by n8n.
• Split Out Node — Splits arrays into individual items for processing.
• Loop Over Items Node — Processes items one by one or in batches; supports rate limiting.
• Batch Size — Number of items processed together in a loop.
• Wait Node — Pauses workflow for a set time between items or batches.

Action Items / Next Steps

• Practice using "Split Out" and "Loop Over Items" nodes with sample data.
• Experiment with different batch sizes and wait times in your own n8n workflows.
• Review the settings for error handling and output in loop nodes to avoid workflow issues.