hello everyone welcome to a new video on N8 and workflows in this video we'll explore context-based chunking in ragge and how it can enhance retrieval accuracy let's dive in let's start by understanding what retrieval augmented generation is retrieval augmented generation is a technique that improves the accuracy and reliability of generative AI models by incorporating information from specific and relevant data sources in simple terms we provide an llm with a data source allowing it to retrieve and use relevant information before generating a response this makes the output more precise and contextually accurate here's a simple diagram illustrating the traditional retrieval augmented generation system now there's a challenge with the traditional retrieval augmented generation approach it often struggles to retrieve highly relevant data based on the current context this can lead to inaccurate or incomplete answers to improve retrieval accuracy we can use different chunking strategies such as recursive text splitting with overlap which helps retain context across chunks let's take a closer look but in this video we'll be implementing context-based chunking as described by anthropic contextual retrieval in context-based chunking each chunk is attached to a broader context from the entire document this helps maintain better coherence and improves retrieval accuracy ensuring the model understands the bigger picture when generating responses let's see how we can set this up in n8n our workflow starts by retrieving The Source document from Google Drive in the next step we extract text Data from the document now here's the interesting part we've added clear boundary lines in the document to Mark different sections these boundaries will help us split the text into meaningful chunks next we use a code node in n8n where we'll write a script to divide the document into sections or in other words create structured chunks this ensures that each chunk retains its context properly once the chunks are extracted from the document we Loop through each chunk using a loop node in n8n next we use an agent node which generates contextual information for each chunk by referencing the entire document this step ensures that every chunk maintains a strong connection to the overall context improving retrieval accuracy in this node we are using open ai's GPT 4.0 mini provided by open router as our llm model once the chunk context is generated we prend that context to the related chunk this enriched chunk is then sent to the next node where we create embeddings for storage in a vector database for our Vector store we are using pine cone and for textto Vector conversion we're leveraging Google's Gemini I text embedding oak4 model we've also added a recursive text splitter but since we've set a large chunk size it won't have much effect in this case that's it now we can connect our Vector store to a rag setup and see the Magic in action with context enriched chunks and optimized retrieval our system is now much more accurate and efficient in this video we explored how context-based chunking enhances retrieval accuracy in a RG setup we walked through an n8n workflow covering retrieving a document from Google Drive extracting text and structuring it into meaningful chunks using an agent node to generate context for each chunk creating embeddings with Google Gemini text embedding Z4 and storing them in pine cone with this setup our RG system is now more accurate and context aware you can try it out yourself the workflow link is in the description thanks for watching if you found this helpful don't forget to like share and subscribe for more n8n tutorials see you in the next one