hello how you doing in this video we're going to talk about pine cone metadata filters have you heard the term pine cone metadata filters and you're not quite sure you know exactly what this is well if so then watch along with me for the next few minutes and I will quickly get you up to speed okay let's get started first what exactly are metadata filters in Pine Cone well it turns out that when you upsert vectors into pine cone you can attach attach metadata to each Vector metadata is stored in key value pairs where the key is a string and the value can be a string number or Boolean pine cone does not enforce uniform inclusion of the same metadata keys on every Vector in a pine cone index but in most use cases you'll probably want to be consistent when performing a query you can apply filters to limit the similarity search to the vectors that meet the metadata conditions this allows you to fine-tune the results ensuring that the return vectors not only match the query Vector in terms of similarity but also conform to specific metadata filter conditions so how does metadata filters affect query performance in most cases pinec cone queries involving metadata filters are more performant than queries that do not include meta metadata filters before we go into the metadata filter examples in Pine Cone first you need to get a pine cone account and second you need an API key in an earlier pine cone video on Pine Cone indexes I cover what you need to do for both of these steps if you want to see this in more detail feel free to go back and watch this video but for now I'll keep moving along in this video I'm going to show you examples in Python you'll first need to down download the pine cone client Library you can do this using pip install quick note I'm using the pine cone standard python SDK versus the pine cone grpc python SDK I will do a video on the differences between these two sdks the next few weeks but for this video please know that I'm using the pine cone standard SDK the python code can be executed using jupyter notebooks or any pyth on code IDE such as Visual Studio code my examples assume you already have a pine cone index created and ready to go if you don't have this you can check out that same previous video on Pine Cone indexes I referenced a minute ago to get details on how to create a pine cone index I also included a python script in the GitHub repo for this video which you can use to create a pine cone index once we have our pine cone index we can create a python index object reference which we will use to perform an upsert of five vectors each with metadata into our index the vectors will contain marine weather information and I have attached a metadata attribute buoy station along with a buoy station value in my example after we upsert we sleep for 5 seconds this is because pine cone is a eventually consistent database and it takes a few seconds for vectors to become visible on fetches and queries after you perform an upster operation once 5 Seconds has elapsed I read back the vectors A B C D and E using a fetch operation the fetch call returns results back and I print them out to the console here are the results printed out to the console you can see vectors A B C D and E are all in the pine cone index you will also notice that each of them has a buoy station metadata attribute so how exactly do we query vectors using metadata filters from a pine cone index well it turns out this is pretty easy here's a python script that performs a query operation using metadata filters you will see the standard query parameters plus an additional attribute called filter this filter attribute specifies that we only want to search for vectors with the metadata attribute buoy station equal to 45609 the S letter res search will only consider vectors that have a metadata attribute of buoy station equal to 45609 now I've set top K to five to allow the query to return all vectors that match this metadata filter condition here are the results of the query only three vectors A B and C with a meta data attribute buy station equal to 45609 are returned okay this makes sense now let's run another query and this time we filter on the metadata buoy station attribute set to 45402 so I run this second query and here are the results of the second query this time only two vectors d& e with the metadata attribute of buoy station equal to 45542 are returned again this makes sense note you can have more than one metadata key value pair associated with a vector you can also build more complex and or filter Expressions that include multiple key value pairs in our example we use the equal comparison operator but there are additional filter comparison operators such as not equal greater than greater than or equals less than less than or equals in not in and exist that are available for your use okay so now you should have a basic understanding of metadata filters in pine cone in upcoming videos I'll be covering pods versus serverless as well as pine cone sdks feel free to check these videos out to determine the best way you can leverage pine cone in your llm driven systems okay thanks for watching this video along with all my other videos in the vector databases playlists are listed in YouTube description I invite you to watch other videos on my channel if you like the way I'm sharing this content please consider subscribing when you subscribe this really helps my channel grow one last thing we all love technology and we're all excited about all the Innovation with a cloud machine learning AI but don't forget to carve out some time to live in the real world go outside go swimming go hiking go climbing go surfing but get out and move your body and if you do tell me in the comments I want to hear about it and with that have a great day thanks [Music]