Pine Cone Metadata Filters

Introduction

• Discussion on Pine Cone metadata filters.
• Aim: To understand what Pine Cone metadata filters are and how they function.

What are Metadata Filters?

• Vectors in Pine Cone: You can attach metadata to each vector.
  • Metadata stored in key-value pairs:
    • Key: string
    • Value: string, number, or Boolean
• Consistency in metadata keys across vectors is recommended.
• Query Filters:
  • Filters limit similarity search to vectors meeting metadata conditions.
  • Ensures results match both query vector similarity and specific metadata conditions.

Impact on Query Performance

• Metadata filters can enhance query performance in Pine Cone.

Getting Started with Pine Cone

1. Account Setup:
   • Create a Pine Cone account.
   • Obtain an API key.
2. Client Library:
   • Use pip install to download the Pine Cone client library.
   • Python SDK: Using Pine Cone standard Python SDK, not gRPC.
   • Code can be executed in Jupyter notebooks or IDEs like Visual Studio Code.

Working with Pine Cone

• Index Creation:
  • Assumes an existing Pine Cone index.
  • Python script available on GitHub for index creation.
• Upsert Vectors:
  • Upsert five vectors with metadata (e.g., buoy station).
  • Sleep for 5 seconds to account for eventual consistency.
• Fetch Vectors:
  • Fetch operation returns vectors A, B, C, D, E with metadata.

Querying with Metadata Filters

• Basic Query Example:
  • Python script to perform a query with metadata filters.
  • Example filter: buoy station = 45609.
  • Result: Vectors A, B, C returned.
• Second Query Example:
  • Filter: buoy station = 45402.
  • Result: Vectors D, E returned.
• Complex Filters:
  • Multiple key-value pairs can be used.
  • Comparison operators include:
    • Equal (=), Not equal (!=)
    • Greater than (>), Less than (<)
    • Greater than or equals (>=), Less than or equals (<=)
    • In, Not in, Exist

Conclusion

• Basic understanding of metadata filters in Pine Cone covered.
• Upcoming videos:
  • Pine Cone SDKs
  • Pods vs Serverless
• Encouragement to engage with content and subscribe for more.
• Reminder to balance technology interest with outdoor activities.