Creating Knowledge Graphs with LangChain and LLMs

Introduction

• Knowledge graphs creation is becoming easier with advancements in LLMs and LLM frameworks.
• LangChain introduced new functionality to create knowledge graphs from raw text.

Required Tools and Installations

• Install relevant LangChain modules: langchain, langchain_community, openai-support.
• Install Neo4j Desktop and Neo4j SDK client for graph databases.
• Main modules used:
  • llm_graph_transformer (new functionality by LangChain).
  • Neo4jGraph for loading knowledge graphs into the graph database.
  • OpenAI or any open-source LLM (such as LLaMA) using OlamaWrapper from LangChain.
• Configure OpenAI API key in environmental variables.

Setting Up LLM and Neo4j

• Define LLM settings:
  • Example: llm = gpt4, temperature = 0 (important for extracting factual information).
• Instantiate LLM and provide it to llm_graph_transformer.
• Neo4j local database credentials (use default settings).
• Ensure Neo4j database is created, active, and running.

Example: Knowledge Graph Creation from Text

• Text about Mary Curie (areas of work, awards, husband info).
• Steps:
  1. Create document using langchain.Document.
  2. Call llm_graph_transformer with the document.
  3. Node extraction:
     • Example nodes: Mary Curie (type: person), Polish (type: nationality), Chemist (type: occupation).
  4. Relationship extraction between nodes:
     • Example: Mary Curie (person) → nationality → Polish.
• Print nodes and relationships for verification:
  • Iterate over nodes and print details (ID, type).
  • Iterate over relationships and print details (source node, target node, relationship type).

Loading Data into Neo4j

• Neo4j client setup and data visualisation:
  • Create new project and make it active.
  • Use Neo4j browser to visualize data.
• Example visualization: Mary Curie as the central node with relationships showing awards, occupation, nationality, and spouse.

Customizing Node and Relationship Types

• Define specific node and relationship types:
  • Example node types: person, country, organization.
  • Example relationships: nationality, located_in, worked_at, spouse.
• Update graph transformer with new node and relationship type definitions.
• Reload and visualize filtered data in Neo4j.

Conclusion

• Creating knowledge graphs has become simpler with advanced frameworks and LLMs.
• Previous methods required extensive text processing; now it’s streamlined with tools like LangChain.