Lecture Notes on Long-Term Memory with Generative AI

Introduction

• Discussed the importance of note-taking for memory retention.
• Transitioned into the topic of long-term memory in generative AI, particularly focusing on large language models (LLMs).
• Key point: LLMs do not inherently have memory or state.

Short-Term Memory vs. Long-Term Memory

• Short-Term Memory in Chatbots:

  • When using a chatbot, it can remember things during a single session because it saves conversational history.
  • Example: If a user provides information about themselves, the chatbot can recall it within the same session.

• Long-Term Memory Issues:

  • If the chat is reset (simulating a new day), the chatbot cannot recall previous information without the provided history.
  • Proposed solution: Summarize and store conversational history at the end of the session using another LLM.

Implementing Long-Term Memory with Amazon Bedrock

• Transition to how to create an agent in Amazon Bedrock with long-term memory capabilities.

• Steps to Create an Agent:

  1. Go to the agents section in the Amazon Bedrock console.
  2. Click on "Create Agent" and provide a name (e.g., "My Demo Agent").
  3. Choose a model (e.g., Sonet or Hau).
  4. Enable memory with the option to set its duration (default is 30 days).
  5. Provide instructions for chatbot behavior.

• Testing the Agent:

  • After creating the agent, simulate a conversation to test memory functionality.
  • Example interaction: User states preferences (e.g., enjoys tea and beach walks).
  • End the session, and the agent summarizes the conversation for future reference.

Code Implementation in Jupyter Notebook

• Overview of a Jupyter notebook provided to demonstrate implementation:
  • Libraries: Uses Boto3 (AWS SDK for Python).
  • Agent Creation Code:
    • Create agent, define roles, and manage permissions through policies.
    • Enable memory configuration in the agent creation code.
  • Agent Interaction:
    • Invoke the agent using the defined session ID and memory ID.
    • Receive and handle responses, including memory traces.

Summary of Features

• Memory allows the agent to remember user preferences across multiple sessions.
• The agent can summarize previous conversations and use that information in future interactions.
• Can handle user requests more intelligently based on prior sessions.

Conclusion

• Amazon Bedrock's agent feature provides a way to implement long-term memory in chatbots.
• Importance of feedback on this service as it is in public preview.
• Encouragement to engage with the content and subscribe for more information on generative AI.