Building Effective AI Agents

Introduction

• Anthropic has been working with various teams to build large language model (LLM) agents across industries.
• Success in implementing LLM agents comes from using simple, composable patterns, rather than complex frameworks.
• This guidance is based on experiences from working with customers and building agents at Anthropic.

Definition of Agents

• Agentic systems: Systems where LLMs dynamically control their processes and tool usage.
  • Workflows: Predefined code paths for orchestrating LLMs and tools.
  • Agents: Systems where LLMs direct their own processes and are flexible in task execution.
• Use agents when flexibility and model-driven decision-making are needed, but simple LLM calls suffice for many applications.

When and How to Use Frameworks

• Frameworks like LangGraph, Amazon Bedrock's AI Agent framework, Rivet, and Vellum can simplify agentic system implementations.
• Frameworks create abstraction layers that might obscure prompts and responses; direct use of LLM APIs is recommended for simplicity.

Building Blocks for Agentic Systems

Augmented LLM

• Enhanced with retrieval, tools, and memory capabilities.
• Focus on tailoring capabilities to specific use cases and providing a well-documented interface.

Workflow Patterns

Prompt Chaining

• Decomposes tasks into steps where each LLM call processes the output of the previous one.
• Ideal for tasks that can be broken into fixed subtasks, trading latency for accuracy.

Routing

• Classifies input and directs it to specialized follow-up tasks.
• Suitable for complex tasks with distinct categories.

Parallelization

• Consists of Sectioning (breaking tasks into subtasks) and Voting (diverse outputs from multiple attempts).
• Effective for tasks that can be parallelized or need multiple perspectives.

Orchestrator-Workers

• Central LLM breaks tasks into subtasks and synthesizes results.
• Useful for tasks where subtasks aren't predefined.

Evaluator-Optimizer

• One LLM call generates a response, another evaluates and provides feedback.
• Effective for tasks with clear evaluation criteria and iterative refinement.

Agents

• Autonomous and can handle sophisticated tasks through reasoning, planning, and tool usage.
• Begin with a command or discussion and operate with ground truth feedback.
• Ideal for open-ended problems without a fixed path.

Using Agents in Practice

• Customer Support: Combines chatbot interfaces with tool integration for handling user queries.
• Coding Agents: Evolve from code completion to autonomous problem-solving, using tests and human reviews.

Engineering Tools for Agents

• Tools are crucial for agentic systems, enabling interaction with external services.
• Good tool definitions should be clear and well-documented, akin to good human-computer interfaces.

Summary

• Build the right system for your needs, starting simple and optimizing as needed.
• Maintain simplicity, transparency, and a well-crafted agent-computer interface.
• Use frameworks to start but simplify as you move to production.

Acknowledgments: Written by Erik Schluntz and Barry Zhang, based on experiences at Anthropic.