Key Takeaways from Building AI Agents as a Service

This summary covers 26 key takeaways from the experience of building over 150 AI agents in the last 9 months under the "agents as a service" model.

Key Takeaway 1: AI Agents Are Not Employees

• Agents differ from automations; agents require specific training.
• Agents have less autonomy than employees and rely heavily on SOPs (Standard Operating Procedures).

Key Takeaway 2: Start from Well-Documented Processes

• Look for documented SOPs; they simplify agent training.

Key Takeaway 3: Business Owners Will Never Build Their Own Agents

• Business owners prefer expert developers.
• The demand for AI agent developers continues to grow.

Key Takeaway 4: Business Owners Have No Idea Which Agents They Need

• Consultation and customer journey mapping are crucial to identify valuable agents.

Key Takeaway 5: You Don’t Need 20+ Agents

• Start with a minimal number of agents to avoid complexity.

Key Takeaway 6: Data with Actions Deliver Results

• Combine data with relevant actions for better performance.

Key Takeaway 7: Prompt Engineering is an Art

• Writing effective prompts is critical and requires careful construction and iteration.

Key Takeaway 8: Integrations Are Important

• Agents should integrate with existing systems for convenience and effectiveness.

Key Takeaway 9: Agent Reliability Has Been Solved

• Use data validation libraries like Penic to ensure reliable agent performance.

Key Takeaway 10: Tools Are Vital

• Focus on building tools (actions) as they provide the most value.

Key Takeaway 11: Limit Tools Per Agent

• Keep the number of tools per agent between four to six to prevent confusion.

Key Takeaway 12: Model Costs Don’t Matter

• Focus on ROI rather than model costs.

Key Takeaway 13: Clients Don’t Care About the Model

• Clients prioritize value and privacy compliance over the specific model used.

Key Takeaway 14: Don’t Automate Until Value is Established

• Ensure a process is valuable before automating it.

Key Takeaway 15: Focus on ROI

• Use the ROI formula to determine the value of automating a process.

Key Takeaway 16: Agent Development is Iterative

• Experiment with different architectures and evaluate results iteratively.

Key Takeaway 17: Use Divide and Conquer Approach

• Break complex problems into manageable tasks and deliver incrementally.

Key Takeaway 18: Evals Matter for Big Companies

• Evaluations (evals) help large enterprises continuously improve their agents.

Key Takeaway 19: Two Types of Agents

• Agents can be individual or part of agentic workflows.

Key Takeaway 20: Agents Need to Adapt Based on Feedback

• Agents should be able to analyze and adapt based on the impact of their actions.

Key Takeaway 21: Don’t Build Around Limitations

• Design agents with the expectation of improving models.

Key Takeaway 22: Deploying Agents is Challenging

• Deployment and integration often take as long as building the agent.

Key Takeaway 23: Waterfall Projects Don’t Work

• Utilize agile, subscription-based models for agent development.

Key Takeaway 24: Include a Human in the Loop for Critical Agents

• Ensure human oversight for agents where errors could have severe consequences.

Key Takeaway 25: 2025 - Year of Vertical AI Agents

• Specialize agents for specific industries for greater scalability and value.

Key Takeaway 26: Agents Enhance Business, Don’t Replace People

• AI agents help businesses scale, not replace human jobs, fostering growth and efficiency.

These takeaways provide insights into the best practices and strategic approaches for developing and deploying AI agents, focusing on delivering value, ensuring reliability, and planning for future advancements.