This session, led by Scott Hanselman and Yina Arenas, focused on demonstrating Azure AI Foundry’s capabilities using the Hansel Minutes podcast production as an example.
Key topics included an overview of AI models, the new agentic platform, and observability tools for monitoring and managing AI applications.
Several live demos highlighted model selection, multi-agent workflows, efficient cost management, and end-to-end security and monitoring, including new features announced at Microsoft Build.
Attendees saw practical automation of podcast workflows, with an emphasis on human oversight, efficiency, and cost reduction.
Action Items
None explicitly assigned during this session.
Introduction and Overview of Azure AI Foundry
Azure AI Foundry is an open, flexible, and secure platform enabling developers to infuse AI into any application, whether new or existing.
Three core focus areas: AI models, an agentic platform, and observability tools.
The session used the Hansel Minutes podcast workflow as a practical example to showcase Foundry’s capabilities in automating complex, repetitive tasks.
Models: Catalog, Selection, Fine-Tuning, and Local Execution
Azure AI Foundry now offers access to over 10,000 models, including from OpenAI, DeepSea, Mistral, Meta, and others, with easy switching and unified access.
The new “Model Router” feature helps route queries to the most appropriate model, optimizing for cost and efficiency (up to 60% cost reduction vs. using single large models).
Fine-tuning and model distillation capabilities demonstrated—for example, reducing cost of transcribing and summarizing 1,000 podcast episodes from about $100 to $1.50 using a dedicated, smaller model.
"Foundry Local" allows running models on local machines, not just in the cloud, demonstrated by running a summarization task on a laptop GPU.
Demonstrated leaderboards for model benchmarking and the use of own datasets to evaluate model results.
Agents: Workflow Automation and Orchestration
Clarified definitions: Agents orchestrate tasks, using tools and sometimes multiple models, supporting memory, multi-modal input/output, and human-in-the-loop workflows.
Recommended building composable, single-purpose agents that can be orchestrated together for complex workflows.
Two workflows supported: connected agents (API-like tool invocation) and multi-agent workflows (for processes with human steps).
General Availability announced for Azure AI Foundry Agent Service, enabling declarative agent creation, enterprise integration, tool use, and interoperability with other agent platforms.
Showcased multi-stage podcast workflow automation: guest intake, bio generation, transcript creation, show notes drafting, link verification, and agentic search across 20 years of podcast content.
Emphasized grounding and restricting agent responses to the relevant corpus and domain, highlighting efficiency, security, and cost benefits.
Observability, Monitoring, and Security
Foundry provides tools for experimental tracking, production monitoring, debugging, and reliability—including integration with OpenTelemetry, Azure Monitor, Application Insights, and third-party tools like Grafana.
Agents gain unique enterprise identities (integration with Entra), supporting scoped permissions and detailed governance.
Security features include data labeling, data protection, and full integration with Microsoft Purview and Defender.
Evaluation metrics for agent performance (e.g., intent, relevance, task adherence) can be incorporated into CI/CD pipelines, with automated evaluation runs upon deployment.
Emphasized the importance of observability in distributed AI systems and maintaining human oversight in automated workflows.
Live demos covered the full podcast production flow: transcribing audio, generating show notes, verifying links, assembling summaries, and producing YouTube-ready transcript files.
Highlighted efficient use of fine-tuned and distilled models tailored for specific tasks (e.g., show notes generation, not general-purpose chat).
Showed error cases and transparent evaluation of model outputs, reinforcing the importance of continuous monitoring and improvement.
Announced upcoming support for markdown in Notepad as a productivity booster.
Use Cases, Customer Impact, and Future Opportunities
Over 70,000 customers are using Azure AI Foundry, with over 10,000 leveraging the agent service to create millions of agents.
Practical applications include guest sourcing automation, localization/translation, and social content generation.
Encouraged attendees to consider how to apply AI Foundry’s automation features to reduce “dull, dirty, or dangerous” work in their own domains.
Decisions
General Availability of the Azure AI Foundry Agent Service — Announced to enable enterprise-ready, declarative agent workflows.
Integration of Observability and Security — Committed to supporting industry standards (OpenTelemetry, Entra, Purview, Defender) for monitoring and governance.
Open Questions / Follow-Ups
None explicitly noted; session concluded with an invitation for attendees to explore further and a call to action to apply these tools.