ELC
Sign in or Join the community to continue

From evals to active observability: The continuous improvement loop for AI agents

Posted Oct 29, 2026 | Views 9
Share

Speaker

user's Avatar
Ameya Bhatawdekar
VP & Field CTO @ Braintrust

Ameya serves as VP, Field CTO at Braintrust, the Agent observability platform for shipping quality agents. He has spent his career using AI to help people be more effective and stay safe, and has driven multiple 0→1 initiatives to internet-scale, shipping intelligent systems for Dropbox Dash, Microsoft Office, Facebook, Windows, and major web browsers. At Dropbox, he led ML Engineering for universal AI search in Dash (search, discovery, recommendations, agentic conversational AI). Before that, he led Microsoft's Office AI team shipping AI features across Word, Excel, and PowerPoint, and created Office Copilot.

+ View details

SUMMARY

Evals are key to defining quality and preventing regressions, but they only test what teams already know to look for. Once agents reach production, new behaviors and failure modes emerge across more traces than any team can review manually.

This talk introduces active observability: a continuous quality loop that connects production traces, pattern discovery, debugging, and evals. Attendees will learn how they can help their teams turn production evidence into prioritized improvements, validate changes, and measure whether agent quality improves over time.

+ View details

CONTENT & TRANSCRIPT


+ View details
Sign in or Join the community
Exclusive to engineering leaders
ELC
Create an account
Exclusive to engineering leaders
Comments (0)
Popular
avatar


Watch More

Making the Transition from Large to Smaller Companies
Posted Jan 27, 2023 | Views 673
# Career
# ELC Annual 2022
# Leadership
# Management
Early company building: how to build recruiting momentum from the start
Posted Sep 21, 2020 | Views 1.1K
# Founders
# Distributed Teams
# Management
# Org Design
# Recruiting
# Retention
# Team Building
# Strategy