AI is changing how work gets done across engineering teams, but it’s also making performance harder to interpret. More activity, faster cycles, and new workflows don’t always translate into clear insights for leaders. In this session, we’ll break down how to build a more accurate view of engineering performance in the age of AI, focusing on how to connect day-to-day development work to delivery health, team effectiveness, and broader business goals. Key Discussion Topics: - AI increases activity but not necessarily clarity - Performance visibility must evolve with AI workflows - Engineering metrics should tie back to business outcomes For more data on how AI is changing engineering measurement, check out Harness’s State of Engineering Excellence report: https://www.harness.io/state-of-engineering-excellence To learn more about Harness AI DLC Insights check out this blog: https://www.harness.io/blog/introducing-ai-dlc-insights-to-prove-the-roi-of-your-ai-engineering-investment To see it in action: https://www.harness.io/demo/ai-dlc-insights

In this episode, Geddes Munson (SVP of Engineering @ Affirm) joins us to discuss operational / engineering excellence, scaling, and AI-native transformation! We explore Affirm’s approach to operational and engineering excellence and how a 2024 outage became a turning point in refining that focus. We deconstruct “AI retooling week”, the internal tools it inspired (including an incident tracing system), how the AI-native transition is impacting operational / engineering excellence, and how to connect these projects to business goals. Plus, we take a look at their early work building in agentic commerce, infrastructure decisions they made years ago setting them up for success now, how they’re thinking about designing for agent-first experiences.

Andrew McNamara, Director of Applied Machine Learning @ Shopify, joins the ELC podcast to share insights on building agentic platforms at scale, like Sidekick, that must keep reliability for its users at the forefront. Andrew describes the building philosophy behind Shopify and what it means to cultivate a culture of prototype-first while prioritizing hiring early-stage talent. We cover Sidekick’s development journey and how user feedback impacted its product vision, why evaluation is so important for determining ground truth sets, and the benefit of user-driven use cases. Andrew also dissects how they went about making product design decisions, such as building proactive agents and identifying subagent specializations. 

Most engineering teams have adopted AI coding tools. The productivity gains are real. Research across 2,172 developer-weeks shows roughly 25% year-over-year improvement for regular AI users, along with meaningful gains in test coverage and review efficiency. But adoption was the easy part. As AI becomes embedded in daily workflows, new questions are surfacing: code churn is climbing faster than output, duplication is expanding, and the metrics most teams rely on weren't designed to capture what's changing underneath. This session digs into what the data actually shows about AI-assisted development, where the gains are durable, where they're fragile, and what engineering leaders should be paying attention to as AI goes from experiment to everyday. Discussion based on findings from the AI Multiplier Effect report: https://www.gitkraken.com/reports/ai-multiplier-effect

Inbal Shani (CPO and Head of R&D @ Twilio) deconstructs the transformation of the R&D org at Twilio! We explore the shift from a GM-led model to a unified platform strategy and “why structure must always follow strategy.” Inbal shares her framework for moving from output-focused metrics to input goals, prioritizing “time-to-value,” and the nuances of measuring AI products. We discuss using "R&D roadshows" as a strategic company transformation tool and why engineering leaders must master product positioning. We also dive into mental models for future-proofing your business, from "working backwards" to solve customer problems, to embedding systems thinking into the DNA of your engineering team, and critical questions to identify and optimize decisions around your company’s moat. 

Rajat Monga, CVP AI Frameworks @ Microsoft, joins the podcast to discuss his leadership and founder journey, from Google Brain / Tensorflow to [inference.io](http://inference.io/) and back to Microsoft. He dissects what it means to refound vs. start from scratch, the value of the open source community, and strategies for discovering what problem to solve when going the startup route. We also cover how to determine your users’ hidden incentives and what that means for both product development & marketing, along with navigating the balance between a product’s usefulness and consumers’ willingness to pay for it. Additionally, Rajat shares about what he’s currently up to at Microsoft and the emerging ML / AI technologies he’s most excited about.