In this episode, Jordi Montes and Matt Wyman discuss the evolving landscape of AI agents, exploring their definitions, applications, and the challenges faced in building complex systems. Matt shares his journey from engineering to product management, highlighting the importance of understanding user behavior and the need for new monitoring tools tailored for AI systems. They delve into diverse use cases for AI agents, from parts catalogs to onboarding processes, and debate the merits of fine-tuning versus in-context learning. In this conversation, Matt Wyman and Jordi Montes discuss the evolving landscape of AI performance metrics, the importance of establishing baselines for continuous improvement, and the role of data scientists in engineering. They explore the shift towards multimodal AI solutions, the challenges of agent deployment and orchestration, and innovations in authentication for AI agents. The discussion also touches on the future of AI model evaluation and adaptation, emphasizing the need for organizations to embrace constant measurement and experimentation.