Fin x Mistral AI: Evaluating your AI Agent

Is your AI agent actually working? It's a deceptively simple question—and one of the hardest to answer honestly in production. As AI agents move from prototype to core product, the gap between benchmark performance and real-world results is becoming impossible to ignore. The teams building at the frontier are finding that smaller, fine-tuned models often outperform bloated generalists—but only if you have the eval infrastructure to know what "better" actually looks like. Join Fin and Mistral for a senior practitioner-level conversation spanning both ends of the stack—from frontier model development to production AI agents—on what it actually takes to evaluate, iterate, and trust AI in the real world. What you'll learn: Why offline metrics alone will mislead you, and what a production-grade eval framework actually looks like When fine-tuned, specialised models outperform larger generalist ones — and what it takes to get there How teams building at the model layer and the product layer think about evaluation differently What building your own models teaches you about the limits of benchmarks Speakers: Pedro Tabacof, Principal Machine Learning Scientist, Fin Henry Lagarde, Software Engineer, Mistral AI