Learn how leading companies like Dropbox, NVIDIA, and Slack tackle LLM security in production. This comprehensive guide covers practical strategies for preventing prompt injection, securing RAG systems, and implementing multi-layered defenses, based on real-world case studies from the LLMOps database. Discover battle-tested approaches to input validation, data privacy, and monitoring for building secure AI applications.
Please read the full blog post here and the associated LLMOps database entries here.
In this episode, we dive deep into the world of LLM optimization and cost management - a critical challenge facing AI teams today. Join us as we explore real-world strategies from companies like Dropbox, Meta, and Replit who are pushing the boundaries of what's possible with large language models. From clever model selection techniques and knowledge distillation to advanced inference optimization and cost-saving strategies, we'll unpack the tools and approaches that are helping organizations squeeze maximum value from their LLM deployments. Whether you're dealing with runaway API costs, struggling with inference latency, or looking to optimize your model infrastructure, this episode provides practical insights that you can apply to your own AI initiatives. Perfect for ML engineers, technical leads, and anyone responsible for maintaining LLM systems in production.
Please read the full blog post here and the associated LLMOps database entries here.
A comprehensive exploration of real-world lessons in LLM evaluation and quality assurance, examining how industry leaders tackle the challenges of assessing language models in production.
Through diverse case studies, we cover the transition from traditional ML evaluation, establishing clear metrics, combining automated and human evaluation strategies, and implementing continuous improvement cycles to ensure reliable LLM applications at scale.
Please read the full blog post here and the associated LLMOps database entries here.
Prompt engineering is the art and science of crafting instructions that unlock the potential of large language models (LLMs). It's a critical skill for anyone working with LLMs, whether you're building cutting-edge applications or conducting fundamental research. But what does effective prompt engineering look like in practice, and how can we systematically improve our prompts over time?
To answer these questions, we've distilled key insights and techniques from a collection of LLMOps case studies spanning diverse industries and applications. From designing robust prompts to iterative refinement, optimization strategies to management infrastructure, these battle-tested lessons provide a roadmap for prompt engineering mastery.
Please read the full blog post here and the associated LLMOps database entries here.
An in-depth exploration of LLM agents in production environments, covering key architectures, practical challenges, and best practices. Drawing from real-world case studies, this article examines the current state of AI agent deployment, infrastructure requirements, and critical considerations for organizations looking to implement these systems safely and effectively.
Please read the full blog post here and the associated LLMOps database entries here.
Discover how embeddings power modern search and recommendation systems with LLMs, using case studies from the LLMOps Database. From RAG systems to personalized recommendations, learn key strategies and best practices for building intelligent applications that truly understand user intent and deliver relevant results.
Please read the full blog post here and the associated LLMOps database entries here.
Explore real-world applications of Retrieval Augmented Generation (RAG) through case studies from leading companies. Learn how RAG enhances LLM applications with external knowledge sources, examining implementation strategies, challenges, and best practices for building more accurate and informed AI systems.
Please read the full blog post [here](www.zenml.io/blog/building-llm-applications-that-know-what-theyre-talking-about) and the associated LLMOps database entries here.
The LLMOps Database offers a curated collection of 300+ real-world generative AI implementations, providing technical teams with practical insights into successful LLM deployments. This searchable resource includes detailed case studies, architectural decisions, and AI-generated summaries of technical presentations to help bridge the gap between demos and production systems.
Please read the full blog post here and the associated database entries here.
This week I spoke with Daniel van Strien, a digital curator working at the British Library. Daniel has worked on a number of projects at the intersection of archives, libraries and machine learning and I was really happy to have the chance to get to unpack some of the ways he's finding to apply these techniques and tools.
In particular, I found it interesting how important the annotation process is as part of many overall workflows, as well as how simple out-of-the-box techniques like image classification using a fine-tuned model could satisfy many low-hanging fruit-type use cases.
Special Guest: Daniel van Strien.
This week I spoke with Lak Lakhshmanan, who worked for years at Google on ML and AI projects and products at a senior level and he also brings years of experience working on meteorology and other scientific projects previously.
Lak brings a ton of experience to the table and it was interesting to hear his suggestions around when it is and isn't appropriate to bring the full set of MLOps tools to the table, for example. We also discussed the fundamentals of doing ML-backed projects as well as the teams needed to make those projects succeed.
Special Guest: Lak Lakshmanan.
Links:
This week I spoke with Charles Frye. Not only has Charles volunteered to be a judge on our Month of MLOps competition happening right now, he's part of the core team working on the Full Stack Deep Learning course.
Naturally, we get into education for practitioners as well as the things that Charles has seen in his own prior background working on production use cases. We also discuss the ways that tooling to support education as well as productive machine learning can and is being improved.
Special Guest: Charles Frye.
Links:
In today's conversation, I'm speaking with Goku Mohandas, founder and creator of the amazing online resource MadeWithML. Goku has a bunch of practical experience, from working with Apple to a startup in the oncology space and much more.
In this conversation we continued to unpack the theme of education in ML, the challenges when it comes to working across the full stack of ML applications, and what he's seen work in his experience working on MadeWithML.
We also discuss some of the patterns he's seen in the production stacks he's seen in his experience consulting with various ML teams as well as where he sees room for improvement in the abstractions that we all rely on to do our work.
Goku has generously agreed to be an external judge for our Month of MLOps competition that starts on October 10. If you haven't signed up yet, or want to learn more, please visit zenml.io/competition.
Special Guest: Goku Mohandas.
Links:
So excited to be able to announce our 🔥 AMAZING 🔥 external judges for the ZenML Month of MLOps competition! We have a stellar panel of ✨ ML and MLOps heroes ✨ to help select the best pipelines from all of your submissions!
💥 Charles Frye, core instructor at the amazing Full Stack Deep Learning course
💥 Anthony Goldbloom, co-founder and former CEO of Kaggle
💥 Chip Huyen, author of 'Designing Machine Learning Systems' and co-founder of Claypot AI
💥 Goku Mohandas, founder of MadeWithML, another essential course in production ML
We're honoured to have them on board for the ride, and we can't wait to see all the amazing ML use cases and problems our competitors solve along the way!
To learn more about the competition and to sign up, visit https://zenml.io/competition
Links:
This week I spoke with Eric Landau, co-founder of Encord, a platform for data-centric computer vision. This podcast contains a lot of geekery about annotation, and even though Encord aren't an annotation tool per se, Eric and his team have tackled a bunch of quite complicated problems relating to that domain.
We also discuss the much-used term 'data-centric AI' and consider where it's useful and where perhaps there's a little bit of hype. We also get into some of the technical tradeoffs and decisions that come when building a platform. I'm really excited to get to present this episode to you today as I really enjoyed the discussion.
Special Guest: Eric Landau.
Links:
This week we dive into the abstractions that we're all trying to layer on top of the core ML processes and workflows. I spoke with Phil Howes, co-founder and chief scientist at BaseTen. BaseTen is a platform that allows data scientists to go from an initial model to an MVP web app quickly.
We got into some of the big challenges he had working to build out the platform, as well as the core issue of iteration speed that motivates why they're building BaseTen.
Phil has experienced quite a few of the industry's end-to-end patterns in the years that he's been working on machine learning and it was great to have that context inform the conversation, too.
Special Guest: Phil Howes.
Links:
This week I spoke with Savin Goyal and Hugo Bowne-Anderson from Outerbounds. They both work on leading, building and helping people put models into production through Metaflow, and I'm sure current users of ZenML will find this conversation interesting to hear how they think through the broader questions and engineering problems involved with MLOps.
Above all, we spoke about the challenges involved in building a tool that handles the whole machine learning story, from collecting data to training models, to deployment and back again. In many ways it's great that there are lots of smart people thinking about this really hard problem, and even though it is by no means 'solved' conversations like this make me feel cautiously optimistic about the space.
Special Guests: Hugo Bowne-Anderson and Savin Goyal.
Links:
This week I spoke with Mateo Rojas-Carulla, the CTO and a co-founder of Lakera and Matthias Kraft, also a co-founder and the CPO there. Lakera is an AI safety company that does a lot of work in the computer vision domain, building a platform and tools for users to gain more confidence in the output and functionality of their models.
We discuss how they think about the testing of machine learning models, and about how having this safety element upfront has implications for how you go about the testing and ensuring robustness. We specifically dive into how to go about testing computer vision models and the various pitfalls that are to be found in that domain.
Special Guests: Mateo Rojas-Carulla and Matthias Kraft.
This week I spoke with Robin Cole, a senior data scientist at Satellite Vu, a company that's about to launch a thermal imaging satellite into space in order to provide new ways of seeing the earth from above.
Robin generously took the time to discuss his day to day work involving satellite data, the stack they work with at Satellite Vu as well as some of the difficulties that come up in the domain. We also discuss the extremely popular satellite-image-deep-learning GitHub repo that presents resources for those working with or seeking to learn about this kind of data.
Special Guest: Robin Cole.
Links:
This week on the podcast I spoke with Gerard Kruisheer, the CTO and co-founder of Captain AI, a company based in the Netherlands working on autonomous shipping out of the busy Rotterdam port.
We discussed the unique problems that come with building autonomous vehicles, the extent to which the latest and greatest research informs their work, their production stack and how they handle deployment for their particular setup.
As always please let us know if you have guests you'd like me to speak to by sending a message to us on slack or by emailing [podcast@zenml.io](podcast@zenml.io).
Special Guest: Gerard Kruisheer.
Links:
I'll be having some conversations with the people behind the tools that ZenML offers as integrations. We spoke with Ben Wilson a few weeks back, and today I'm pleased to publish this conversation with Emeli Dral, co-founder and CTO of Evidently, an open-source tool tackling the problem of monitoring of models and data for machine learning.
We discussed the challenges around building a tool that is both straightforward to use while also customisable and powerful. We also got into the thinking behind how they grew their community and blog along the way.
Special Guest: Emeli Dral.
Links:
