David Gildea has learned that traditional security models collapse when AI agents start delegating tasks to 50 or 60 other agents in enterprise workflows. As VP of Product for AI at Druva, he's building deterministic security harnesses that solve the authentication nightmare of multi-agent systems while maintaining the autonomous capabilities that make AI valuable. David explains why MCP specifications gained faster enterprise adoption than A2A despite having weaker security features, telling Ravin how his team is addressing authentication gaps through integration with existing identity management systems like Okta. He shares Druva's approach to wrapping AI agents in security frameworks that require human approval for high-risk actions while learning from user behavior to reduce approval friction over time. He also covers Druva's evolution from custom RAG systems to AWS Bedrock Knowledge Bases, demonstrating how to build knowing that components will be replaced by better solutions.  Topics discussed: Multi-agent workflow security challenges with 50+ agent delegation chains MCP specification adoption advantages over A2A for enterprise authentication Deterministic security harnesses wrapping non-deterministic AI agent behaviors Identity management complexity when agents impersonate human users in enterprise systems Human-in-the-loop scaling problems and supervisor agent solutions for authorization AI-first capability layers replacing traditional API structures for agent interactions Hyper-personalization learning from individual user behavior patterns over time Objective-based chat interfaces eliminating traditional software navigation complexity Building replaceable AI components while maintaining development velocity and learning Listen to more episodes:  Apple  Spotify  YouTube Website

The fastest path to production AI isn't perfect architecture, according to Alexander Page. It's customer validation. In his former role of Principal AI Architect at BigPanda, he transformed an LLM-based prototype into "Biggy," an AI system for critical incident management. BigPanda moved beyond basic semantic search to build agentic integrations with ServiceNow and Jira, creating AI that understands organizational context and learns from incident history while helping with the entire lifecycle from detection through post-incident documentation. Alexander also gives Ravin BigPanda's framework for measuring AI agent performance when traditional accuracy metrics fall short: combine user feedback with visibility into agent decision-making, allowing operators to drag-and-drop incorrect tool calls or sequence errors. He reveals how they encode this feedback into vector databases that influence future agent behavior, creating systems that genuinely improve over time.    Topics discussed: LLM accessibility compared to traditional ML development barriers Fortune 500 IT incident management across 10-30 monitoring tools Building Biggy, an AI agent for incident analysis and resolution Customer-driven development methodology with real data prototyping Agentic integrations with ServiceNow and Jira for organizational context Moving beyond semantic search to structured system queries AI agent performance evaluation when accuracy is subjective User feedback mechanisms for correcting agent tool calls and sequences Encoding corrections into vector databases for behavior improvement Sensory data requirements for human-level AI reasoning Listen to more episodes:  Apple  Spotify  YouTube Website

UCHealth’s healthcare AI methodology currently enables 1 nurse to monitor 14 fall-risk patients, with plans to scale to 140, then 1,400 through computer vision and predictive analytics. Instead of exhausting pilots, they deploy in phases: test, prove, optimize, then scale. This has created a system that prioritizes force multiplication of current staff rather than replacing them, enabling healthcare professionals to work at the top of their scope. Richard Zane, Chief Innovation Officer also tells Ravin how their computational linguistics system automatically categorizes thousands of chest X-ray incidental findings into risk levels and manages closed-loop follow-up communication, ensuring critical findings don't fall through administrative cracks. Richard's three-part evaluation framework for technology partners — subject matter expertise, technical deep dive, and financial viability — helps them avoid the startup graveyard.    Topics discussed: UCHealth's phase deployment methodology: test, prove, optimize, scale Force multiplication strategy enabling 1 nurse to monitor 14+ patients Computational linguistics for automating incidental findings Three-part startup evaluation: subject matter, technical, and financial assessment FDA regulatory challenges with learning algorithms in healthcare AI Problem-first approach versus solution-seeking in healthcare AI adoption Cultural alignment and operational cadence in multi-year technology partnerships Listen to more episodes:  Apple  Spotify  YouTube Website

The construction industry sits on goldmines of unstructured data that could revolutionize how buildings get built. From thousands of contracts to communication logs spanning 12-36 month building cycles, Dan Williamson, Director of Artificial Intelligence at Ryan Companies US, says all that data remains largely untapped. Dan walks Ravin through how Ryan is building AI systems to unlock this data, from contract risk analysis to robots doing reality capture on job sites. But the real challenge is organizational. Getting trade workers who've operated the same way for decades to embrace robotic assistants requires finding business evangelists willing to co-create change rather than having it imposed from above. Topics discussed: Building enterprise AI strategy in traditional construction and real estate industries Leveraging unstructured data from contracts, communications, and building drawings Finding business evangelists to co-create change rather than imposing technology top-down Deploying robots on job sites for reality capture and progress tracking Processing hundreds of thousands of leases and construction contracts with AI Transforming construction drawings from unstructured data into actionable insights Managing 60+ contracts per project across 100-250 annual construction projects Automating safety risk assessment through job site communication analysis Replacing manual data entry with AI-powered construction workflow applications

At Elastic, CISO Mandy Andress learned that pragmatic guardrails work better than blanket bans for managing AI adoption across their 3,500-person distributed workforce. Instead, she enables AI tools with smart controls rather than block them entirely. As both a customer and provider in the AI ecosystem, Elastic faces unique challenges in AI strategy. Mandy explains how they're applying hard-learned lessons from cloud vendor lock-in to build flexible AI systems that can switch foundation models with minimal engineering effort.  She also shares why AI ownership is naturally migrating from security teams to business units as organizations mature their understanding of the technology.   Topics discussed: Elastic's dual role as vector database provider and AI customer. Transitioning AI ownership from security teams to business units. Building foundation model flexibility to avoid vendor lock-in. Quantifying AI business value through time auditing versus traditional ROI. Managing enterprise AI tool procurement floods without innovation barriers. Pragmatic AI guardrails versus blanket AI-blocking strategies. AI team organizational structures based on technical maturity. Focusing AI governance on access controls and API fundamentals. Behavioral analytics for credential-based attack detection. Listen to more episodes:  Apple  Spotify  YouTube Website

At Yext, evaluating every AI tool through a security-first lens sparked comprehensive AI governance frameworks that protect enterprise data without stifling productivity. Rohit Parchuri, SVP & CISO, explains how they developed "AI fact sheets" for these evaluations, comparing each tool against specific business goals, data protection requirements, and existing capabilities. This process prevents tool duplication while ensuring security standards are met before deployment. But governance is just one piece of Yext's AI strategy. As a company born from AI technology, they've already built their own ML models to filter false positives from security tools, and they have direct experience with AI's data amplification risks — like how incorrect restaurant ingredient data could trigger FDA issues across all client listings. Rohit explores how enterprises can build sustainable AI programs that accelerate business outcomes while maintaining robust security controls.   Topics discussed: AI's intent recognition versus traditional RPA systems. Implementing "AI fact sheets" for vendor evaluation. Building security checkpoints. Balancing employee productivity with data protection. Managing free consumer AI tools like ChatGPT. Developing AI acceptable use policies. Replacing tier-1 analysts with AI systems. Creating feedback loops for vulnerability categories. Evaluating AI vendor security frameworks. Predicting AI replacement timelines for security roles. Listen to more episodes:  Apple  Spotify  YouTube Website

What happens when you build AI agents trusted enough to handle production incidents while engineers sleep? At Datadog, it sparked a fundamental rethink of how enterprise AI systems earn developer trust in critical infrastructure environments. Diamond Bishop, Director of Eng/AI, outlines for Ravin how their Bits AI initiative evolved from basic log analysis to sophisticated incident response agents. By focusing first on root cause identification rather than full automation, they're delivering immediate value while building the confidence needed for deeper integration. But that's just one part of Datadog's systematic approach. From adopting Anthropic's MCP standard for tool interoperability to implementing multi-modal foundation model strategies, they're creating AI systems that can evolve with rapidly improving underlying technologies while maintaining enterprise reliability standards. Topics discussed: Defining AI agents as systems with control flow autonomy rather than simple workflow automation or chatbot interfaces. Building enterprise trust in AI agents through precision-focused evaluation systems that measure performance across specific incident scenarios. Implementing root cause identification agents that diagnose production issues before engineers wake up during critical outages. Adopting Anthropic's MCP standard for tool interoperability to enable seamless integration across different agent platforms and environments. Using LLM-as-judge evaluation methods combined with human alignment scoring to continuously improve agent reliability and performance. Managing multi-modal foundation model strategies that allow switching between OpenAI, Anthropic, and open-source models based on tasks. Balancing organizational AI adoption through decentralized experimentation with centralized procurement standards and security compliance oversight. Developing LLM observability products that cluster errors and provide visibility into token usage and model performance. Navigating the bitter lesson principle by building evaluation frameworks that can quickly test new foundation models. Predicting timeline and bottlenecks for AGI development based on current reasoning limitations and architectural research needs.

How do you achieve 75% weekly active AI tool usage across an enterprise? At UserTesting, it started with a deliberate strategy to foster experimental adoption while maintaining governance guardrails. Michael J. Domanic, Head of Generative AI Business Strategy, tells Ravin their approach to transforming internal workflows through AI on this episode of The AI Adoption Playbook. Rather than treating AI as purely a technical concern, UserTesting has built an impressive culture of AI adoption by empowering cross-functional teams, establishing clear usage guidelines, and meticulously tracking tangible business value. Their 800-person team has created 600 custom GPTs, many of which have transformed workflows across departments. Michael also explores how UserTesting balanced centralized governance with democratized experimentation, their methods for measuring ROI, and why creativity — not just technical expertise — might be the critical ingredient for successful enterprise AI transformation. Topics discussed: - Building an enterprise AI strategy that achieved 75% weekly active users through balanced governance and democratized experimentation. - Creating custom GPTs trained on organizational knowledge to streamline processes like OKR development across all company levels. - Establishing cross-functional ”AI ambassadors” and regular office hours to drive adoption and showcase successful use cases. - Implementing clear AI usage guidelines that protect sensitive customer data while encouraging internal experimentation and innovation. - Measuring AI ROI by focusing on business outcomes rather than time savings, with metrics tied to specific operational improvements. - Balancing centralized AI governance through a dedicated council with decentralized, department-level experimentation and implementation. - Exploring the differences between AI transformation and traditional digital transformation, including executive-level buy-in from day one. - Discussing the challenges incumbent software companies face when integrating AI versus new AI-native applications built from scratch. - Developing strategies for scaling AI applications from experimentation to production through appropriate security and governance protocols. - Examining the philosophical aspects of AGI development and the importance of creativity rather than technical skills in leading AI transformation.

When technical systems fail at companies like Netflix or Etsy, every minute of downtime can cost millions. That’s why incident.io is building AI systems that can automatically investigate and diagnose technical problems faster than human engineers. In this episode of The AI Adoption Playbook, Lawrence Jones, Product Engineer at incident.io, tells Ravin how they’re creating an automated incident investigator that can analyze logs, traces, and metrics to determine what went wrong during an outage. He shares their methodical approach to AI development, focusing on measurable progress through evaluation metrics and scorecards rather than intuitive ”vibe-based” changes. Lawrence also discusses the evolution of their AI teams and roles, including their newly launched AI Engineer position designed specifically for the unique challenges of AI development, and how they use LLMs themselves to evaluate AI system performance. Topics discussed: - Building an AI incident investigator that can automatically analyze logs, traces, and metrics to determine the root cause of technical outages. - Creating comprehensive evaluation frameworks with scorecards and metrics to measure AI performance against historical incident data. - Using LLMs as evaluators to determine if AI responses were helpful by analyzing post-incident conversations and user feedback. - Developing internal tooling that enables teams to rapidly test and improve AI systems while maintaining quality standards. - Evolving from individual ”vibe-based” AI development to team-based systematic improvement with clear metrics for success. - Structuring AI engineering roles and teams to balance product engineering skills with specialized AI development knowledge. - Implementing product-focused AI features like chatbots that can help automate routine tasks during incident response. - Leveraging parallel human and AI processes to collect validation data and improve AI system performance over time. - Building versus buying AI evaluation tools and the advantages of custom solutions integrated with existing product data. - Exploring the future of AI in technical operations and whether AI will enhance or replace human roles in incident management. Listen to more episodes:  Apple  Spotify  YouTube

What happens when AI capabilities outpace organizational readiness? At Shopify, this tension has pushed them to develop a practical implementation approach that balances rapid experimentation with sustainable value creation.  Spencer Lawrence, Director of Data Science & Engineering, shares how they’ve evolved from simple text expansion experiments to sophisticated AI assistants like Help Center and Sidekick that are transforming both customer support and merchant operations. At the heart of their strategy is a barbell approach enabling self-service for small AI use cases while making targeted investments in transformative projects. Spencer also explains how their one-week sprint cycles, sophisticated evaluation frameworks, and cross-functional collaboration have helped them overcome the common challenges that prevent organizations from realizing AI’s full potential. Successful AI implementation requires more than just technical solutions — it demands new organizational structures, evaluation methods, and a willingness to constantly reevaluate what knowledge work means in an AI-augmented world. Topics discussed: - Shopify’s evolution from early text expansion experiments to production-level AI assistants that support both customers and merchants. - Creating sophisticated evaluation frameworks that combine human annotators with LLM judges to ensure quality and consistency of AI outputs. - Implementing a barbell strategy that balances small self-service AI use cases with strategic investments in high-impact projects. - Running one-week sprints across all AI work to maximize iteration cycles and maintain velocity even at enterprise scale. - Addressing the gap between AI capabilities and real-world impact through both technological solutions and organizational change. - Building feedback loops between technical teams and legal/compliance departments to create AI solutions that meet governance requirements. - Fostering a culture that values experimentation while developing clear policies that give employees confidence to innovate responsibly. - Exploring how AI will raise productivity expectations rather than simply reducing workloads across all roles and functions. - Using AI as a strategic thought partner to generate novel ideas and help evaluate different perspectives on complex problems. - Developing a forward-looking perspective on knowledge work that embraces AI augmentation while maintaining human judgment and oversight. Listen to more episodes:  Apple  Spotify  YouTube

Forget theoretical planning — MongoDB dove headfirst into AI adoption and let real-world usage guide their strategy. David Vainchenker, Sr. Director of Enterprise Initiatives & Tools at MongoDB, joins Ravin on this episode of The AI Adoption Playbook to share this practical approach and unpack their evolution from simple chatbots to sophisticated agent-based systems.  David shares their practical challenges with measuring AI’s business impact, explaining why time savings metrics alone weren’t convincing to leadership without translating to actual dollar savings or increased capacity. He also offers candid insights about security concerns, copyright issues with AI-generated code, and the delicate balance between innovation and governance. Topics discussed: - Why shipping AI tools quickly and learning from actual usage patterns proved more effective than predicting theoretical use cases. - The challenge of translating AI time savings into measurable business impact that resonates with leadership and affects the P&L. - Security and compliance considerations when implementing AI at enterprise scale, including permission-aware retrieval requirements. - Managing the balance between build vs. buy decisions in the fast-evolving AI landscape while ensuring business continuity. - The reality of AI-assisted coding adoption rates varying significantly between junior and senior engineers in large organizations, and the copyright implications of having non-human-generated code. - How MongoDB approaches vertical (specialized) vs. horizontal (platform) AI solutions for different use cases across the enterprise. - The budgeting challenges created when every existing software vendor offers AI capabilities as premium add-ons. - The importance of maintaining cross-system AI capabilities that match human workflows spanning multiple applications. Listen to more episodes:  Apple  Spotify  YouTube Website

In this episode of The AI Adoption Playbook, Allen Jeter, Director of IT at Lattice, describes how his team transformed internal operations by strategically implementing AI assistants across multiple departments. Starting with a clear focus on reducing manual work and response times, Allen walks Ravin through how Lattice built their first AI solutions, from an experimental chatbot using Okta Workflows and Pinecone to production-grade systems serving their People Operations and security teams. What sets Lattice's approach apart is their pragmatic focus on solving real business problems rather than chasing AI for its own sake. By identifying specific pain points, implementing security guardrails from the beginning, and deploying AI directly within existing workflows like Slack, they've achieved impressive adoption across the organization. Allen also shares invaluable advice for IT leaders looking to implement AI, emphasizing early experimentation, stakeholder involvement, and the importance of understanding your business problems before attempting AI solutions. Topics discussed: Implementing AI assistants for People Operations that provide 24/7 support for employee questions about benefits and company policies. Building a security bot that helps sales teams respond to customer security questionnaires faster, reducing bottlenecks and accelerating sales cycles. Evaluating the crowded AI vendor landscape with specific requirements rather than getting caught up in marketing hype. The importance of integrating AI tools into existing workflows like Slack channels to maximize adoption without changing user behavior. Creating effective prompt engineering strategies to help teams customize AI responses and maintain accuracy across different domains. Implementing proper governance and permissions structures that respect existing data access controls to ensure compliance. Measuring success through concrete metrics like reduction in manual work hours and decreased time-to-answer across departments. Using AI to enrich support ticket metadata automatically, enabling better insights without manual categorization work. Balancing experimentation with security guardrails to enable innovation while protecting sensitive company and customer data. Resources Mentioned: Credal’s blog post, “The Enterprise Adoption Curve: Lessons Learned So Far”

What's the secret to successful AI adoption? According to Robert Mitchell, Chief AI Officer at WSI, it's not just about choosing the right tools, but it's about mastering a delicate balance between executive vision and hands-on experimentation. After helping countless mid-sized businesses implement AI, Robert explains why these organizations need fundamentally different approaches than enterprises, focusing on quick wins and easy implementation over maximum capability. His conversation with Ravin on this episode of The AI Adoption Playbook explores WSI's unique dual-track implementation approach, combining executive planning with grassroots experimentation. Robert shares practical insights on building effective AI councils with representation across all business functions, ensuring that AI initiatives benefit from diverse perspectives and real-world operational knowledge. Robert also walks through WSI’s proven framework for balancing top-down strategy with bottom-up experimentation, why SMBs require different solutions than enterprises, and how to build truly cross-functional AI governance. Topics discussed: A proven "top-down, bottom-up" implementation framework that combines executive buy-in with identifying and empowering internal AI champions who can drive adoption through monthly AI Council meetings and team challenges Detailed ROI calculation methodology for AI initiatives, illustrated through a case study showing how 10% productivity gains on a $5M payroll can translate to $3M in additional business value at 6x EBITDA multiple Specific approach to AI governance using three core documents - policies for internal data usage, client data handling, and vendor data management - that must be established before any employee training begins Concrete example of high-ROI automation: a $20K investment to eliminate 7 days of manual accounting work monthly, improving employee satisfaction while enabling team to focus on higher-value activities Strategic methodology for creating "aha moments" by having employees first experiment with AI in their personal domain expertise before applying it to work processes, making adoption more intuitive Practical framework for quick wins: identifying 90-minute process improvements through Loom video analysis of employee pain points, then rapidly implementing targeted AI solutions

What happens when you enable SQL engineers to build sophisticated AI systems with just 26 lines of code? At Ramp, it sparked a transformation in how they approach AI implementation: instead of relying on ML specialists, they're democratizing AI development across their organization. But that's just one part of Ramp's unconventional story. As Head of Analytics Engineering & Data Science, Ian Macomber explains how processing financial data for 30,000 companies led them to reimagine enterprise AI architecture. By building model-agnostic systems that can switch seamlessly between foundation models, they're creating sustainable competitive advantages while maintaining cost efficiency. In this episode of The AI Adoption Playbook, Ravin sits down with Ian to unpack how Ramp evolved from basic receipt matching to sophisticated cross-functional AI systems that are reshaping enterprise financial operations.   Topics discussed: How Ramp's decentralized approach to AI procurement enabled rapid experimentation while maintaining standards and leading to the discovery that most point-solution AI tools show surprisingly weak retention rates. Their data architecture strategy: managing complex databases with 133,000 columns by implementing semantic layers and guardrails that make enterprise-scale text-to-SQL accessible to non-specialists. The small team philosophy they implemented: breaking larger teams into units of 12 or fewer people, fostering rapid iteration while maintaining the velocity of a startup despite growing to over 1,000 employees. Their approach to foundation model selection: treating AI providers like ride-sharing services, constantly evaluating performance and cost metrics to switch between models based on specific use cases. Their strategy for building defensible AI products: focus on sophisticated integrations that combine organizational data, spending policies, and real-time market information in ways that point solutions can't replicate. Listen to more episodes:  Apple  Spotify  YouTube Website   Episode 2.

What happens when you pause all company operations for a week to teach everyone — from marketers to legal teams — how to build AI co-pilots? At Checkr, it sparked an innovation wave from an unexpected source: their non-technical teams started outpacing engineers in creating practical AI solutions. But that's just one part of Checkr's unconventional story. As Head of ML/AI, Muskan Kukreja explains how the rise of the gig economy forced them to reimagine background checks for a world where workers change jobs daily. By using AI to drive costs down to $1 per check, they're expanding trust-building tools beyond enterprise clients to serve entirely new markets. In our first episode of The AI Adoption Playbook, Ravin Thambapillai, CEO of Credal.AI sits down with Muskan to unpack how Checkr is using AI to transform background checks from a weeks-long process into a same-day service. How Checkr's week-long company-wide AI hackathon yielded an unexpected outcome: non-technical teams (legal, marketing) outpaced engineers in creating practical AI solutions by focusing on their daily pain points rather than technical capabilities Their data processing architecture: handling billions of searches across thousands of data sources by implementing AI-powered verification workflows with human-in-the-loop fallbacks for edge cases The "T-shaped team" structure they implemented: instead of hiring separate specialists (data scientists, ML engineers, AI researchers), they built teams with broad skills across disciplines who deeply understood business problems Their approach to high-stakes AI applications: enforcing a strict checklist process that includes PII masking, data retention policies, and mandatory security/legal/ethics reviews for any customer-facing AI feature Their 90-day shipping philosophy: breaking down large AI initiatives into quarter-sized chunks with clear KPIs, allowing rapid iteration while maintaining compliance with emerging AI regulations Learn more about Checkr's approach to AI implementation on their engineering blog or read about their compliance framework in their trust center.