Best Llmops Podcasts (2025)

1
The LLMOps Shift with Abi Aryan 32:16

Play Pause

3d ago32:16

32:16

MLOps is dead. Well, not really, but for many the job is evolving into LLMOps. In this episode, Abide AI founder and LLMOps author Abi Aryan joins Ben to discuss what LLMOps is and why it’s needed, particularly for agentic AI systems. Listen in to hear why LLMOps requires a new way of thinking about observability, why we should spend more time unde…

1
Generative AI in the Real World: The LLMOps Shift with Abi Aryan

15d ago

—

MLOps is dead. Well, not really, but for many the job is evolving into LLMOps. In this episode, Abide AI founder and LLMOps author Abi Aryan joins Ben to discuss what LLMOps is and why it’s needed, particularly for agentic AI systems. Listen in to hear why LLMOps requires a new way of thinking about […]…

1
Laurence Moroney on AI at the Edge 36:32

8d ago36:32

36:32

In this episode, Laurence Moroney, director of AI at Arm, joins Ben Lorica to chat about the state of deep learning frameworks—and why you may be better off thinking a step higher, on the solution level. Listen in for Laurence’s thoughts about posttraining; the evolution of on-device AI (and how tools like ExecuTorch and LiteRT are helping make it …

1
Generative AI in the Real World: Laurence Moroney on AI at the Edge

22d ago

—

In this episode, Laurence Moroney, director of AI at Arm, joins Ben Lorica to chat about the state of deep learning frameworks—and why you may be better off thinking a step higher, on the solution level. Listen in for Laurence’s thoughts about posttraining; the evolution of on-device AI (and how tools like ExecuTorch and LiteRT […]…

1
Chris Butler on GenAI in Product Management 38:20

23d ago38:20

38:20

In this episode, Ben Lorica and Chris Butler, director of product operations for GitHub's Synapse team, chat about the experimentation Chris is doing to incorporate generative AI into the product development process—particularly with the goal of reducing toil for cross-functional teams. It isn’t just automating busywork (although there’s some of th…

1
Generative AI in the Real World: Chris Butler on GenAI in Product Management

1M ago

—

In this episode, Ben Lorica and Chris Butler, director of product operations for GitHub’s Synapse team, chat about the experimentation Chris is doing to incorporate generative AI into the product development process—particularly with the goal of reducing toil for cross-functional teams. It isn’t just automating busywork (although there’s some of th…

1
Context Engineering with Drew Breunig 41:01

2M ago41:01

41:01

In this episode, Ben Lorica and Drew Breunig, a strategist at the Overture Maps Foundation, talk all things context engineering: what’s working, where things are breaking down, and what comes next. Listen in to hear why huge context windows aren’t solving the problems we hoped they might, why companies shouldn’t discount evals and testing, and why …

1
Generative AI in the Real World: Context Engineering with Drew Breunig

2M ago

—

In this episode, Ben Lorica and Drew Breunig, a strategist at the Overture Maps Foundation, talk all things context engineering: what’s working, where things are breaking down, and what comes next. Listen in to hear why huge context windows aren’t solving the problems we hoped they might, why companies shouldn’t discount evals and testing, and […]…

1
Emmanuel Ameisen on LLM Interpretability 29:04

2M ago29:04

29:04

In this episode, Ben Lorica and Anthropic interpretability researcher Emmanuel Ameisen get into the work Emmanuel’s team has been doing to better understand how LLMs like Claude work. Listen in to find out what they’ve uncovered by taking a microscopic look at how LLMs function—and just how far the analogy to the human brain holds.…

1
Generative AI in the Real World: Emmanuel Ameisen on LLM Interpretability

2M ago

—

In this episode, Ben Lorica and Anthropic interpretability researcher Emmanuel Ameisen get into the work Emmanuel’s team has been doing to better understand how LLMs like Claude work. Listen in to find out what they’ve uncovered by taking a microscopic look at how LLMs function—and just how far the analogy to the human brain holds. […]…

1
Understanding A2A with Heiko Hotz and Sokratis Kartakis 33:10

2M ago33:10

33:10

Everyone is talking about agents: single agents and, increasingly, multi-agent systems. What kind of applications will we build with agents, and how will we build with them? How will agents communicate with each other effectively? Why do we need a protocol like A2A to specify how they communicate? Join Ben Lorica as he talks with Heiko Hotz and Sok…

1
Faye Zhang on Using AI to Improve Discovery 22:11

2M ago22:11

22:11

In this episode, Ben Lorica and AI Engineer Faye Zhang talk about discoverability: how to use AI to build search and recommendation engines that actually find what you want. Listen in to learn how AI goes way beyond simple collaborative filtering—pulling in many different kinds of data and metadata, including images and voice, to get a much better …

1
Luke Wroblewski on When Databases Talk Agent-Speak 29:00

2M ago29:00

29:00

Join Luke Wroblewski and Ben Lorica as they talk about the future of software development. What happens when we have databases that are designed to interact with agents and language models rather than humans? We’re starting to see what that world will look like. It’s an exciting time to be a software developer.…

1
Jay Alammar on Building AI for the Enterprise 42:37

3M ago42:37

42:37

Jay Alammar, director and Engineering Fellow at Cohere, joins Ben Lorica to talk about building AI applications for the enterprise, using RAG effectively, and the evolution of RAG into agents. Listen in to find out what kinds of metadata you need when you’re onboarding a new model or agent; discover how an emphasis on evaluation helps an organizati…

1
Phillip Carter on Where Generative AI Meets Observability 38:00

3M ago38:00

38:00

Phillip Carter, formerly of Honeycomb, and Ben Lorica talk about observability and AI—what observability means, how generative AI causes problems for observability, and how generative AI can be used as a tool to help SREs analyze telemetry data. There’s tremendous potential because AI is great at finding patterns in massive datasets, but it’s still…

1
ELO Ratings Questions 3:39

3M ago3:39

3:39

Key Argument Thesis: Using ELO for AI agent evaluation = measuring noise Problem: Wrong evaluators, wrong metrics, wrong assumptions Solution: Quantitative assessment frameworks The Comparison (00:00-02:00) Chess ELO FIDE arbiters: 120hr training Binary outcome: win/loss Test-retest: r=0.95 Cohen's κ=0.92 AI Agent ELO Random users: Google engineer?…

1
Raiza Martin on Building AI Applications for Audio 36:00

3M ago36:00

36:00

Audio is being added to AI everywhere: both in multimodal models that can understand and generate audio and in applications that use audio for input. Now that we can work with spoken language, what does that mean for the applications that we can develop? How do we think about audio interfaces—how will people use them, and what will they want to do?…

1
The 2X Ceiling: Why 100 AI Agents Can't Outcode Amdahl's Law" 4:19

3M ago4:19

4:19

AI coding agents face the same fundamental limitation as parallel computing: Amdahl's Law. Just as 10 cooks can't make soup 10x faster, 10 AI agents can't code 10x faster due to inherent sequential bottlenecks. 📚 Key Concepts The Soup Analogy Multiple cooks can divide tasks (prep, boiling water, etc.) But certain steps MUST be sequential (can't sti…

1
Stefania Druga on Designing for the Next Generation 33:07

3M ago33:07

33:07

How do you teach kids to use and build with AI? That’s what Stefania Druga works on. It’s important to be sensitive to their creativity, sense of fun, and desire to learn. When designing for kids, it’s important to design with them, not just for them. That’s a lesson that has important implications for adults, too. Join Stefania Druga and Ben Loric…

1
Douwe Kiela on Why RAG Isn’t Dead 34:46

3M ago34:46

34:46

Join our host Ben Lorica and Douwe Kiela, cofounder of Contextual AI and author of the first paper on RAG, to find out why RAG remains as relevant as ever. Regardless of what you call it, retrieval is at the heart of generative AI. Find out why—and how to build effective RAG-based systems. Points of Interest 0:25: Today’s topic is RAG. With frontie…

1
Danielle Belgrave on Generative AI in Pharma and Medicine 31:32

3M ago31:32

31:32

Join Danielle Belgrave and Ben Lorica for a discussion of AI in healthcare. Danielle is VP of AI and machine learning at GSK (formerly GlaxoSmithKline). She and Ben discuss using AI and machine learning to get better diagnoses that reflect the differences between patients. Listen in to learn about the challenges of working with health data—a field …

1
The Startup Opportunity with Gabriela de Queiroz 30:51

3M ago30:51

30:51

Ben Lorica and Gabriela de Queiroz, director of AI at Microsoft, talk about startups: specifically, AI startups. How do you get noticed? How do you generate real traction? What are startups doing with agents and with protocols like MCP and A2A? And which security issues should startups watch for, especially if they’re using open weights models? Poi…

1
Securing AI with Steve Wilson 42:50

3M ago42:50

42:50

Join Steve Wilson and Ben Lorica for a discussion of AI security. We all know that AI brings new vulnerabilities into the software landscape. Steve and Ben talk about what makes AI different, what the big risks are, and how you can use AI safely. Find out how agents introduce their own vulnerabilities, and learn about resources such as OWASP that c…

1
Shreya Shankar on AI for Corporate Data Processing 30:13

3M ago30:13

30:13

Businesses have a lot of data—but most of that data is unstructured textual data: reports, catalogs, emails, notes, and much more. Without structure, business analysts can’t make sense of the data; there is value in the data, but it can’t be put to use. AI can be a tool for finding and extracting the structure that’s hidden in textual data. In this…

1
Vibe Coding with Steve Yegge 39:58

3M ago39:58

39:58

Ever since Andrej Karpathy first tweeted it, “vibe coding” has been on every software developer’s mind. Join Ben Lorica and Steve Yegge to find out what vibe coding means, especially in a professional context. Going beyond the current memes, what will the future of software development look like when we have multiple agents? And how do you prepare …

1
Interactions Between Humans and AI with Rajeshwari Ganesan 33:22

3M ago33:22

33:22

In this edition of Generative AI in the Real World, Ben Lorica and Rajeshwari Ganesan talk about how to put generative AI in closer touch with human needs and requirements. AI isn’t all about building bigger models and benchmarks. To use it effectively, we need better interfaces; we need contexts that support groups rather than individuals; we need…

1
Getting Beyond the Demo with Hamel Husain 32:08

3M ago32:08

32:08

In this episode, Ben Lorica and Hamel Husain talk about how to take the next steps with artificial intelligence. Developers don’t need to build their own models—but they do need basic data skills. It’s important to look at your data, to discover your model’s weaknesses, and to use that information to develop test suites and evals that show whether …

1
Agents—The Next Step in AI with Shelby Heinecke 27:24

3M ago27:24

27:24

Join Shelby Heinecke, senior research manager at Salesforce, and Ben Lorica as they talk about agents, AI models that can take action on behalf of their users. Are they the future—or at least the hot topic for the coming year? Where are we with smaller models? And what do we need to improve the agent stack? How do you evaluate the performance of mo…

1
Measuring Skills with Kian Katanforoosh 30:55

3M ago30:55

30:55

How do we measure skills in an age of AI? That question has an effect on everything from hiring to productive teamwork. Join Kian Katanforoosh, founder and CEO of Workera, and Ben Lorica for a discussion of how we can use AI to assess skills more effectively. How do we get beyond pass/fail exams to true measures of a person’s ability? Points of Int…

1
Chloé Messdaghi on AI Security, Policy, and Regulation 30:05

3M ago30:05

30:05

Chloé Messdaghi and Ben Lorica discuss AI security—a subject of increasing importance as AI-driven applications roll out into the real world. There’s a knowledge gap: Security workers don’t understand AI, and AI developers don’t understand security. It’s important to be aware of all the resources that are available. Make sure to bring everyone toge…

1
Tom Smoker on Getting Started with GraphRAG 35:24

3M ago35:24

35:24

Join Ben Lorica and Tom Smoker for a discussion of GraphRAG, one of the hottest topics of the last few months. GraphRAG goes a step beyond RAG to make the output of language models more consistent, accurate, and explainable. But what is a graph? A graph is a way of structuring data. In the end, it’s the structure that’s important, along with the wo…

1
Robert Nishihara on AI and the Future of Data 29:58

3M ago29:58

29:58

Robert Nishihara is one of the creators of Ray and cofounder of Anyscale, a platform for high-performance distributed data analysis and artificial intelligence. Ben Lorica and Robert discuss the need for data for the next generation of AI, which will be multimodal. What kinds of data will we need to develop models for video and multimodal data? And…

1
Getting Ahead of the Curve with Claire Vo 26:50

3M ago26:50

26:50

In this episode, Ben Lorica talks with Claire Vo, chief product officer at Launch Darkly and founder of ChatPRD. AI gives us a new set of tools that make everyone more productive and efficient. Those tools will allow more experimentation; they will allow more people to participate in product development; and they will create new opportunities for s…

1
The Future of Programming with Matt Welsh 46:02

3M ago46:02

46:02

Join us for a conversation between Ben Lorica and Matt Welsh, cofounder of Fixie.ai, former engineer at Apple and Google, and one of Mark Zuckerberg’s professors at Harvard. Learn how AI is changing computing. Whether it’s in C or a human language, programming is telling a computer what you want it to do—but AI opens up new classes of things that w…

1
Kingsley Ndoh on Improving Cancer Care with AI 35:12

3M ago35:12

35:12

What can AI do to improve healthcare? Kingsley Ndoh, founder of Hurone AI, talks with Ben Lorica about how Hurone is making cancer care more effective for people who are underserved by the medical system. He discusses how AI can streamline the medical process, both helping doctors to treat patients more effectively and making clinical trials more d…

1
Putting AI in the Hands of Farmers with Rikin Gandhi 34:54

3M ago34:54

34:54

Rikin Gandhi, CTO of Digital Green, talks with Ben Lorica about using generative AI to help farmers in developing countries become more productive. Farmer.Chat integrates information from training videos, sources of weather and crop information, and other data sources in a multimodal app that farmers can use in real-time. Points of Interest 0:45: D…

1
Adopting AI in the Enterprise with Timothy Persons 33:56

4M ago33:56

33:56

Timothy Persons of PricewaterhouseCoopers (PwC) talks with Ben Lorica about adoption of AI in the enterprise. They discuss the challenges enterprises experience, including the need to change corporate culture. To succeed, it’s important to focus on solving well-defined problems rather than just doing something cool with AI. Good data strategies and…

1
Learning How to Do AI Effectively with Alfred Spector 40:11

4M ago40:11

40:11

Alfred Spector has been a leader in AI and machine learning at Google, IBM, and Two Sigma. He is now a visiting scholar at MIT, an advisor at Blackstone, and coauthor of the text book Data Science in Context. Alfred talks with Ben Lorica about what people developing with AI need to be successful. Succeeding with AI is about more than just a model. …

1
Generative AI in the Real World: Understanding A2A with Heiko Hotz and Sokratis Kartakis

4M ago

—

Everyone is talking about agents: single agents and, increasingly, multi-agent systems. What kind of applications will we build with agents, and how will we build with them? How will agents communicate with each other effectively? Why do we need a protocol like A2A to specify how they communicate? Join Ben Lorica as he talks with […]…

1
Andrew Ng on where AI is headed. It’s about agents. 27:39

4M ago27:39

27:39

Andrew Ng is one of the pioneers of modern AI. He was Google Brain’s founding technical lead, Coursera’s founder, Baidu’s Chief Scientist, DeepLearning.ai’s founder, a Professor at Stanford—and much more. Andrew talks with Ben Lorica about scaling AI, agents, the future of open source AI, and openness among AI researchers. Have you experienced an “…

1
Democratizing AI with Gwendolyn Stripling 34:26

4M ago34:26

34:26

Gwendolyn Stripling, author of Low-Code AI, talks about the democratization of AI, the primacy of data, the future of data science, and the coming of agents. It’s easy to think that AI is all about algorithms and models but it’s not; it’s really about understanding the business use case and the data that can be applied to that use case. We’re only …

1
Competing in a Generative World with Justin Norman 36:56

4M ago36:56

36:56

Justin Norman, author of Product Management for AI and co-founder of Vera, a startup focused on security for generative AI, talks with Ben Lorica about how product management has changed since Generative AI came on the scene. He discusses the issues retrieval-augmented generation (RAG) raises for product management; how reliability has become part …

1
Pete Warden on Running AI on Small Systems 33:58

4M ago33:58

33:58

Pete Warden, founder of Useful Sensors and co-author of TinyML, discusses use cases for artificial intelligence that we rarely think about: how can you run AI on very small systems? How can you put AI on consumer devices in ways that are actually useful and not just buzzword-compliant? AI doesn’t have to rely on massive GPU farms. Pete talks about …

1
Chip Huyen on Finding Business Use Cases for Generative AI 34:59

4M ago34:59

34:59

O’Reilly’s Generative AI in the Enterprise survey reported that people have trouble coming up with appropriate enterprise use cases for AI. Why is it hard to come up with appropriate use cases? Chip Huyen, cofounder of Claypot AI and author of Designing Machine Learning Systems, talks about why many companies have trouble coming up with appropriate…

1
Generative AI in the Real World: Jay Alammar on Building AI for the Enterprise

4M ago

—

Jay Alammar, director and Engineering Fellow at Cohere, joins Ben Lorica to talk about building AI applications for the enterprise, using RAG effectively, and the evolution of RAG into agents. Listen in to find out what kinds of metadata you need when you’re onboarding a new model or agent; discover how an emphasis on evaluation […]…

1
Generative AI in the Real World: Phillip Carter on Where Generative AI Meets Observability

4M ago

—

Phillip Carter, formerly of Honeycomb, and Ben Lorica talk about observability and AI—what observability means, how generative AI causes problems for observability, and how generative AI can be used as a tool to help SREs analyze telemetry data. There’s tremendous potential because AI is great at finding patterns in massive datasets, but it’s still…

1
Generative AI in the Real World: Raiza Martin on Building AI Applications for Audio

5M ago

—

Audio is being added to AI everywhere: both in multimodal models that can understand and generate audio and in applications that use audio for input. Now that we can work with spoken language, what does that mean for the applications that we can develop? How do we think about audio interfaces—how will people use them, […]…

1
Generative AI in the Real World: Danielle Belgrave on Generative AI in Pharma and Medicine

6M ago

—

Join Danielle Belgrave and Ben Lorica for a discussion of AI in healthcare. Danielle is VP of AI and machine learning at GSK (formerly GlaxoSmithKline). She and Ben discuss using AI and machine learning to get better diagnoses that reflect the differences between patients. Listen in to learn about the challenges of working with health […]…

1
Plastic Shamans of AGI 10:32

7M ago10:32

10:32

The plastic shamans of OpenAI 🔥 Hot Course Offers:- 🤖 Master GenAI Engineering - Build Production AI Systems- 🦀 Learn Professional Rust - Industry-Grade Development- 📊 AWS AI & Analytics - Scale Your ML in Cloud- ⚡ Production GenAI on AWS - Deploy at Enterprise Scale- 🛠️ Rust DevOps Mastery - Automate Everything🚀 Level Up Your Career:- 💼 Production…

1
The Toyota Way: Engineering Discipline in the Era of Dangerous Dilettantes 14:38

7M ago14:38

14:38

Dangerous Dilettantes vs. Toyota Way Engineering Core Thesis The influx of AI-powered automation tools creates dangerous dilettantes - practitioners who know just enough to be harmful. The Toyota Production System (TPS) principles provide a battle-tested framework for integrating automation while maintaining engineering discipline. Historical Conte…