Best Llm Evaluation Podcasts (2025)

1
EP 628: What’s the best LLM for your team? 7 Steps to evaluate and create ROI for AI 38:52

Play Pause

7d ago38:52

38:52

How can you measure ROI on GenAI for your team? 🤔 Internal evaluations and intentionality. We've helped thousands of orgs put LLMs to work and ACTUALLY save time. On today's show, we're dishing the 7 steps you need to follow. What’s the best LLM for your team? 7 Steps to evaluate and create ROI for AI -- An Everyday AI chat with Jordan Wilson Newsl…

1
Elena Samuylova on Large Language Model (LLM) Based Application Evaluation and LLM as a Judge 35:44

10d ago35:44

35:44

In this podcast, InfoQ spoke with Elena Samuylova from Evidently AI, on best practices in evaluating Large Language Model (LLM) based applications. She also discussed the tools for evaluating, testing and monitoring applications powered by AI technologies.Read a transcript of this interview: https://bit.ly/4mHAKvN Subscribe to the Software Architec…

1
How Good Is It, Really? - A Guide to LLM Evaluation 7:45

27d ago7:45

7:45

In the season finale of "All Things LLM," hosts Alex and Ben turn to one of the most important—and challenging—topics in AI: How do we objectively evaluate the quality and reliability of a language model? With so many models, benchmarks, and metrics, what actually counts as “good”? In this episode, you’ll discover: The evolution of LLM evaluation: …

1
Ep 633: The 3 Big Obstacles Holding AI Adoption Back 33:04

35m ago33:04

33:04

Jeetu Patel knows a few AI secrets. As the President of one of the largest companies in the world, he's helped pave the AI adoption roadmap. At Cisco, they provide full-stack, enterprise AI solutions spanning infrastructure, security, observability, and operations to the world's largest companies. So naturally, Jeetu could write a legit playbook on…

1
Ep 632: ChatGPT Apps: 3 Hands-on approaches to save time today 43:34

1d ago43:34

43:34

You haven't used ChatGPT's Apps yet? 🫠 Oh.... you like wasting time? Even for free users, ChatGPT rolled out its new Apps mode that promises to shift the future of work. Don't know how to work it? Don't know where to start? Join us as we share 3 practical ways to start saving time today. ChatGPT Apps: 3 Hands-on approaches to save time today -- An …

1
Dataflow Computing for AI Inference with Kunle Olukotun - #751 57:37

2d ago57:37

57:37

In this episode, we're joined by Kunle Olukotun, professor of electrical engineering and computer science at Stanford University and co-founder and chief technologist at Sambanova Systems, to discuss reconfigurable dataflow architectures for AI inference. Kunle explains the core idea of building computers that are dynamically configured to match th…

1
Ep 631: AI’s App Store Moment? Are ChatGPT’s Apps The Next Big Thing or Smoke and Mirrors? 35:58

2d ago35:58

35:58

Will this be AI's 'App Store Moment'? 🤔 OpenAI's Apps are live, and the consensus is split. Some are calling them a revolutionary step forward while others are saying it's another marketing flop. What's our hot take? Join us and find out. AI’s App Store Moment? Are ChatGPT’s Apps The Next Big Thing or Smoke and Mirrors? An Everyday AI Chat with Jor…

1
EP 630: OpenAI brings Apps and Agents to ChatGPT, Google drops Gemini Enterprise, is the AI bubble here and more 38:19

3d ago38:19

38:19

OpenAI debuted the future of ChatGPT with Agents and Apps. How will that impact work? 🤖 Google dropped Gemini for Enterprise. Does that make them the top AI option for the big players? 🏢 Everyone is talking about the AI bubble. Is it real and will it burst? 🫧 If you have questions over what's happening in the world of AI news, we've got answers. Jo…

1
#741: Modernizing Edge Infrastructure: Booking.com's Journey with AWS CloudFront and Lambda@Edge 1:03:45

3d ago1:03:45

1:03:45

In this episode of the AWS Podcast, host Jillian Forde discusses the migration journey of Booking.com to AWS with Ali and Sarah. They explore the challenges faced by Booking.com , the benefits of using CloudFront and Lambda at Edge, and the importance of observability and cost optimization. The conversation also delves into chaos engineering practi…

1
Mental Models in Architecture & Societal Views of Technology: A Conversation with Nimisha Asthagiri 51:51

3d ago51:51

51:51

In this podcast, Michael Stiefel spoke with Nimisha Asthagiri about the importance of system thinking, multi-agent systems, the consequences of society applying a technology into an area for which it was not designed, and whether we can ever have a healthy relationship with artificial intelligence. System thinking emphasizes the importance of menta…

1
Ep 629: Google’s surprise release: Will Gemini Enterprise Compete with ChatGPT and Microsoft Copilot? 39:45

6d ago39:45

39:45

Breaking: Google just released Gemini Enterprise. 🚨 Will it be a ChatGPT or Microsoft Copilot killer? We got our hands on a version of the newest release and will break down everything you need to know, including the ONE feature that could ultimately set Gemini Enterprise apart. Google Gemini Enterprise: Coming for ChatGPT and Microsoft Copilot? --…

1
Ep 627: NotebookLM: New features, what’s next and complete walkthrough 39:00

8d ago39:00

39:00

Have you been sleeping on NotebookLM? 😴 If so, you're leaving hours of productivity (and probably a lot of money) at the door. But real talk -- the team is shipping fast. The NotebookLM you met last year from the viral Audio Overviews is not the NotebookLM of today. It's slowly turned into a robust, multimedia powerhouse. And the last feature updat…

1
Recurrence and Attention for Long-Context Transformers with Jacob Buckman - #750 57:23

9d ago57:23

57:23

Today, we're joined by Jacob Buckman, co-founder and CEO of Manifest AI to discuss achieving long context in transformers. We discuss the bottlenecks of scaling context length and recent techniques to overcome them, including windowed attention, grouped query attention, and latent space attention. We explore the idea of weight-state balance and the…

1
Ep 626: ChatGPT’s new Agent Builder, Apps and 5 BIG takeaways you probably missed 34:02

9d ago34:02

34:02

Will this be the AI update that finally brings AI agents to millions? 🤔 Probably. OpenAI had a straight up feast of AI drops at its Dev Day conference, but one of the biggest was its drag-and-drop agent builder. Oh, and literally bringing entire website experiences into ChatGPT via apps. Don't miss this one. Ep 626: ChatGPT’s new Agent Builder, App…

1
EP 625: Sora 2 release and update, OpenAI and AMD partner, ChatGPT Visual Agent Builder incoming and more AI News That Matters 44:11

10d ago44:11

44:11

OpenAI is dropping a visual agent builder 🤯 There's a HUGE report on AI job losses.... Sora 2 gets good and bad news... And that's just the beginning. Make sure to join us for this week's AI News That Matters! EP 625: Sora 2 release and update, OpenAI and AMD partner, ChatGPT Visual Agent Builder incoming and more Newsletter: Sign up for our free d…

1
#740: AWS News: EC2 Instance Attestation, S3 Conditional Deletes, and more! 31:46

10d ago31:46

31:46

Simon and Jillian keep you up to date with over 70 new releases and capabilities!By Amazon Web Services

1
From Automation to Autonomy: Practical Steps for Enterprise AI Adoption with Accenture's Mary Hamilton (Replay) 29:48

13d ago29:48

29:48

If most companies are using the same AI systems, how can they stand out and get ahead? And as agentic AI becomes table stakes, what do enterprises need to keep in mind to make AI work? And how can we even trust an AI-powered workplace when most people can't even explain the basics of AI? We're learning from the experts. Accenture's Mary Hamilton jo…

1
Ep 623: Sora 2: AI TikTok Brain Rot or Your Company’s Secret Creative Weapon? 47:32

14d ago47:32

47:32

Everyone's talking about Sora 2. 🗣️ - How good it is - How it's going after TikTok as a social media app - The downsides of cyclical AI brain rot But, you're missing the big upside. On today's show, we're going to tell you how businesses should be focusing on the untapped potential of Sora 2. Want to enter our Sora code giveaway? Go repost today's …

1
Ep 622: Gemini AI in Google Sheets: What’s new and 5 daily tasks you didn’t know you could do 39:07

15d ago39:07

39:07

You definitely missed this AI drop 👇 AI functions are live inside of Google Sheets thanks to Gemini. I know what you're saying... AI functions? I can barely spread the sheets! That's the beauty of it. With a simple =AI of your keyboard, you can talk to your spreadsheets in natural language and tap into the full power of Google Gemini. Don't worry..…

1
The Decentralized Future of Private AI with Illia Polosukhin - #749 1:05:03

16d ago1:05:03

1:05:03

In this episode, Illia Polosukhin, a co-author of the seminal "Attention Is All You Need" paper and co-founder of Near AI, joins us to discuss his vision for building private, decentralized, and user-owned AI. Illia shares his unique journey from developing the Transformer architecture at Google to building the NEAR Protocol blockchain to solve glo…

1
Ep 621: Microsoft’s new Agent mode: Microsoft marketing or Future of Work in Windows? 41:51

16d ago41:51

41:51

Is this the AI agent we've all been waiting for? 🤔 Maybe. Microsoft just unveiled their highly capable Agent Mode and Office Agent. Capabilities? Through the roof. Execution, rollout and availability? ummmm...... Join us as we cut through the fluff on these new AI agents from Microsoft and separate the game-changing features from the shiny marketin…

1
Ep 620: Apple’s ChatGPT rival, OpenAI’s multi-billion partnerships, Accenture lays off 11,000 due to AI and more AI News That Matters 35:53

17d ago35:53

35:53

Is Apple serious about AI now with a new internal model? 🤔 Why did Accenture lay off 11,000 and how many more will go due to AI? How the heck is OpenAI signing so many multi-billion dollar partnerships? So many AI questions. We've got the AI answers. Don't waste hours each week trying to keep up with the AI news. We do that for you with our Mondays…

1
#739: Applying Multi-region Strategies on AWS 41:18

17d ago41:18

41:18

Amazon Application Recovery Controller (ARC) Region Switch - https://docs.aws.amazon.com/r53recovery/latest/dg/region-switch.htmlBy Amazon Web Services

1
The Hidden Vulnerability of The Open Source Software Supply Chain: The Underlying Infrastructure 41:48

17d ago41:48

41:48

Software supply chain veteran Brian Fox unpacks the security implications of the new EU Cyber Resilience Act and its profound impact on open-source projects. He reveals the hidden infrastructure risks threatening open-source projects and shares insights for senior software leaders navigating this regulatory landscape.Read a transcript of this inter…

1
EP 619: Nano Banana Uncovered: A practical guide from inside Google with Paige Bailey 28:38

20d ago28:38

28:38

Everyone know Google's Nano Banana is bonkers good. 🍌 But did you know you can create an app in minutes that embeds Nano Banana.... and it takes zero coding experience?! 🤯 If you haven't used Google's Gemini 2.5 Flash (AKA Nano Banana), you're in for a treat as Google's Paige Bailey gives us the insider's guide. Nano Banana Uncovered: A practical g…

1
EP 618: RSL vs. the AI Scrape: Can LLM licensing save the open web? 24:02

21d ago24:02

24:02

AI scraping vs. the open web. Who wins? 🥊 Let's say the quiet part out loud: AI companies have trained their models for years on your company's website data, regardless of if you want them to. Fast forward to today: many publishers have lost up to 70% of website traffic (and huge chunks of revenue) because of this. So what happens if some of these …

1
EP 617: Gemini in Google Chrome: What’s new and 6 easy use cases 33:32

22d ago33:32

33:32

AI is coming for the world's most popular browser. 🤖 Google recently announced Gemini in Chrome, bringing AI (and eventually) agentic features to the browser used by more than 3 billion people. We've been using Gemini in Chrome for a few months, so we're recapping some of our favorite features, tips and 6 easy use cases that you can take advantage …

1
AI, ML, and Data Engineering InfoQ Trends Report 2025 53:02

22d ago53:02

53:02

In this episode of the podcast, members of the InfoQ editorial staff and friends of InfoQ discuss the current trends in the domain of AI, ML and Data Engineering.One of the regular features of InfoQ are the trends reports, which each focus on a different aspect of software development. These reports provide the InfoQ readers and listeners with a hi…

1
Inside Nano Banana 🍌 and the Future of Vision-Language Models with Oliver Wang - #748 1:03:39

23d ago1:03:39

1:03:39

Today, we’re joined by Oliver Wang, principal scientist at Google DeepMind and tech lead for Gemini 2.5 Flash Image—better known by its code name, “Nano Banana.” We dive into the development and capabilities of this newly released frontier vision-language model, beginning with the broader shift from specialized image generators to general-purpose m…

1
EP 616: If Microsoft is haunted by AI, should everyone else be nervous too? 35:28

23d ago35:28

35:28

The headline was kinda shocking 👇 "Satya Nadella is haunted at the prospect of Microsoft not surviving the AI era" This was a story in The Verge that detailed an internal Microsoft town hall and its CEO's kinda urgent plea on Microsoft keeping pace. But real talk now..... if one of the key companies pushing AI innovation is kinda haunted by keeping…

1
Ep 615: OpenAI going after Apple hardware, Google brings Gemini to Chrome, Microsoft CEO’s dire warning on AI and more AI News That Matters 37:37

24d ago37:37

37:37

OpenAI is now going after .... Apple? 🍎 Google is finally bringing more AI to its browser. 🖥️ And Microsoft's CEO has cast a kinda gloomy warning on AI. ⚠️ And that's just the beginning. Don't waste hours each day trying to keep up with AI. That's what we do. OpenAI going after Apple hardware, Google brings Gemini to Chrome, Microsoft CEO’s dire wa…

1
#738: AWS News: Global Cross-Region Inference, Aurora Limitless and lots more. 25:01

24d ago25:01

25:01

Simon and Jillian keep you up to date with all the latest releases and capabilities!By Amazon Web Services

1
EP 614: 5 Business Use-Cases for Comet, Perplexity’s agentic browser 39:18

27d ago39:18

39:18

Make way for the next wave of GenAI..... Agentic AI Browsers. And while we've seen rumors that OpenAI is going all-in on an AI browser, the first big player is already here in Perplexity's Comet Browser. Join us as we break down how Perplexity Comet works, what makes it different, and 5 Business Use-Cases for ROI. Newsletter: Sign up for our free d…

1
From Prediction to Action - Autonomous Agents and LAMs 8:39

27d ago8:39

8:39

What happens when AI not only understands the world, but acts in it? In this trailblazing episode of "All Things LLM," Alex and Ben chart the rise of next-generation AI: autonomous agents and Large Action Models (LAMs). Discover how LLMs are evolving from passive text generators to powerful doers—reshaping workflows, business automation, and the ve…

1
The Paradigm Shift & The Black Box: Reasoning Models and the Quest for Understanding 11:38

27d ago11:38

11:38

In the grand finale of "All Things LLM," hosts Alex and Ben look ahead to the bleeding edge—and reflect on the ultimate question for AI: can we ever truly understand how these models think? Inside this episode: The rise of reasoning models: Discover why the next leap for AI isn’t just bigger models, but smarter thinking. Explore how OpenAI’s o1 and…

1
More Then Words - The Rise of Multimodal LLMs 7:42

27d ago7:42

7:42

AI’s next great leap isn’t about bigger models—it’s about broader senses. In this season premiere of "All Things LLM," Alex and Ben explore the revolutionary world of multimodal large language models (LLMs)—the new frontier where AI can “see,” “hear,” and “understand” the world far beyond text. In this episode: Journey to Multimodality: Discover wh…

1
Prompt Injections and Data Poisoning - Securing Your LLM 8:38

27d ago8:38

8:38

As LLMs power more business workflows, security risks grow. In this essential episode of "All Things LLM," hosts Alex and Ben break down the new wave of cybersecurity threats targeting language models—and what you can do to defend your AI infrastructure. What you’ll learn: The OWASP Top 10 for LLMs: Explore the most pressing LLM security risks and …

1
Open vs. Closed - The Great Model Debate 9:00

28d ago9:00

9:00

Season 4 of "All Things LLM" kicks off with one of the most crucial debates in AI today: open-source vs. closed-source (proprietary) language models. Hosts Alex and Ben cut through the hype to explain what’s at stake for businesses, developers, and the entire AI ecosystem. In this episode, you’ll discover: The fundamentals: What truly sets open-sou…

1
Hallucinations, Bias, and Black Boxes - The Biggest Challenges 9:04

28d ago9:04

9:04

Powerful language models are reshaping the world, but serious challenges remain. In this revealing episode of "All Things LLM," hosts Alex and AI expert Ben tackle the core limitations and ethical risks facing all large language models—open or closed. This episode covers: Hallucinations: Why LLMs make up plausible-sounding but false or misleading a…

1
The Human Touch - How RLHF Aligns Models with Our Values 5:54

28d ago5:54

5:54

How do we make AI not just smart, but safe and genuinely helpful? In this episode of "All Things LLM," Alex and Ben break down the vital process of alignment—transforming a powerful language model into a trustworthy assistant you can rely on. Inside this episode: What is RLHF? Discover Reinforcement Learning from Human Feedback—the multi-stage proc…

1
Cooking with Terabytes - Training an LLM from Scratch 7:33

28d ago7:33

7:33

Get an insider’s look behind the curtain of modern AI with this episode of "All Things LLM." Join hosts Alex and AI expert Ben as they reveal the colossal effort, expense, and ingenuity required to take a language model from “blank slate” to foundational intelligence. What you’ll learn: The massive scale of LLM training: how developers assemble and…

1
The Magic of "Attention" - How LLMs Understand Context 5:39

28d ago5:39

5:39

Unlock the key to modern AI with this deep-dive episode of "All Things LLM"! Hosts Alex and our resident AI expert Ben unpack the “self-attention mechanism”—the heart of every powerful Transformer model powering GPT, Llama, Gemini, and more. Discover: What “self-attention” actually means in the context of language models—and why it’s a game-changer…

1
The Art of the Ask - A Practical Guide to Prompt Engineering 6:11

28d ago6:11

6:11

Unlock the full potential of large language models with this hands-on episode of "All Things LLM." Hosts Alex and AI expert Ben break down the essential (and rapidly evolving) discipline of prompt engineering—your steering wheel for directing AI toward more relevant, accurate, and actionable outputs. What you’ll learn: Prompt Engineering 101: Why c…

1
Making It Yours - Fine-Tuning, PEFT, and RAG 7:26

28d ago7:26

7:26

Discover how generalist AIs become powerful specialists in this episode of "All Things LLM." Hosts Alex and AI expert Ben break down the next stage of the LLM lifecycle—customization—and unpack the practical techniques that transform foundation models into domain experts or business-ready assistants. Learn about: Fine-Tuning: Why it’s essential for…

1
The Blueprint - Encoders, Decoders, and Model Families 5:42

28d ago5:42

5:42

Unlock the mysteries of modern AI with "All Things LLM." In this episode, Alex and Ben break down the Transformer—the revolutionary engine powering today’s Large Language Models (LLMs) like GPT-4, Llama, and Gemini. If you’ve ever wondered how AI can both understand and generate text, this deep dive into Transformer architecture is your essential g…

1
From Turing to Transformers - A Brief History of Language AI 6:52

28d ago6:52

6:52

Curious how AI language models like ChatGPT burst into the mainstream? Welcome to "All Things LLM," where hosts Alex and AI expert Ben unravel the true origins and evolution of Large Language Models. In this episode, we journey through more than a century of discoveries that paved the way for today’s groundbreaking AI. Discover: The surprising root…