MP3•Episode home
Manage episode 478821138 series 2814833
Content provided by Databricks. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Databricks or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.
In this episode, Pallavi Koppol, Research Scientist at Databricks, explores the importance of domain-specific intelligence in large language models (LLMs). She discusses how enterprises need models tailored to their unique jargon, data, and tasks rather than relying solely on general benchmarks.
Highlights include:
- Why benchmarking LLMs for domain-specific tasks is critical for enterprise AI.
- An introduction to the Databricks Intelligence Benchmarking Suite (DIBS).
- Evaluating models on real-world applications like RAG, text-to-JSON, and function calling.
- The evolving landscape of open-source vs. closed-source LLMs.
- How industry and academia can collaborate to improve AI benchmarking.
43 episodes