Jeff Huber, Founder & CEO of Chroma, joins Jay to discuss why context engineering remains the core job of AI engineers, and how modern search infrastructure is evolving for AI-native applications.

Jeff brings deep experience building Chroma into the leading AI application database, serving thousands of production AI agents with advanced search capabilities across code, dependencies, and unstructured data.

The conversation centers around the reality of context rot in long-context models, why RegX vs semantic search debates miss the point, and how memory systems need to move beyond simple retrieval to enable true agent learning.

Jeff explains why Chroma has indexed all major open source dependencies across NPM, PyPI, Cargo, and Go, enabling agents to search exact package versions instead of hallucinating APIs.

Tune into the full episode to learn why context engineering remains the bottleneck for AI reliability and how search infrastructure will evolve beyond simple vector similarity!

HIGHLIGHTS:
0:00 Intro
2:24 AI databases vs traditional databases
4:39 Context engineering as core AI developer job
6:21 Context rot research - million-token degradation
8:32 Long context performance vs marketing claims
12:15 Prior failures boost agent performance, successes hurt it
16:36 LLMs as query planners inside databases
19:15 Why coding became first dominant AI use case
21:06 RegX vs semantic search propaganda wars
24:21 Language servers on crack for code search
28:03 Multi-branch agent coordination
30:09 Code Collections - searching NPM/PyPI packages
31:21 Forkable collections enable 100ms Git indexing
34:09 Deep research agents are "incredibly mid"
38:17 Memory as context engineering vs weights
40:36 Agent task learning vs user personalization
43:18 Auditability problem with model-weight memory
47:21 Agents need apprenticeship models for reliability
49:54 Embarrassing lack of AI UX innovation

Connect with Jeff - https://www.linkedin.com/in/jeffchuber/

Connect with Jay - https://www.linkedin.com/in/jayhack/ or https://x.com/mathemagic1an

Visit trychroma.com for AI application database infrastructure