Weaviate Podcast

Sam O'Nuallain joins the Weaviate Podcast to discuss AutoIndex, research from UMass Amherst and Databricks on learning representation programs for retrieval. Instead of tuning the retriever or re-ranker, AutoIndex asks how the data itself should be represented: a two-agent system, an analysis agent and a code agent, writes and refines Python programs that chunk, enrich, and reorganize a corpus to optimize downstream metrics like Recall and nDCG.The conversation opens with why indexing is such a natural target for code optimization. Frontier LLMs are exceptional at writing code, a representation program applies cheaply across an entire corpus without passing every document through an LLM, and code is verifiable. Every hypothesis the code agent proposes is gated against a validation set before it is accepted. From there, the discussion dives into the optimization signal. The analysis agent uses tools to read documents, query the retriever, and inspect where gold documents rank, turning a bare score like "recall went up" into rich natural language feedback about why a representation is failing, echoing ideas like GEPA's reflective metrics and the value of small-margin positives for training re-rankers.That leads into why BM25 pairs so well with agents: its lexical transparency makes failures easy to diagnose, illustrated by case studies from the CRUMB benchmark. This includes LaTeX formatting errors sinking Stack Overflow retrieval and Tip-of-the-Tongue movie search, where AutoIndex learned to repeat plots to up-weight terms and expand documents with synonym dictionaries. The conversation moves through connections to document enrichment methods like Anthropic's contextual retrieval, doc2query, and EnrichIndex, generalizing representation programs to text-to-SQL schemas and data lakehouses. The podcast concludes by discussing Sam's lessons transitioning from research to production AI engineering: loop engineering, QA, and evals. It lands on the directions that excite Sam most: harness design, continual learning, and memory as a retrieval problem, squeezing more out of the models we already have without touching the weights.

Chapters
0:24 An Overview of AutoIndex
4:04 Retrieval Indexing as Code Optimization
8:47 Optimizing Chunking and Database Schemas
15:57 Feedback for Search Optimization
23:57 Future Directions for AutoIndex
27:34 Document Enrichment for RAG
36:32 Web Search vs. Databases
40:40 AI Engineering
49:30 Exciting Directions for AI

Alex Zhang, a PhD student at MIT, joins the Weaviate Podcast to discuss Recursive Language Models (RLMs), a new abstraction for designing agent harnesses. Instead of the standard ReAct-style loop that stuffs every tool observation into an ever-growing prompt, an RLM treats the prompt as a variable in a program. The model writes code that manipulates its own context and spawns recursive LLM calls over pieces of it. The published headline was long-context performance, but the deeper intention, inspired by how DSPy programmers decompose tasks, is letting the model do that decomposition itself, relieving context pressure so each call only sees a small, local, in-distribution problem. Stacking tool outputs into one giant sequence is terribly out of distribution for how models were trained. Frontier labs spend enormous resources making those long trajectories in-distribution, but RLMs aim to get more from the model without that.From there the conversation turns to PrimeAgent, Prime Intellect's production harness centered on an RLM, where the only tool is a persistent IPython REPL: search, compaction, sub-agents, and skills are all callable inside code, yielding strong results on ARC-AGI-3 and long-horizon benchmarks with big token savings. Alex shares why they moved from prompt tuning to RL on the weights of Qwen3, and why Claude Code's dynamic workflows are RLM-like but constrained by legacy tool-calling defaults.Alex then previews an unreleased library for speculative programmatic tool calling. Borrowing from speculative decoding and CPU speculative execution, a shadow executor queues up sub-agent calls while the main model is still generating code, delivering roughly 2x speedups and better GPU utilization for locally served agents. The discussion moves into running RLMs in the cloud with sandbox providers like Modal and Daytona, security concerns around code execution and prompt injection, and why serving engines should be redesigned around agents. It closes with RLMs and search, combining ColBERT-style retrievers as callable tools for problems like agent log analysis, and a theory that everything "long" is naturally decomposable. The podcast concludes with exciting ideas about where AI goes next from principled harness design to whether the transformer itself gets replaced with a newer neural architecture design.

Mathew Jacob, lead author of "Drowning in Documents: Consequences of Scaling Reranker Inference" and now a PhD student in ML systems at the University of Washington, joins the Weaviate Podcast to unpack one of the most surprising results in modern search: cross-encoder rerankers get worse as you give them more documents. The paper began during his Databricks internship, where scaling reranking past roughly 100 documents sent recall@10 plummeting, a result so counterintuitive he assumed it was a bug.The conversation digs into why this happens, reframing rerankers through the lens of boosting, rather than being strictly stronger than first-stage retrievers. Cross-encoders are very good at correcting retriever errors within the distribution they were trained on. Full-scoring experiments over 10,000 randomly sampled documents drive the point home, with BM25 beating state-of-the-art cross-encoders. From there, the discussion moves into phantom hits, cases where wildly irrelevant documents scored highly. For example, a dishwasher document surfacing for a query about disease in Gabonese children. We also discuss whether ensembling rerankers can patch these false positives.The second half explores what comes next for reranking: prompt-based listwise reranking with sliding windows, which proved far more robust than pointwise scoring; RankZephyr-style fine-tuning versus encoding learning signal in prompts with GEPA and DSPy, reasoning rerankers like Rank1 and their latency trade-offs, hard negative mining behind ZeroEntropy's zELO, and pairwise and setwise designs that sit between cross-encoders and full listwise ranking. Adaptive retrieval comes into focus through Natural Language Query to Configuration for Retrieval Agents, predicting per query whether to run simple retrieval, multi-hop, or full agentic search to push the cost-quality frontier.The conversation lands on TraceLab, from Mathew's lab at UW: 40,000 real traces harvested from Claude Code and Codex usage, revealing how coding agents actually behave, prefix cache patterns, long-tailed tool calls, and how understanding these workloads unlocks the next generation of serving optimizations.

Weaviate co-founders Bob van Luijt and Etienne Dilocker return to the Weaviate Podcast to celebrate seven years of building the company, answering questions submitted by the community. The conversation opens with what excites them most in AI right now: Etienne on agentic coding and the "Moore's law" of how long models can sustain autonomous loops, and Bob on world models, new architectures that could slash training energy costs, open source frontier models, and inference on new chips.From there, the discussion dives into taste and the "AI slopification" problem, why AI-generated emails, websites, and decks all look the same, how three job candidates submitted nearly identical AI-built presentations in one week, and why Weaviate runs a dedicated "slop pass" skill over every pull request to strip out phrases like "the smoking gun" and "load-bearing invariant." The human touch, they argue, is now the easiest way to stand out.The Co-Founders then retell their origin story: meeting at a European enterprise company, rewriting a NodeJS prototype in Go, betting on NLP before anyone called it AI, adopting HNSW when it was still a niche paper, and raising a $1.2M seed round from Zeta during COVID. When ChatGPT and the RAG paper hit, Weaviate had a fully working product ready for the wave.Looking forward, Bob breaks down the commoditization playbook that hits every new database category, the same skepticism MongoDB faced, and shares that the number one reason new customers cite for choosing Weaviate is that an LLM recommended it. Etienne makes the case that vector databases are evolving into context engines: context rot is real, stuffing everything into a long context window is inefficient, and retrieval, hybrid search, and structured data all serve one goal, the best possible context. The conversation lands on memory for AI agents, where the hard problem isn't what's worth remembering, but what's worth recalling.

Dr. Bradley Allen brings five decades of AI history into a deep conversation on knowledge engineering, neurosymbolic AI, and the future of enterprise intelligence. The discussion begins with the boom-and-bust cycle of rule-based expert systems, AI winters, and why today’s large language model wave may be different. The conversation then turns to how knowledge is organized in practice, from personal piles of papers searched on demand to formal knowledge graphs built with classes, relations, ontologies, A boxes, T boxes, description logic, and subsumption-based reasoning. Allen explains why semantic web and biomedical ontology successes still leave unresolved questions about cost, maintenance, and whether LLMs can dynamically structure information in ways that preserve meaning. That leads into natural language concept definitions, LLM-based classifiers, rationales, probabilistic reasoning, and the challenge of updating classes as new edge cases emerge.From there, the focus widens to vector databases, semantic search, RAG, topic modeling, distributional semantics, and the ongoing revision required for systems that can never be “once and done.” Allen connects modern LLM behavior to the long history of formal languages, from Frege, Russell, Wittgenstein, Turing, and Gödel to theorem proving, soundness, completeness, paraconsistency, paracompleteness, and the pragmatic tradition of meaning through use. The closing stretch explores world models, reinforcement learning, tool-using agents, enterprise knowledge workflows, role-based access control, governance, normativity, and alignment, ending on the need to build accountable AI systems that channel powerful technology toward responsible outcomes.