Weaviate Podcast

Ghita Houir Alami, Co-Founder of ZeroEntropy and now leading Model Research at Notion, joins the Weaviate Podcast to discuss training rerankers, calibrated relevance, and where search models fit in the age of agents. The conversation opens with the ZeroEntropy story: starting from an end-to-end search API, realizing the models powering it mattered most, and shipping a reranker first because swapping in a stateless last-stage API call is far easier for customers than re-embedding a whole corpus. A partnership powering Notion's enterprise search, plus fast query-rewriting models for parallel searches, led to the acquisition.From there, the discussion dives into small, task-specific decision models and turning cross-encoders into classifiers. Most rerankers trained with triplet loss cluster their scores between 0.4 and 0.6, but a calibrated score unlocks a dynamic K, returning three results or twenty depending on actual relevance, a natural partner to Weaviate's autocut. Ghita then explains zELO, the method behind zerank-1 and zerank-2: hard negative mining eventually surfaces unlabeled positives, so ZeroEntropy instead trained on pairwise comparisons, fit Elo scores through a Thurstone model with sparse sampling of pairwise battles, and distilled a pairwise reranker into an efficient pointwise one that nearly matches its accuracy.The conversation moves into infrastructure, from quantization-aware training and reranker-specific batching across US GPUs to blending reserved capacity with Modal for bursty traffic. Ghita makes the case for multi-stage retrieval, since embeddings alone hit dimensionality limits, with a small listwise stage on top only when needed. On agentic search, she argues a lightweight reranking step saves expensive LLM tokens, citing Cursor's finding that semantic search made Composer use fewer tokens with fewer undos, while Notion's harness shows that models that search more answer better. Instruction-following rerankers redefine relevance itself: for a memory system, "I'm highly allergic to nuts" is the most relevant answer to "What should I cook for dinner tonight?" Metadata like price is often better handled by a separate model such as XGBoost or Weaviate's Boost API.The episode closes on the excitement of post-training open-weight models for repeatable tasks inside companies, RSI and using AI to train AI!

Shangyin Tan, a Ph.D. student at U.C. Berkeley joins the Weaviate Podcast to discuss Scaling Discovery Requires Forgetting. This paper explores why long-running AI optimizers stall and how to get them climbing again. The conversation opens by defining discovery through the lens of GEPA, the prompt optimizer that uses an LLM as the proposer, and its successor, Optimize Anything, which extends the idea to code, GPU kernels, and agent harnesses. Shangyin separates three kinds of problems: generalization (prompts and harnesses that must work on unseen inputs), single-problem optimization (one kernel where the score you see is the score you get), and multi-problem optimization in between.From there, the discussion dives into forgetting. Running GEPA, Karpathy-style autoresearch, and Meta Harness behind one front end, every backend hit the same plateau after a couple of hours, and handing the best candidate to a fresh optimizer with no other context unblocked it every time. The evidence is striking: as sessions grow, agents use fewer thinking tokens per turn, edit fewer lines, and drift toward tweaking numbers instead of trying new algorithms, with one model repeatedly declaring it had "reached the practical limit" right before finding another improvement. And the best candidate isn't a loss of history, since it already encodes the ideas from every iteration that built it.The conversation then moves into multi-agent teams, where agents communicate only by reading each other's Git diffs rather than full files, so a weaker agent borrows an idea without collapsing into a copy. Teams climb faster and forgetting prevents plateaus, together making a strong autoresearch harness. Shangyin and Connor also cover GEPA's Pareto frontier as a tool for diversity, routing inputs to specialized harnesses, the Fork and Flush paper's idea basins, just-in-time databases, and why expensive or undefinable evaluations are the biggest gap between autoresearch and production.The episode closes with the mismanaged genius hypothesis: frontier models are already incredibly smart, and the real work is building harnesses that stop getting in their way!

Jacob X. Li , a Ph.D. student at MIT joins the Weaviate Podcast to explain Machine Studying, a new framing for how AI agents can prepare themselves for a domain defined only by a corpus of documents, whether that's a code base, a pile of drug discovery papers, or a system's logs. Just like a student studying for an exam they haven't seen, the agent studies the corpus without knowing the downstream tasks, and still has full access to the corpus at test time. Rather than leaning entirely on inference-time compute the way Codex or Claude Code explore a repository from scratch, can an agent develop arbitrarily deep understanding ahead of time?The conversation dives into how the paper defines expertise: an expert is an agent that efficiently turns inference-time search into accurate work, measured as the weighted area under the performance versus compute curve, with the search budget of a ReAct-style agent loop as the knob instead of reasoning effort. Stacking that one level higher gives studying intelligence, expertise as a function of study-time budget, with retraining from scratch as the baseline. From there, Jacob and Connor discuss learning to search, including a literature review experiment where an outdated model with BM25 finds roughly the same papers as a newer model but is far worse at deciding which ones matter, and why studying is more than retrieval.Jacob then separates Machine Studying from continual learning, which is framed around sequences of benchmarks and often collapses into long-context processing, and lays out the three bets every studying method makes: self-supervised objectives or synthetic data, updating weights through LoRA, KV-cache cartridges or a trained retriever, and amortized context management like notes and cheat sheets. Cheat sheets helped on DSPy but less on the larger, newer OpenClaw code base, and weights versus context, along with where prompt optimization like GEPA fits, is left deliberately open.The episode closes with how human study habits like self-quizzing and annotating inspire these algorithms, agents that study you before teaching you, and why evaluation as a science matters when recursive self-improvement happens in domains with no benchmark at all!

Shu Liu, Alexander Krentsel, Mert Cemri, and Shubham Agarwal, PhD students at U.C. Berkeley's Sky Computing Lab, join the Weaviate Podcast to discuss SkyDiscover and what it means for AI to build entire systems, not just code! AI coding went from Copilot-style line completion to function- and file-level generation with Claude Code. The next step is to optimize or synthesize systems that used to take PhD-level expertise, such as databases, inference engines, and key-value stores. With coding agents, "just-in-time systems" become possible: software tailored to one workload, free of the "generality tax" of general-purpose systems.The conversation centers on synthesizing a key-value store, where workloads vary in read/write ratio, hot keys, consistency, reliability, and latency versus throughput goals. Running on curated and production traces from Tencent and Twitter, the agents designed new caching and hierarchical memory schemes that reached up to 4x better performance than specialized systems from Microsoft. The hard part is trust. On YCSB traces, early builds showed 30–40x throughput because Claude was reward hacking: it worked out values from the keys. SkySynth acts as a "vaccine" against this. It asks the user questions to build the spec, then takes either a test-driven route or a formal verification route. In the test-driven route, a test only counts if it can tell a correct reference implementation from an incorrect one.From there, the discussion covers how the spec decides the programming language (Rust for memory safety, C for raw performance, Lean, Rocq, CompCert, and Verus for proofs). A surprising result: building on top of Microsoft FASTER gave lower throughput than synthesizing from scratch. The panel's advice is to "skate to where the puck is going" and let capable models design without constraints. The panel also imagines hosted systems resynthesized daily on recent traffic, and PRDs as the spec for consumer products.The episode closes on optimization. Unlike text optimization with GEPA, 80–90% of system optimization time goes to evaluation, and vLLM alone has more than 25,000 tests. That makes finding the right problem, building proxy evaluators, and generating adversarial workloads a meta-problem in its own right.

Chapters
0:00 Welcome Shu, Shubham, Alexander, and Mert!
2:22 An Overview of SkyDiscover
7:55 Specialized Database Systems
12:33 DSPy for Systems
23:23 Formal Verification of AI-Generated Systems
26:34 Programming Languages and Vibe Coding
32:48 Building Systems from Scratch
40:26 Software for Everyone!
45:26 GEPA for Vibe Coding

Siddharth Gollapudi, a researcher at UC Berkeley, joins the Weaviate Podcast to discuss in-context retrieval and his paper "Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale." The idea is simple and radical: instead of encoding documents into embeddings and building a nearest neighbor index, put the entire corpus into an LLM's context window and use attention itself as the retriever. The conversation opens by separating this from generative retrieval, which memorizes documents into model parameters and needs gradient updates every time the corpus changes, and from listwise re-ranking, which Siddharth frames as an easier subset of first-stage retrieval. Lost in the middle, he argues, turned out to be much less of a problem with today's long-context models.The discussion then dives into BlockSearch, a 0.6B parameter in-context retriever trained on MS MARCO (cleaned with RLHN) plus a mix of BEIR datasets. Reaching 500 documents was relatively easy; scaling to 2,500, 5,000, and 10,000 documents, or roughly a million tokens, is where things break. Siddharth explains why random document codes beat positional codes (they force the model to actually read the documents), and how an on-policy loss that corrects the model's own rollouts keeps it disciplined when hard negatives pull it off course midway through generating a code.From there, the conversation moves to attention dilution: as the corpus grows, the softmax denominator swamps the relevant document's score. Two fixes help, length-dependent temperature scaling and sparse attention that drops irrelevant documents before attention runs, bringing BlockSearch roughly level with dense retrieval at a million tokens. This opens a big question for vector databases: is there a sublinear, ANN-style version of attention that can be trained end to end, in the spirit of ReFrag and ColBERT's late interaction?The episode closes on timelines for LLM-based re-rankers, the latency trade-offs of high-latency search, "No More Free Lunch" and whether long-context LLMs can eat the database, recursive language models, and Siddharth's excitement for unsupervised notions of relevance that could help models make genuinely new discoveries, such as proving open theorems.
Chapters0:00 Welcome Siddharth!1:21 In-Context Retrieval8:16 Drowning in Documents and Reranker Scaling13:05 BlockSearch, 0.6B In-Context Retriever28:21 Vector Databases for LLM Inference40:46 Timelines for In-Context Retrievers46:54 Will LLMs eat Databases?51:38 Exciting Directions for AI