REFRAG with Xiaoqiang Lin - Weaviate Podcast #130!
Xiaoqiang Lin is a Ph.D. student at the National University of Singapore. During his time at Meta, Xiaoqiang lead the research behind REFRAG: Rethinking RAG-based Decoding. Traditional RAG systems use vectors to retrieve relevant context with semantic search, but then throw away the vectors when passing the context to the LLM. REFRAG instead feeds the LLM these pre-compute vectors, achieving massive gains in long context processing and LLM inference speed! REFRAG makes Time-To-First-Token (TTFT) 31x faster and Time-To-Iterative-Token (TTIT) 3x faster, boosting overall LLM throughput by 7x while also being able to handle much longer contexts!There are so many interesting aspects to this and I really loved diving into the details with Xiaoqiang! I hope you enjoy the podcast!
--------
1:00:00
--------
1:00:00
Weaviate and SAS with Saurabh Mishra and Bob van Luijt - Weaviate Podcast #129!
This episode dives into Weaviate's partnership with SAS! We are super excited about our recent collaboration on the SAS Retrieval Agent Manager (RAM), featuring a first party integration with Weaviate! The podcast dives into all sorts of aspects of Enterprise AI adoption from what has changed, to what has NOT changed with recent breakthroughs in AI systems!
--------
43:55
--------
43:55
Weaviate's Query Agent with Charles Pierse - Weaviate Podcast #128!
Charles Pierse is the Director of the Weaviate Labs team, where he has recently lead the GA release of the Weaviate Query Agent. The podcast begins with the journey from alpha to GA release, discussing unexpected lessons and the collaborations between teams at Weaviate. Continuing on the product design, we cover the design of the Python and TypeScript clients and how to think about response models with Agent products. Then diving into the tech, we cover several different aspects of the Query Agent from question answering with citations, to schema introspection and typing for database querying, multi-collection routing, and the newly introduced Search Mode. We also discuss the Weaviate Query Agent's integration with the Cloud Console, a GUI home for the Weaviate Database! We are also super excited to share a case study from one of the Query Agent's power uses, MetaBuddy! The podcast concludes with the MetaBuddy case study and some exciting directions for the future development of the Query Agent.
--------
1:01:32
--------
1:01:32
GEPA with Lakshya A. Agrawal - Weaviate Podcast #127!
Lakshya A. Agrawal is a Ph.D. student at U.C. Berkeley! Lakshya has lead the research behind GEPA, one of the newest innovations in DSPy and the use of Large Language Models as Optimizers! GEPA makes three key innovations on how exactly we use LLMs to propose prompts for LLMs, (1) Pareto-Optimal Candidate Selection, (2) Reflective Prompt Mutation, and (3) System-Aware Merging. The podcast discusses all of these details further, as well as topics such as Test-Time Training and the LangProBe benchmarks used in the paper! I hope you find the podcast useful!
--------
1:01:55
--------
1:01:55
Agentic Topic Modeling with Maarten Grootendorst - Weaviate Podcast #126!
Maarten Grootendorst is a psychologist turned AI engineer who has created BERTopic and authored "Hands-On Large Language Models" with Jay Alammar. The rise of LLMs and Agents are transforming many areas of software! This podcast dives deep into their impact on Topic Modeling! Maarten designed BERTopic from the start with modularity in mind -- letting you ablate embedding models, dimensionality reduction, clustering algorithms, and more. This early insight to prioritize modularity makes BERTopic incredibly well structured to become more "Agentic". An "Agentic" Topic Modeling algorithm can use LLMs to generate topics or topic descriptions, as well as contrast them with other topics. It can decide which topics to subdivide, and it can integrate human feedback and evaluate topics in novel ways... I hope you find the podcast interesting!
South Africa