Zum Inhalt springen
Global sound
Weaviate Podcast

Weaviate Podcast

REFRAG with Xiaoqiang Lin - Weaviate Podcast #130!

· 1 Std. 0 Min.

Shownotes

Xiaoqiang Lin is a Ph.D. student at the National University of Singapore. During his time at Meta, Xiaoqiang lead the research behind REFRAG: Rethinking RAG-based Decoding. Traditional RAG systems use vectors to retrieve relevant context with semantic search, but then throw away the vectors when passing the context to the LLM. REFRAG instead feeds the LLM these pre-compute vectors, achieving massive gains in long context processing and LLM inference speed! REFRAG makes Time-To-First-Token (TTFT) 31x faster and Time-To-Iterative-Token (TTIT) 3x faster, boosting overall LLM throughput by 7x…

Website: podcasters.spotify.com

Live, letzte Stunde: 6.643 Songs auf 2.313 Sendern in 108 Ländern

Entdecke die Welt über das Radio

Mehr zum Entdecken (7)

Etwas Schönes gehört? Speicher es oder öffne es direkt in Spotify.

★ Künstler der Woche Der schnellste Aufsteiger im Radio diese Woche
Dein Song hier? Ab € 9 pro Woche →
Free Radio RotterdamWeiterhören