Snacks Weekly on Data Science
by Pan Wu
Compressing LLM Agent context [Shopify]
About this podcast
This podcast is about making data science and machine learning knowledge accessible and less intimidating. Every week, I will handpick one selected industrial tech blog to break it down. We will discuss some key data science concepts and machine learning algorithms, and how they are applied in those real-world applications.
Subscribe to the channel and enjoy Snacks Weekly on Data Science!
Episodes
The newest 20 of 164 episodes
- Compressing LLM Agent context [Shopify] · 9 min · Episode 163In this episode, we discuss how Shopify tackled a recurring cost in its AI agent by compressing detailed system instructions into a smaller learned representation. The approach…
- Scale Human Labeling with LLMs for Better Search Relevance [Dropbox] · 8 min · Episode 162In this episode, we talk about how Dropbox improved the search ranking performance of one of its key products. The team used a small amount of human judgment to calibrate an LLM…
- Unifying Ads Engagement Modeling Across Surfaces [Pinterest] · 9 min · Episode 161In this episode, we discuss how Pinterest moved from three independently evolving ads engagement models toward a unified modeling framework. The team started with a simple merged…
- Predicting Rider Conversion with Sparse Data Using Bayesian Trees [Lyft] · 8 min · Episode 160In this episode, we talk about Lyft’s need to predict whether riders will convert across a wide range of highly specific situations, where the available data becomes increasingly…
- Budget Allocation at Scale [Atlassian] · 9 min · Episode 159In this episode, we discuss the budget allocation problem faced by Atlassian when trying to optimize its marketing investment decisions across many channels, products, and…
- Semantic Search for LinkedIn’s Hiring Assistant [LinkedIn] · 9 min · Episode 158In this episode, we discuss how LinkedIn moves beyond keyword matching to find candidates based on actual qualification fit. The team first created a scalable relevance…
- Measuring AI Infrastructure Impact with Causal Inference [Meta] · 9 min · Episode 157In this episode, we discuss how Meta measures the business value of shared ML infrastructure when that value appears indirectly through downstream model launches. Instead of…
- Building Taxonomies from Unstructured Text Using LLMs [Microsoft] · 9 min · Episode 156In this episode, we discuss two LLM-assisted pipelines for turning unstructured text into a structured taxonomy with minimal manual annotation. The bottom-up approach lets…
- Building Semantic IDs for Product Understanding at Scale [Instacart] · 9 min · Episode 155In this episode, we talk about how Instacart identified a limitation of a standard taxonomy: it could classify products, but it couldn't connect them the way real shoppers do…
- Agent-driven ML Exploration [Faire] · 11 min · Episode 154In this episode, we discuss how Faire addresses a fundamental bottleneck in machine learning development: there are more potential experiments than human scientists can…
- Building food metadata with LLM juries [DoorDash] · 10 min · Episode 153In this episode, we discuss how DoorDash built a reliable metadata generation platform that could operate across millions of constantly changing menu items. The solution combined…
- Scaling subscriptions with causal machine learning [NYTimes] · 11 min · Episode 152In this episode, we explore how The New York Times built a real-time decision system that estimates the causal impact of different paywall actions, optimizes those decisions…
- Recommending Travel Destinations [Airbnb] · 10 min · Episode 151In this episode, we explore how Airbnb built a destination recommendation model to predict the places users are most likely to travel to next. The model predicts travel intent…
- Building AI Second Brain for Knowledge Workers [Meta] · 10 min · Episode 150In this episode, we discuss how Meta built an AI system with persistent memory, structured organization, secure access to enterprise tools, and reusable workflows. The result was…
- Scaling Experimentation Quality [Booking.com] · 10 min · Episode 149In this episode, we explore how Booking.com scaled experimentation quality across the organization. Instead of relying on strict enforcement, the team chose to optimize for…
- Evaluating Show Synopses with LLM-as-a-Judge [Netflix] · 12 min · Episode 148In this episode, we discuss how Netflix built an LLM-based evaluation framework for creative writing quality without losing the nuance that makes expert review valuable. They…
- Improved Candidate Generation with a Two-Tower Model [Glassdoor] · 10 min · Episode 147In this episode, we explore how Glassdoor customized the Two-Tower model for its specific production environment. The gap between generic semantic embeddings and behavior-driven…
- Measuring Rider Experience Impact with Causal Inference [Lyft] · 11 min · Episode 146In this episode, we talk about why Lyft needed to understand which rider experiences drive long-term retention. We discuss how raw observational data can produce misleading…
- Unified Ranking for Ads and Organic Recommendations [Flipkart] · 9 min · Episode 145In this episode, we explore how Flipkart rethought ranking in its "Similar Products" widget. Instead of relying on fixed ad slots, the team built a unified ranking system that…
- Re-architecting Serving Stack for Ads Ranking Models [Pinterest] · 9 min · Episode 144In this episode, we explore how Pinterest rethought its ads serving infrastructure to support more expressive ranking models beyond the traditional two-tower architecture. Their…







