Linear Digressions
tekijä: Katie Malone
Uusin jakso
GPU Architecture for LLM Inference: Memory, Bandwidth and Hardware…
GPU Architecture for LLM Inference: Memory, Bandwidth and Hardware…
Tietoa podcastista
In each episode, your hosts explore machine learning and data science through interesting (and often very unusual) applications. 896520
Jaksot
Uusimmat 50 jaksoa 332 jaksosta
- GPU Architecture for LLM Inference: Memory, Bandwidth and Hardware Explained · 20 minGPUs are the engines powering every AI conversation you've ever had — but what's actually happening inside those chips when an LLM generates a response? Understanding GPU…
- KV Cache Fundamentals · 24 minThe unsung hero of every LLM conversation you've ever had? A clever optimization called the KV cache. Self-attention is computationally brutal — every token attending to every…
- AI Inference 101 · 28 minEveryone talks about training the model — the data, the compute, the alignment magic. But what happens after the weights are frozen and millions of users start hammering it with…
- Mechanistic Interpretability: How Researchers Try To See Inside Models · 29 minAs AI systems grow more powerful, they grow harder to understand — and that tradeoff has never mattered more. Most interpretability tricks give us approximations from the…
- LLM As A Judge · 30 minLabeled data is expensive, slow, and painfully hard to come by — and when you're trying to evaluate whether an AI workflow is actually doing what you want, the problem gets even…
- The Impact of AI on Podcasting (Harvard Data Science Review Cross-Post) · 30 minOriginally aired on the Harvard Data Science Review podcast. What can podcasting teach us about AI — and what can AI teach us about the future of podcasting? Katie Malone…
- Better Know A Benchmark: ExploitGym · 33 minWhen OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were hunting for answer keys. The real story is stranger and more unsettling…
- Constitutional AI · 32 minHow do you teach a model the difference between helpful and harmful when it has no inherent sense of either? This episode dives into Constitutional AI, Anthropic's framework for…
- A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College) · 40 minTom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a reality check on AI. As one of the most seasoned observers of how…
- Understanding AI Text Watermarking · 30 minAnthropic just announced they're baking invisible watermarks directly into Claude's generated text — and while everyone else was busy having opinions about it, we were busy…
- Better Know a Benchmark: Humanity's Last Exam · 23 minHumanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI models can't. Originally dubbed "Humanity's Last Stand," this benchmark is…
- A Scientific Deep Dive into Overconfident LLMs: Interview with Kaitlyn Zhou (Cornell) · 34 minWhen a language model tells you it's absolutely certain, is it actually more likely to be right? Kaitlyn Zhou's research says: not necessarily — sometimes confident phrasing…
- Reasoning Models: When LLMs Went Beyond Fancy Autocomplete · 25 minReasoning models don't just answer your question — they *think out loud* first. In this episode we dig into the class of AI models that generate intermediate chains of thought…
- Distillation, or, How to Steal a Model · 24 minThis week we’re covering model distillation: the technique of using a large "teacher" model's outputs to train a smaller, cheaper "student" model that mimics it. They cover the…
- Invisible LLM Failures and AI Fluency with Chris Potts (Stanford) · 41 minWhat happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisingly invisible ways humans misunderstand them? Chris Potts joins the show…
- Interviewing the Linear Digressions Agents (The Agents Season, Episode 11) · 38 minAfter a five-year hiatus, the podcast that burned out partly over the tedium of writing episode descriptions is back — and using AI agents to handle exactly that task. The…
- Agent Economics (The Agents Season, Episode 10) · 24 minWhat if building more highways made your commute *slower*? That's the paradox at the heart of AI agent economics: even as per-token inference costs have plummeted dramatically…
- Agent Trust, Oversight and Control (The Agents Season, Episode 9) · 26 minCapabilities get all the attention when it comes to AI agents — but what happens when a highly capable agent makes a bad decision in the real world? Trust, oversight, and control…
- Many Agents, Many Problems (The Agents Season, Episode 8) · 28 minWhether you work best solo or thrive in a team, you know collaboration is complicated — and it turns out AI agents face the same tensions. This episode dives into multi-agent…
- How Do You Evaluate An AI Agent? (The Agents Season, Episode 7) · 32 minKnowing when an AI agent has failed sounds straightforward — until it isn't. Agents have a frustrating habit of finishing confidently while quietly doing the wrong thing, or…
- AI Agent Failure Modes (The Agents Season, Episode 6) · 33 minDespite what the marketing hype might suggest, AI agents are far from infallible — and if you've ever actually used one, you already know this. Today's episode dives deep into…
- Agentic Planning (The Agents Season, Episode 5) · 24 minWhen tackling a complex, multi-step task, even the smartest AI agent can fail without a solid game plan. This episode dives into the research around agentic planning — how agents…
- Memory Management for AI Agents (The Agents Season, Episode 4) · 25 minContext windows are powerful — but finite, and surprisingly easy to overwhelm. When an AI agent is tackling a long, complex task, the information it needs has to fit inside that…
- Lost in the Middle (The Agents Season, Episode 3) · 20 minJust like a memorable talk lives or dies by its opening and closing, LLMs have a surprisingly similar quirk: they pay close attention to what's at the beginning and end of their…
- ReAct and Tool Usage (The Agents Season, Episode 2) · 24 minBefore 2022, there was a wall between AI and the real world — models could reason impressively, but couldn't look anything up, run code, or check whether anything they said was…
- What's an AI Agent? And Why's That Hard to Define? (The Agents Season, Episode 1) · 19 minAI agents are having a moment — and unpacking them properly takes more than a single conversation. This episode kicks off a dedicated multi-part season exploring AI agents from…
- Unfaithful Chain of Thought · 25 minWhat's actually happening when an LLM "thinks out loud"? Research on human decision-making suggests that much of the reasoning we believe drives our choices is actually post hoc…
- Benchmark Bank Heist · 13 minWhat if an AI decided the smartest way to pass its test was to find the answer key? That's exactly what Anthropic's Claude Opus did when faced with a benchmark evaluation —…
- Benchmarking AI Models · 30 minHow do you know if a new AI model is actually better than the last one? It turns out answering that question is a lot messier than it sounds. This week we dig into the world of…
- The Hot Mess of AI (Mis-)Alignment · 23 minThe paperclip maximizer — the classic AI doom scenario where a hyper-competent machine single-mindedly converts the universe into office supplies — might not be the AI risk we…
- The Bitter Lesson · 19 minEvery AI builder knows the anxiety: you spend months engineering prompts, tuning pipelines, and chaining calls together — then a new model drops and half your work evaporates…
- From Atari to ChatGPT: How AI Learned to Follow Instructions · 26 minFrom Atari to ChatGPT: How AI Learned to Follow Instructions by Katie Malone
- It's RAG time: Retrieval-Augmented Generation · 17 minToday we are going to talk about the feature with the worst acronym in generative AI: RAG, or Retrieval Augmented Generation. If you've ever used something like "Chat with My…
- Chasing Away Repetitive LLM Responses with Verbalized Sampling · 19 minOne of the things that LLMs can be really helpful with is brainstorming or generating new creative content. They are called Generative AI, after all—not just for summarization…
- We're Back · 3 minIt's been (*checks watch*) about five and a half years since we last talked. Fortunately nothing much has happened in the AI/data science world in that time. So let's just pick…
- A Key Concept in AI Alignment: Deep Reinforcement Learning from Human Preferences · 19 minModern AI chatbots have a few different things that go into creating them. Today we're going to talk about a really important part of the process: the alignment training, where…
- The Impact of Generative AI on Critical Thinking · 26 minI use LLMs a lot. I use them in my work, I use them in my personal life, and sometimes I use them to help me with stuff that I already know how to do. I’m working on something…
- So long, and thanks for all the fish · 36 minAll good things must come to an end, including this podcast. This is the last episode we plan to release, and it doesn’t cover data science—it’s mostly reminiscing, thanking our…
- A Reality Check on AI-Driven Medical Assistants · 14 minThe data science and artificial intelligence community has made amazing strides in the past few years to algorithmically automate portions of the healthcare process. This episode…
- A Data Science Take on Open Policing Data · 24 minA few weeks ago, we put out a call for data scientists interested in issues of race and racism, or people studying how those topics can be studied with data science methods…
- Procella: YouTube's super-system for analytics data storage · 30 minThis is a re-release of an episode that originally ran in October 2019. If you’re trying to manage a project that serves up analytics data for a few very distinct uses, you’d be…
- The Data Science Open Source Ecosystem · 23 minOpen source software is ubiquitous throughout data science, and enables the work of nearly every data scientist in some way or another. Open source projects, however, are…
- Rock the ROC Curve · 16 minThis is a re-release of an episode that first ran on January 29, 2017. This week: everybody's favorite WWII-era classifier metric! But it's not just for winning wars, it's a…
- Criminology and Data Science · 31 minThis episode features Zach Drake, a working data scientist and PhD candidate in the Criminology, Law and Society program at George Mason University. Zach specializes in bringing…
- Racism, the criminal justice system, and data science · 32 minAs protests sweep across the United States in the wake of the killing of George Floyd by a Minneapolis police officer, we take a moment to dig into one of the ways that data…
- An interstitial word from Ben · 6 minA message from Ben around algorithmic bias, and how our models are sometimes reflections of ourselves.
