Squeezing AI into your Pocket

Send us Fan Mail By 2026, language models have moved off the cloud and onto the device in your pocket. What was a research demonstration two years ago is now a routine engineering capability, and the centre of gravity for artificial intelligence has begun to migrate from distant data centres to local silicon. The episode traces the four engineering moves that made this possible. Quantization, which shrinks a model by storing its parameters with less precision. Optimized key-value caches, whic...