Muon-Optimized Distillation and Quantization for LLM Deployment

In this episode:
• Introduction & The Edge Deployment Problem: Professor Norris and Linda introduce the episode's paper, focusing on the difficulty of running large language models on edge devices with limited memory and compute.
• The Multi-Teacher Distillation Pipeline: Linda explains the dual-teacher approach, using Llama 4 Scout for synthetic data generation and Llama 3.3 70B for logit-based knowledge distillation to a 3B parameter student.
• Hyperparameter Surprises with Optuna: The hosts discuss the surprising finding that Bayesian optimization consistently drops the cross-entropy loss…