Show notes
Training modern AI systems is increasingly expensive, yet many of the choices that determine this cost still depend on extensive empirical search. This talk will review how combining ideas from theoretical physics with targeted experiments make large-scale training more predictable, transferable, and efficient. Cengiz Pehlevan will begin by examining how parameterization and hyperparameter choices shape optimization dynamics. Pehlevan will then show how this understanding can guide the design of training schedules, emerging optimizers, and architectures. Throughout, Pehlevan will highlight…

