56. Eldar Kurtic - Efficient Inference through sparsity and quantization - Part 1/2

Hello and welcome back to the AAIP

If you are an active Machine Learning engineer or are simply interested in Large Language models, I am sure you have seen the discussions around quantized models and all kind of new frameworks that have appeared recently and achieve astonishing inference performance of LLM's on consumer devices.

If you are curious how modern Large Language Models with their billions of parameters can run on a simple laptop or even an embedded device, than this episode is for you.

Today I am talking to Eldar Kurtic, researcher in the Alistarh group at the IST in lower…