57. Eldar Kurtic - Efficient Inference through sparsity and quantization - Part 2/2

Hello and welcome back to the AAIP

This is the second part of my interview with Eldar Kurtic and his research on how to optimiz inference of deep neural networks.

In the first part of the interview, we focused on sparsity and how high unstructured sparsity can be achieved without loosing model accuracy on CPU's and in part on GPU's.

In this second part of the interview, we are going to focus on quantization. Quantization tries to reduce model size by finding ways to represent the model in numeric representations with less precision while retaining model performance. This means that a…