Benchmarking TensorFlow Lite Quantization Algorithms for Deep Neural Networks

Abstract

Deploying deep neural network models on the resource constrained devices, e.g., lost-cost microcontrollers, is challenging because they are mostly limited in terms of memory footprint and computation capabilities. Quantization is one of the widely used solutions to reduce the size of a model. For parameter representation, it employs for example just 8-bit integer or less instead of 32-bit floating point. The TensorFlow Lite deep learning framework currently provides four methods for post-training quantization. The aim of this paper is to benchmark these quantization methods using various deep neural models of different sizes. The main outcomes of the paper are: (1) the compression ratio obtained for each quantization method for deep neural models of small, medium, and large sizes, (2) a comparison of the accuracy results relative to the original accuracy, and (3) a viewpoint for the decision to choose the quantization method depending on the model size.