MSE Analysis of 24-Bit Fixed-Point Format in a Wide Variance Range

Abstract

In this paper, we consider a fixed-point format which is one of the commonly used binary format for data representation. In particular, we focus on the 24-bit fixed-point format (FXP24) and the analysis provided here can be observed as our further research in this area. Using the relationship that exists between the structure of the fixed-point format and uniform scalar quantization, we actually use SQNR (signal-to-quantization noise ratio) to measure the performance of mentioned binary format. Assuming data modelled with a Laplacian distribution, we theoretically investigate the performance of the FXP24 format across the wide dynamic range of variances under the constraint that the number of bits in the integer part $\boldsymbol{n}$ changes. We have shown that with an adequate selection of $\boldsymbol{n}$, it is possible to find the range of variances in which FXP24 provides better SQNR (that is, a higher accuracy) than its floating-point counterpart FP24.