Quantization and fixed-point arithmetic
The dynamic range of fixed-point numbers is much less than floating-point numbers with equivalent word sizes. When working with fixed-point data types, you must carefully consider the effects of quantization on precision, range, and scaling. A fixed-point data type is characterized by the word length in bits, the position of the binary point, and whether it is signed or unsigned. The position of the binary point is the means by which fixed-point values are scaled and interpreted.
Precision, range, and scaling of fixed-point data types
- Fixed-Point Arithmetic
Fixed-point arithmetic operations in the software, and effects of data type and scaling