Semi-Analytical Solution for Rate Distortion Function and OPTAfor Sources with Arbitrary Distribution

Abstract:
The rate distortion function is a widely used theoretical bound which describes the minimum mean square error (MMSE) distortion for a given number of quantization bits when quantizing a scalar random variable. An analytical solution for this function is only available for a small number of probability density functions (pdf), such as the Gaussian pdf. For arbitrary pdfs, the Blahut-Arimoto algorithm needs to be applied to iteratively estimate the rate distortion function. We propose a novel (semi-)analytical and non-iterative method to calculate the rate distortion function for sources with arbitrary pdfs. This method is based on the Guo–Shamai–Verdú (GSV) theorem. Furthermore, it is possible to apply the proposed method for calculating the Optimum Performance Theoretically Attainable (OPTA) for arbitrarily distributed input symbols observed through an AWGN channel.