A Parametric Information Bottleneck Algorithm for Gaussian Random Variables and Gaussian Mixtures

Konferenz: SCC 2019 - 12th International ITG Conference on Systems, Communications and Coding
11.02.2019 - 14.02.2019 in Rostock, Germany


Tagungsband: SCC 2019

Seiten: 6Sprache: EnglischTyp: PDF

Persönliche VDE-Mitglieder erhalten auf diesen Artikel 10% Rabatt

Stark, Maximilian; Lewandowsky, Jan; Bauch, Gerhard (Hamburg University of Technology, Institute of Communications, 21073 Hamburg, Germany)

Recently, the information bottleneck method, a machine learning framework, was incorporated in several communication engineering related applications. However, most of these applications are limited to discrete relevant random variables. This is mostly due to the lack of appropriate deterministic information bottleneck algorithms suitable for continuous random variables. In this paper, we present a novel deterministic information bottleneck algorithm, which we call the parametric information bottleneck algorithm, suitable for continuous relevant variables with a Gaussian distribution. We show that our proposed algorithm operates close to the theoretically achievable Gaussian information bottleneck bound. In addition, our proposed algorithm allows to efficiently compress any continuous random variable whose distribution can be approximated by a Gaussian mixture distribution. Exemplarily, using the proposed parametric information bottleneck algorithm, we devise a relevant-information- preserving temperature sensor. Although the resolution of the sensor’s analog-to-digital converter is only 5 bit, the proposed information bottleneck algorithm finds quantization regions such that 99.9% relevant information is preserved.