A Parametric Information Bottleneck Algorithm for Gaussian Random Variables and Gaussian Mixtures
Konferenz: SCC 2019 - 12th International ITG Conference on Systems, Communications and Coding
11.02.2019 - 14.02.2019 in Rostock, Germany
Tagungsband: SCC 2019
Seiten: 6Sprache: EnglischTyp: PDFPersönliche VDE-Mitglieder erhalten auf diesen Artikel 10% Rabatt
Stark, Maximilian; Lewandowsky, Jan; Bauch, Gerhard (Hamburg University of Technology, Institute of Communications, 21073 Hamburg, Germany)
Recently, the information bottleneck method, a machine learning framework, was incorporated in several communication engineering related applications. However, most of these applications are limited to discrete relevant random variables. This is mostly due to the lack of appropriate deterministic information bottleneck algorithms suitable for continuous random variables. In this paper, we present a novel deterministic information bottleneck algorithm, which we call the parametric information bottleneck algorithm, suitable for continuous relevant variables with a Gaussian distribution. We show that our proposed algorithm operates close to the theoretically achievable Gaussian information bottleneck bound. In addition, our proposed algorithm allows to efficiently compress any continuous random variable whose distribution can be approximated by a Gaussian mixture distribution. Exemplarily, using the proposed parametric information bottleneck algorithm, we devise a relevant-information- preserving temperature sensor. Although the resolution of the sensor’s analog-to-digital converter is only 5 bit, the proposed information bottleneck algorithm finds quantization regions such that 99.9% relevant information is preserved.