Overview and Investigation of Algorithms for the Information Bottleneck Method

Conference: SCC 2017 - 11th International ITG Conference on Systems, Communications and Coding
02/06/2017 - 02/09/2017 at Hamburg, Germany

Proceedings: ITG-Fb. 268: SCC 2017

Pages: 6Language: englishTyp: PDF

Personal VDE Members are entitled to a 10% discount on this title

Hassanpour, Shayan; Wuebben, Dirk; Dekorsy, Armin (Department of Communications Engineering, University of Bremen, 28359 Bremen, Germany)

Lossy data compression has been studied under the celebrated Rate-Distortion theory which provides the compression rate in order to quantize a signal without exceeding a given distortion measure. Recently, with information bottleneck an alternative approach has been emerged in the field of machine learning. The fundamental idea is to include the original source into the problem setup when quantizing an observation variable and to use strictly information theoretic measures to design the quantizer. This paper yields an insight to this framework, discusses corresponding algorithms and their performance, and provides a new algorithmic approach of low complexity.