On the Relationship Between the KL Means Algorithm and the Information Bottleneck Method

Conference: SCC 2017 - 11th International ITG Conference on Systems, Communications and Coding
02/06/2017 - 02/09/2017 at Hamburg, Germany

Proceedings: ITG-Fb. 268: SCC 2017

Pages: 6Language: englishTyp: PDF

Personal VDE Members are entitled to a 10% discount on this title

Authors:
Kurkoski, Brian M. (School of Information Science, Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, 923-1292, Japan)

Abstract:
The problem of finding the mutual informationmaximizing quantizer of a discrete memoryless channel is important in the implementation of communication receivers, LDPC code decoders, and in the design of polar codes. Two algorithms algorithms that provide suboptimal solutions in polynomial time are the information bottleneck method and the KL means algorithm. The contribution of this paper is show that the information bottleneck method with β → ∞ is algorithmically equivalent to the KL means algorithm. This is done by showing that both the DMC channel outputs, and the quantizer outputs, are in the same J-dimensional space, where J is the cardinality of the input alphabet.