On the Relationship Between the KL Means Algorithm and the Information Bottleneck Method

Abstract:
The problem of finding the mutual informationmaximizing quantizer of a discrete memoryless channel is important in the implementation of communication receivers, LDPC code decoders, and in the design of polar codes. Two algorithms algorithms that provide suboptimal solutions in polynomial time are the information bottleneck method and the KL means algorithm. The contribution of this paper is show that the information bottleneck method with β → ∞ is algorithmically equivalent to the KL means algorithm. This is done by showing that both the DMC channel outputs, and the quantizer outputs, are in the same J-dimensional space, where J is the cardinality of the input alphabet.