A Paired-Comparison Listening Test for Collecting Voice Likability Scores

Abstract:
The research on acoustic correlates and on the automatic classification of voice likability commonly faces the undesirable low agreement between human raters. This may partly hinder the good performance of automatic likability detection techniques. Whereas only Likert scales have been employed for subjective likability assessments of utterances, this paper presents a paired-comparison listening test for obtaining the likability scores. Consistency checks and the Bradley-Terry-Luce probabilistic choice model are applied to the data in order to derive a meaningful ordering of voices conveying the listeners’ preferences. Our focus is to examine the applicability and reliability of this test paradigm and whether the agreement between listeners can be enhanced relative to a past listening test based on direct scaling, performed by the same listeners and using the same speech material.