Evaluating the Impact of Crowdsourced Audio Data on Speech Quality Assessment

Abstract:
Speech quality assessment is a well-established area of study, such tests are standardized and typically rely on speech data that has been recorded in a studio environment and artificially degraded afterward. Recently, an alternative approach to dataset collection via a crowdsourcing environment was introduced. In this study, we investigate the effect of speech data source on perceived quality judgements. Two auditory experiments were conducted under laboratory conditions, in which identical degradations were introduced to two datasets of different origins and rated on both overall quality and perceptual dimensions. Instrumental analysis revealed measurable differences between the two datasets, while analysis of listener ratings showed similar perceptual patterns and found no statistically significant interaction between data source and degradation conditions. These findings suggest that listeners evaluate speech quality similarly, regardless of the data origin — crowdsourced or studio.