How prosody affects ASR performance in conversational Austrian German

Currently available Automatic Speech Recognition (ASR) systems achieve good word error rates (WER) for read speech ( 2 − 10% ), but not for conversational speech ( 20 − 40% ), a speaking style especially relevant for dialogue systems, as they become more conversational and interactional. Here, we anal-yse how prosody affects WER in a Kaldi-based speech recognition system for a corpus of conversational Austrian German. This analysis is a step towards improving ASR systems and increasing our knowledge about which aspects are relevant to consider for ASR of conversational speech. For this purpose, we compare a typical language model (LM) with an oracle LM trained on the utterances from the whole corpus, thus knowing each possible N -gram in advance. We find that short, deaccented words have the lowest recognition accuracy, which also cannot be compensated for by the oracle LM. Despite our over-all high WERs, the highly prominent words were recognised significantly better. Our findings suggest that reporting global WERs for an ASR system of conversational speech does not predict its usefulness in dialogue systems. Given the role of prominent words in carrying meaning and function in conver-sation, our analysis is relevant for researchers developing automatic speech understanding systems.