A Multi-layered Annotation Scheme and Computational Model for Co-Learning Semantic and Prosodic Structures of Chinese Discourse

This paper presents a novel annotation scheme of Chinese discourse structures to model the complex interactions among grammar, semantics and phonology. The scheme mainly contains three layers, i.e., grammatical, semantic and prosodic layers. Within each layer, the representations of dependency relations, rhetorical structure, information structure, topic chain, prosodic boundaries and stress distributions are specified. Based on the scheme, a large scale corpus of transcribed speech data is constructed and annotated. We further propose a machine learning methodology to learn from the annotated corpus a computational representation of the internal structure of each layer and the interactions across different layers. Specifically, we employ the Recursive Neural Network (RNN) to model the fine-grained structure in natural language information, through learning a distributed representation of the structural units. The proposed annotation scheme and machine learning methodology to expected to underpin more effective and intelligent speech engineering and understanding technologies of the future.