Pre-Experiments on Annotation of Russian Coreference Corpus

Building benchmark corpora in the domain of coreference and anaphora resolution is an important task for developing and evaluating NLP systems and models. Our study is aimed at assessing the feasibility of enhancing corpora with information about coreference relations. The annotation procedure includes identification of text segments that are subject to annotation (markables), marking their syntactic heads and identifying coreferential links. Markables are classified according to their morphological, syntactic and reference structure. The annotation is performed manually, providing gold standard data for high-level NLP tasks such as anaphora and coreference resolution. The paper reports on inconsistencies in selecting NPs of various types as markables and their borders, and in ways of constructing anaphoric pairs. We consider the types of NPs missed by annotators, and the discourse and semantic factors that may have affected the annotators’ judgements