Human and machine error analysis on dependency parsing of ancient Greek texts

Abstract

Automatically generated metadata from large collections is an essential component of digital libraries. It is beginning to emerge as fundamental to the study of languages. Morphosyntactic annotation captures the form of individual words and their function. Nonetheless automated syntactic analysis is still imperfect and human annotators can be significantly more accurate. On the other hand, human work is expensive and even humans find some constructions difficult to annotate correctly. Comparing the performance of human annotators with that of an automatic parser is thus important for exploring how the two methods can best be combined. In the present study, we compare the frequency of the different types of errors made by student annotators with those made by different dependency parsers when annotating ancient Greek. With a few exceptions, the frequency of the different types of errors was similar for human and machine. The significance of these results is briefly discussed.