Differences in syntactic annotation affect retrieval

Abstract

Prepositional phrases (PPs) play an important part in English argument structure constructions, but pose considerable challenges for linguistic investigations of any kind. In addition to the fact that PP-attachment is generally notoriously difficult to model computationally, a particularly striking methodological challenge in investigating verb-dependent PPs across (synchronic and/or diachronic) corpora is that such cross-corpus studies may have to rely on material annotated with different tools. This study evaluates the impact that such differences in corpus annotation may have on retrieval of verb-attached PPs by means of data from Early and Late Modern English corpora. Our intrinsic (recall/precision) and extrinsic parser evaluation shows that annotation does play a role, but that the noise introduced is negligible as far as frequency developments are concerned.