A Task-Based Approach for Large-Scale Evaluation of the Gene Ontology

The Gene Ontology (GO) provides a framework to systematically classify and annotate gene function. The annotations associated with GO play a critical role in modern biology and cover many organisms. For the human genome, over 10,000 GO terms are used to annotate gene function in an expansive database of over 200,000 annotations. Due to the importance of the GO annotations in modern biology, significant effort has been put into assessing the quality of the annotations. Providing measures of annotation completeness, accuracy, and precision is critical if researchers are to use the annotations in real-world applications with confidence. Here, we describe a task-based approach that examines the completeness and utility of GO annotations through the lens of gene enrichment analysis. Our approach can be used to model the progression of the GO annotations over time, either for a particular area of interest or for the body of annotations as a whole. Using this framework, we conducted a large-scale analysis of gene expression datasets from the NCBI Gene Expression Omnibus (GEO). In particular, we identified terms of interest for each dataset through semantic annotation of biomedical data, then tracked the behavior of these terms as a function of time. The preliminary results provide significant information about the progress and character of GO annotations over time. This framework is flexible enough to examine all or part of the GO annotations, across multiple species, and with various enrichment methods. We also discuss how this framework can be used to evaluate different annotation methods. For example, by comparing the performance of annotations generated with a particular method to the performance of canonical annotations, it is possible to determine their relative quality.