Towards an evaluation protocol for RDF semantic annotations (RDF SemAnnotEval protocol)

Abstract

Semantic annotation in semantic web is a formal representation of the content related to the information resource. Generated annotations can be in RDF language, expressed by using concepts, relations and instances. Although, some evaluation, attempts of the quality of RDF annotations, were inspired from information retrieval domain. In fact most of studies use Recall, Precision and F-measure as evaluation measures. The aim of this paper is to propose an evaluation protocol for RDF semantic annotations to find a new measure that can help to rank annotations by combining the evaluation of instances, concepts and relations correctness.