Information Retrieval Evaluation Measures Defined on Some Axiomatic Models of Preferences

IF 5.4 2区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

ACM Transactions on Information Systems Pub Date : 2023-11-08 DOI:10.1145/3632171

Fernando Giner

{"title":"Information Retrieval Evaluation Measures Defined on Some Axiomatic Models of Preferences","authors":"Fernando Giner","doi":"10.1145/3632171","DOIUrl":null,"url":null,"abstract":"Information retrieval (IR) evaluation measures are essential for capturing the relevance of documents to topics, and determining the task performance efficiency of retrieval systems. The study of IR evaluation measures through their formal properties enables a better understanding of their suitability for a specific task. Some works have modelled the effectiveness of retrieval measures with axioms, heuristics or desirable properties, leading to order relationships on the set where they are defined. Each of these ordering structures constitute an axiomatic model of preferences (AMP), which can be considered as an ’ideal’ scenario of retrieval. Based on lattice theory and on the representational theory of measurement, this work formally explores numeric, metric and scale properties of some effectiveness measures defined on AMPs. In some of these scenarios, retrieval measures are completely determined from the scores of a subset of document rankings: join-irreducible elements. All the possible metrics and pseudometrics, defined on these structures are expressed in terms of the join-irreducible elements. The deduced scale properties of the precision, recall, F -measure, RBP , DCG and AP confirm some recent results in the IR field.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":5.4000,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Transactions on Information Systems","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3632171","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Information retrieval (IR) evaluation measures are essential for capturing the relevance of documents to topics, and determining the task performance efficiency of retrieval systems. The study of IR evaluation measures through their formal properties enables a better understanding of their suitability for a specific task. Some works have modelled the effectiveness of retrieval measures with axioms, heuristics or desirable properties, leading to order relationships on the set where they are defined. Each of these ordering structures constitute an axiomatic model of preferences (AMP), which can be considered as an ’ideal’ scenario of retrieval. Based on lattice theory and on the representational theory of measurement, this work formally explores numeric, metric and scale properties of some effectiveness measures defined on AMPs. In some of these scenarios, retrieval measures are completely determined from the scores of a subset of document rankings: join-irreducible elements. All the possible metrics and pseudometrics, defined on these structures are expressed in terms of the join-irreducible elements. The deduced scale properties of the precision, recall, F -measure, RBP , DCG and AP confirm some recent results in the IR field.

查看原文本刊更多论文

基于公理偏好模型的信息检索评价方法

信息检索评价指标是获取文档与主题的相关性，决定检索系统任务执行效率的关键。通过形式属性对IR评价措施进行研究，可以更好地理解它们对特定任务的适用性。一些作品用公理、启发式或理想属性来模拟检索措施的有效性，从而在定义检索措施的集合上建立有序关系。这些排序结构中的每一个都构成了一个公理偏好模型(AMP)，这可以被认为是检索的“理想”场景。基于晶格理论和测量的表征理论，本文正式探讨了在amp上定义的一些有效性度量的数值、度量和尺度性质。在其中一些场景中，检索方法完全由文档排名子集的分数决定:连接不可约元素。在这些结构上定义的所有可能度量和伪度量都用连接不可约元素表示。通过对精度、召回率、F -测度、RBP、DCG和AP等指标的推导，证实了近年来红外领域的一些研究成果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

ACM Transactions on Information Systems 工程技术-计算机：信息系统

CiteScore

9.40

自引率

14.30%

发文量

165

审稿时长

>12 weeks

期刊介绍： The ACM Transactions on Information Systems (TOIS) publishes papers on information retrieval (such as search engines, recommender systems) that contain: new principled information retrieval models or algorithms with sound empirical validation; observational, experimental and/or theoretical studies yielding new insights into information retrieval or information seeking; accounts of applications of existing information retrieval techniques that shed light on the strengths and weaknesses of the techniques; formalization of new information retrieval or information seeking tasks and of methods for evaluating the performance on those tasks; development of content (text, image, speech, video, etc) analysis methods to support information retrieval and information seeking; development of computational models of user information preferences and interaction behaviors; creation and analysis of evaluation methodologies for information retrieval and information seeking; or surveys of existing work that propose a significant synthesis. The information retrieval scope of ACM Transactions on Information Systems (TOIS) appeals to industry practitioners for its wealth of creative ideas, and to academic researchers for its descriptions of their colleagues'' work.