Information Retrieval Journal最新文献_第3页

An in-depth analysis of passage-level label transfer for contextual document ranking 深入分析用于上下文文档排序的段落级标签转移

IF 2.5 3区计算机科学

Information Retrieval Journal Pub Date : 2023-12-08 DOI: 10.1007/s10791-023-09430-5

Koustav Rudra, Zeon Trevor Fernando, Avishek Anand

{"title":"An in-depth analysis of passage-level label transfer for contextual document ranking","authors":"Koustav Rudra, Zeon Trevor Fernando, Avishek Anand","doi":"10.1007/s10791-023-09430-5","DOIUrl":"https://doi.org/10.1007/s10791-023-09430-5","url":null,"abstract":"<p>Pre-trained contextual language models such as BERT, GPT, and XLnet work quite well for document retrieval tasks. Such models are fine-tuned based on the query-document/query-passage level relevance labels to capture the ranking signals. However, the documents are longer than the passages and such document ranking models suffer from the token limitation (512) of BERT. Researchers proposed ranking strategies that either truncate the documents beyond the token limit or chunk the documents into units that can fit into the BERT. In the later case, the relevance labels are either directly transferred from the original query-document pair or learned through some external model. In this paper, we conduct a detailed study of the design decisions about splitting and label transfer on retrieval effectiveness and efficiency. We find that direct transfer of relevance labels from documents to passages introduces <i>label noise</i> that strongly affects retrieval effectiveness for large training datasets. We also find that query processing times are adversely affected by fine-grained splitting schemes. As a remedy, we propose a careful passage level labelling scheme using weak supervision that delivers improved performance (3–14% in terms of nDCG score) over most of the recently proposed models for ad-hoc retrieval while maintaining manageable computational complexity on four diverse document retrieval datasets.</p>","PeriodicalId":54352,"journal":{"name":"Information Retrieval Journal","volume":"64 1","pages":""},"PeriodicalIF":2.5,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138556906","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Privacy-aware document retrieval with two-level inverted indexing 具有两级倒排索引的隐私感知文档检索

IF 2.5 3区计算机科学

Information Retrieval Journal Pub Date : 2023-11-17 DOI: 10.1007/s10791-023-09428-z

Yifan Qiao, Shiyu Ji, Changhai Wang, Jinjin Shao, Tao Yang

引用次数: 0

Heterogeneous graph attention networks for passage retrieval 文章检索的异构图注意网络

IF 2.5 3区计算机科学

Information Retrieval Journal Pub Date : 2023-11-16 DOI: 10.1007/s10791-023-09424-3

Lucas Albarede, Philippe Mulhem, Lorraine Goeuriot, Sylvain Marié, Claude Le Pape-Gardeux, Trinidad Chardin-Segui

引用次数: 0

Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems 构建和元评估交互式搜索系统的状态感知评估指标

3区计算机科学

Information Retrieval Journal Pub Date : 2023-10-31 DOI: 10.1007/s10791-023-09426-1

Marco Markwald, Jiqun Liu, Ran Yu

{"title":"Constructing and meta-evaluating state-aware evaluation metrics for interactive search systems","authors":"Marco Markwald, Jiqun Liu, Ran Yu","doi":"10.1007/s10791-023-09426-1","DOIUrl":"https://doi.org/10.1007/s10791-023-09426-1","url":null,"abstract":"Abstract Evaluation metrics such as precision, recall and normalized discounted cumulative gain have been widely applied in ad hoc retrieval experiments. They have facilitated the assessment of system performance in various topics over the past decade. However, the effectiveness of such metrics in capturing users’ in-situ search experience, especially in complex search tasks that trigger interactive search sessions, is limited. To address this challenge, it is necessary to adaptively adjust the evaluation strategies of search systems to better respond to users’ changing information needs and evaluation criteria. In this work, we adopt a taxonomy of search task states that a user goes through in different scenarios and moments of search sessions, and perform a meta-evaluation of existing metrics to better understand their effectiveness in measuring user satisfaction. We then built models for predicting task states behind queries based on in-session signals. Furthermore, we constructed and meta-evaluated new state-aware evaluation metrics. Our analysis and experimental evaluation are performed on two datasets collected from a field study and a laboratory study, respectively. Results demonstrate that the effectiveness of individual evaluation metrics varies across task states. Meanwhile, task states can be detected from in-session signals. Our new state-aware evaluation metrics could better reflect in-situ user satisfaction than an extensive list of the widely used measures we analyzed in this work in certain states. Findings of our research can inspire the design and meta-evaluation of user-centered adaptive evaluation metrics, and also shed light on the development of state-aware interactive search systems.","PeriodicalId":54352,"journal":{"name":"Information Retrieval Journal","volume":"2013 24","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135813980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

DeepQFM: a deep learning based query facets mining method DeepQFM:基于深度学习的查询面挖掘方法

3区计算机科学

Information Retrieval Journal Pub Date : 2023-10-30 DOI: 10.1007/s10791-023-09427-0

Zhirui Deng, Zhicheng Dou, Ji-Rong Wen

引用次数: 0

Learning heterogeneous subgraph representations for team discovery 学习异构子图表示用于团队发现

3区计算机科学

Information Retrieval Journal Pub Date : 2023-10-09 DOI: 10.1007/s10791-023-09421-6

Radin Hamidi Rad, Hoang Nguyen, Feras Al-Obeidat, Ebrahim Bagheri, Mehdi Kargar, Divesh Srivastava, Jaroslaw Szlichta, Fattane Zarrinkalam

引用次数: 0

Investigating better context representations for generative question answering 为生成式问答研究更好的上下文表示

3区计算机科学

Information Retrieval Journal Pub Date : 2023-10-02 DOI: 10.1007/s10791-023-09420-7

Sumam Francis, Marie-Francine Moens

引用次数: 0

Multimodal video retrieval with CLIP: a user study 多模式视频检索与剪辑:一个用户研究

3区计算机科学

Information Retrieval Journal Pub Date : 2023-09-29 DOI: 10.1007/s10791-023-09425-2

Tayfun Alpay, Sven Magg, Philipp Broze, Daniel Speck

{"title":"Multimodal video retrieval with CLIP: a user study","authors":"Tayfun Alpay, Sven Magg, Philipp Broze, Daniel Speck","doi":"10.1007/s10791-023-09425-2","DOIUrl":"https://doi.org/10.1007/s10791-023-09425-2","url":null,"abstract":"Abstract Recent machine learning advances demonstrate the effectiveness of zero-shot models trained on large amounts of data collected from the internet. Among these, CLIP (Contrastive Language-Image Pre-training) has been introduced as a multimodal model with high accuracy on a number of different tasks and domains. However, the unconstrained nature of the model begs the question whether it can be deployed in open-domain real-word applications effectively in front of non-technical users. In this paper, we evaluate whether CLIP can be used for multimodal video retrieval in a real-world environment. For this purpose, we implemented impa , an efficient shot-based retrieval system powered by CLIP. We additionally implemented advanced query functionality in a unified graphical user interface to facilitate an intuitive and efficient usage of CLIP for video retrieval tasks. Finally, we empirically evaluated our retrieval system by performing a user study with video editing professionals and journalists working in the TV news media industry. After having the participants solve open-domain video retrieval tasks, we collected data via questionnaires, interviews, and UI interaction logs. Our evaluation focused on the perceived intuitiveness of retrieval using natural language, retrieval accuracy, and how users interacted with the system’s UI. We found that our advanced features yield higher task accuracy, user ratings, and more efficient queries. Overall, our results show the importance of designing intuitive and efficient user interfaces to be able to deploy large models such as CLIP effectively in real-world scenarios.","PeriodicalId":54352,"journal":{"name":"Information Retrieval Journal","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135194990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MuMUR: Multilingual Multimodal Universal Retrieval MuMUR:多语言多模态通用检索

3区计算机科学

Information Retrieval Journal Pub Date : 2023-09-25 DOI: 10.1007/s10791-023-09422-5

Avinash Madasu, Estelle Aflalo, Gabriela Ben Melech Stan, Shachar Rosenman, Shao-Yen Tseng, Gedas Bertasius, Vasudev Lal

引用次数: 0

Temporal information retrieval using bitwise operators 使用位运算符的时态信息检索

3区计算机科学

Information Retrieval Journal Pub Date : 2023-09-23 DOI: 10.1007/s10791-023-09423-4

Prasanna Koirala, Ramazan Aygun, Tathagata Mukherjee, Haeyong Chung

引用次数: 0