Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval最新文献_第4页

More Robust Dense Retrieval with Contrastive Dual Learning 基于对比双重学习的更稳健的密集检索

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472245

Yizhi Li, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu

{"title":"More Robust Dense Retrieval with Contrastive Dual Learning","authors":"Yizhi Li, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu","doi":"10.1145/3471158.3472245","DOIUrl":"https://doi.org/10.1145/3471158.3472245","url":null,"abstract":"Dense retrieval conducts text retrieval in the embedding space and has shown many advantages compared to sparse retrieval. Existing dense retrievers optimize representations of queries and documents with contrastive training and map them to the embedding space. The embedding space is optimized by aligning the matched query-document pairs and pushing the negative documents away from the query. However, in such training paradigm, the queries are only optimized to align to the documents and are coarsely positioned, leading to an anisotropic query embedding space. In this paper, we analyze the embedding space distributions and propose an effective training paradigm, Contrastive Dual Learning for Approximate Nearest Neighbor (DANCE) to learn fine-grained query representations for dense retrieval. DANCE incorporates an additional dual training object of query retrieval, inspired by the classic information retrieval training axiom, query likelihood. With contrastive learning, the dual training object of DANCE learns more tailored representations for queries and documents to keep the embedding space smooth and uniform, thriving on the ranking performance of DANCE on the MS MARCO document retrieval task. Different from ANCE that only optimized with the document retrieval task, DANCE concentrates the query embeddings closer to document representations while making the document distribution more discriminative. Such concentrated query embedding distribution assigns more uniform negative sampling probabilities to queries and helps to sufficiently optimize query representations in the query retrieval task. Our codes are released at https://github.com/thunlp/DANCE.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133735426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 21

Modality Effects When Simulating User Querying Tasks 模拟用户查询任务时的模态效果

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472244

J. Mackenzie, Alistair Moffat

引用次数: 2

Session details: Session 4A - Question Answering 会议详情:第4A部分-问答

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3484644

Debasis Ganguly

引用次数: 0

Session details: Session 1B - User Interaction 会议详情:会议1B -用户交互

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3484638

F. Crestani

引用次数: 0

Towards Facet-Driven Generation of Clarifying Questions for Conversational Search 面向面驱动的会话搜索澄清问题生成

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472257

Ivan Sekulic, Mohammad Aliannejadi, F. Crestani

{"title":"Towards Facet-Driven Generation of Clarifying Questions for Conversational Search","authors":"Ivan Sekulic, Mohammad Aliannejadi, F. Crestani","doi":"10.1145/3471158.3472257","DOIUrl":"https://doi.org/10.1145/3471158.3472257","url":null,"abstract":"Clarifying an underlying user information need is an important aspect of a modern-day IR system. The importance of clarification is even higher in limited-bandwidth scenarios, such as conversational or mobile search, where a user is unable to easily browse through a long list of retrieved results. Thus, asking clarifying questions about user's potentially ambiguous queries arises as one of the main tasks of conversational search. Recent approaches have, while making significant progress in the field, remained limited to selecting a clarifying question from a predefined set or prompting the user with vague or template-based questions. However, with the recent advances in text generation through large-scale language models, an ideal system should generate the next clarifying question. The challenge of generating an appropriate clarifying question is twofold: 1) to produce the question in coherent natural language; 2) to ask a question that is relevant to the user query. In this paper, we propose a model that generates clarifying questions with respect to the user query and query facets. We fine-tune the GPT-2 language model to generate questions related to the query and one of the extracted query facets. Compared to competitive baselines, results show that our proposed method is both natural and useful, as judged by human annotators. Moreover, we discuss the potential theoretical framework this approach would fit in. We release the code for future work and reproducibility purposes.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129734380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 22

Session details: Keynote Talk 会议详情:主题演讲

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3484636

Yizhou Fang

引用次数: 0

An Integrated Model of Task, Information Needs, Sources and Uncertainty to Design Task-Aware Search Systems 任务感知搜索系统的任务、信息需求、资源和不确定性集成模型

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472231

Shawon Sarkar, C. Shah

{"title":"An Integrated Model of Task, Information Needs, Sources and Uncertainty to Design Task-Aware Search Systems","authors":"Shawon Sarkar, C. Shah","doi":"10.1145/3471158.3472231","DOIUrl":"https://doi.org/10.1145/3471158.3472231","url":null,"abstract":"The varieties of information seeking behavior encompass a range of practices and constructs such as the realization of an information need, selecting the nature of information, as well as information sources. Most of the past works have studied various constructs of the information seeking process, i.e., information, information need, and information sources individually. However, a person forms and re-forms his or her information seeking strategy based on continually shifting values of these dimensions associated with information seeking. This preliminary study conducted a survey with 15 search scenarios and multiple-choice characteristics completed by 114 Amazon's Mechanical Turk workers to find out more about how these constructs play a role in people's preferences regarding information seeking strategies. The study took an exploratory and inferential research approach to investigate how different forms of information and information needs might lead to different information sources by building binary classification models. The results show that the choice of sources can be predicted (with 80% accuracy) if the information need, representation, and form of information are apparent.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126095035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Session details: Session 4B - Semantic Retrieval 会话详细信息:会话4B -语义检索

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3484645

Harrie Oosterhuis

引用次数: 0

How do Metric Score Distributions affect the Type I Error Rate of Statistical Significance Tests in Information Retrieval? 度量分数分布如何影响信息检索中统计显著性检验的第一类错误率?

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472242

Julián Urbano, Matteo Corsi, A. Hanjalic

{"title":"How do Metric Score Distributions affect the Type I Error Rate of Statistical Significance Tests in Information Retrieval?","authors":"Julián Urbano, Matteo Corsi, A. Hanjalic","doi":"10.1145/3471158.3472242","DOIUrl":"https://doi.org/10.1145/3471158.3472242","url":null,"abstract":"Statistical significance tests are the main tool that IR practitioners use to determine the reliability of their experimental evaluation results. The question of which test behaves best with IR evaluation data has been around for decades, and has seen all kinds of results and recommendations. Definitive answer to this question has recently been attempted via stochastic simulation of IR evaluation data, allowing researchers to compute actual Type I error rates because they can control the null hypothesis. One such research line simulates metric scores for a fixed set of systems on random topics, and concluded that the t-test behaves the best. Another such line simulates retrieval runs by random systems on a fixed set of topics, and concluded that the Wilcoxon test behaves the best. Interestingly, two recent surveys of the IR literature have shown that the community has a clear preference precisely for these two tests, so further investigation is critical to understand why the above simulation studies reach opposite conclusions. It has been recently postulated that a reason for the disagreement is the distributions of metric scores used by one of these simulation methods. In this paper we investigate this issue and extend the argument to another key aspect of the simulation, namely the dependence between systems. Following a principled approach, we analyze the robustness of statistical tests to different factors, thus identifying under what conditions they behave well or not with respect to the Type I error rate. Our results suggest that differences between the Wilcoxon and t-test may be explained by the skewness of score differences.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129787197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Extracting per Query Valid Explanations for Blackbox Learning-to-Rank Models 为黑箱学习排序模型提取每个查询的有效解释

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472241

Jaspreet Singh, Megha Khosla, Wang Zhenye, Avishek Anand

{"title":"Extracting per Query Valid Explanations for Blackbox Learning-to-Rank Models","authors":"Jaspreet Singh, Megha Khosla, Wang Zhenye, Avishek Anand","doi":"10.1145/3471158.3472241","DOIUrl":"https://doi.org/10.1145/3471158.3472241","url":null,"abstract":"Learning-to-rank (LTR) is a class of supervised learning techniques that apply to ranking problems dealing with a large number of features. The popularity and widespread application of LTR models in prioritizing information in a variety of domains makes their scrutability vital in today's landscape of fair and transparent learning systems. However, limited work exists that deals with interpreting the decisions of learning systems that output rankings. In this paper we propose a model agnostic local explanation method that seeks to identify a small subset of input features as explanation to the ranked output for a given query. We introduce new notions of validity and completeness of explanations specifically for rankings, based on the presence or absence of selected features, as a way of measuring goodness. We devise a novel optimization problem to maximize validity directly and propose greedy algorithms as solutions. In extensive quantitative experiments we show that our approach outperforms other model agnostic explanation approaches across pointwise, pairwise and listwise LTR models in validity while not compromising on completeness.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126112615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9