Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval最新文献_第2页

Multi-label Classification of Short Texts with Label Correlated Recurrent Neural Networks 基于标签相关递归神经网络的短文本多标签分类

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472246

Zhiyuan Peng, B. Abdollahi, Min Xie, Yi Fang

引用次数: 4

BERT-based Dense Retrievers Require Interpolation with BM25 for Effective Passage Retrieval 基于bert的密集检索需要BM25插值来进行有效的通道检索

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472233

Shuai Wang, Shengyao Zhuang, G. Zuccon

{"title":"BERT-based Dense Retrievers Require Interpolation with BM25 for Effective Passage Retrieval","authors":"Shuai Wang, Shengyao Zhuang, G. Zuccon","doi":"10.1145/3471158.3472233","DOIUrl":"https://doi.org/10.1145/3471158.3472233","url":null,"abstract":"The integration of pre-trained deep language models, such as BERT, into retrieval and ranking pipelines has shown to provide large effectiveness gains over traditional bag-of-words models in the passage retrieval task. However, the best setup for integrating such deep language models is still unclear. When BERT is used to re-rank passages (i.e., BERT re-ranker), previous work has empirically shown that, while in practice BERT re-ranker cannot act as initial retriever due to BERT's high query time costs, and thus a bag-of-words model such as BM25 is required. It is not necessary to interpolate BERT re-ranker and bag-of-words scores to generate the final ranking. In fact, the BERT re-ranker scores alone can be used by the re-ranker: the BERT re-ranker score appears to already capture the relevance signal provided by BM25. In this paper, we further investigate the topic of interpolating BM25 and BERT-based rankers. Unlike previous work that considered the BERT re-ranker, however, here we consider BERT-based dense retrievers (RepBERT and ANCE). Dense retrievers encode queries and documents into low dimensional BERT-based embeddings. These methods overcome BERT's high computational costs at query time, and can thus be feasibly used in practice as whole-collection retrievers, rather than just as re-rankers. Our novel empirical findings suggest that, unlike for BERT re-ranker, interpolation with BM25 is necessary for BERT-based dense retrievers to perform effectively; and the gains provided by the interpolation are significant. Further analysis reveals why this is so: dense retrievers are very effective at encoding strong relevance signals, but they fail in identifying weaker relevance signals -- a task that the interpolation with BM25 is able to make up for.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125555678","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 40

Asking Clarifying Questions Based on Negative Feedback in Conversational Search 会话搜索中基于负面反馈提出澄清性问题

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472232

Keping Bi, Qingyao Ai, W. Bruce Croft

{"title":"Asking Clarifying Questions Based on Negative Feedback in Conversational Search","authors":"Keping Bi, Qingyao Ai, W. Bruce Croft","doi":"10.1145/3471158.3472232","DOIUrl":"https://doi.org/10.1145/3471158.3472232","url":null,"abstract":"Users often need to look through multiple search result pages or reformulate queries when they have complex information-seeking needs. Conversational search systems make it possible to improve user satisfaction by asking questions to clarify users' search intents. This, however, can take significant effort to answer a series of questions starting with \"what/why/how\". To quickly identify user intent and reduce effort during interactions, we propose an intent clarification task based on yes/no questions where the system needs to ask the correct question about intents within the fewest conversation turns. In this task, it is essential to use negative feedback about the previous questions in the conversation history. To this end, we propose a Maximum-Marginal-Relevance (MMR) based BERT model (MMR-BERT) to leverage negative feedback based on the MMR principle for the next clarifying question selection. Experiments on the Qulac dataset show that MMR-BERT outperforms state-of-the-art baselines significantly on the intent identification task and the selected questions also achieve significantly better performance in the associated document retrieval tasks.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"28 2 Suppl 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116567779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 14

Recommending Search Queries in Documents Using Inter N-Gram Similarities 使用内部N-Gram相似性推荐文档中的搜索查询

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472252

Eilon Sheetrit, Yaroslav Fyodorov, Fiana Raiber, Oren Kurland

{"title":"Recommending Search Queries in Documents Using Inter N-Gram Similarities","authors":"Eilon Sheetrit, Yaroslav Fyodorov, Fiana Raiber, Oren Kurland","doi":"10.1145/3471158.3472252","DOIUrl":"https://doi.org/10.1145/3471158.3472252","url":null,"abstract":"Reading a document can often trigger a need for additional information. For example, a reader of a news article might be interested in information about the persons and events mentioned in the article. Accordingly, there is a line of work on recommending search-engine queries given a document read by a user. Often, the recommended queries are selected from a query log independently of each other, and are presented to the user without any context. We address a novel query recommendation task where the recommended queries must be n-grams (sequences of consecutive terms) in the document. Furthermore, inspired by work on using inter-document similarities for document retrieval, we explore the merits of using inter n-gram similarities for query recommendation. Specifically, we use a supervised approach to learn an inter n-gram similarity measure where the goal is that n-grams that are likely to serve as queries will be deemed more similar to each other than to other n-grams. We use the similarity measure in a wide variety of query recommendation approaches which we devise as adaptations of ad hoc document retrieval techniques. Empirical evaluation performed using data gathered from Yahoo!'s search engine logs attests to the effectiveness of the resultant recommendation methods.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125305567","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Sentiment Intensity Prediction using Neural Word Embeddings 基于神经词嵌入的情感强度预测

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472254

Amal Htait, L. Azzopardi

{"title":"Sentiment Intensity Prediction using Neural Word Embeddings","authors":"Amal Htait, L. Azzopardi","doi":"10.1145/3471158.3472254","DOIUrl":"https://doi.org/10.1145/3471158.3472254","url":null,"abstract":"Sentiment analysis is central to the process of mining opinions and attitudes from online texts. While much attention has been paid to the sentiment classification problem, much less work has tried to tackle the problem of predicting the intensity of the sentiment. The go to method is VADER --- an unsupervised lexicon based approach to scoring sentiment. However, such approaches are limited because of the vocabulary mismatch problem. In this paper, we present in detail and evaluate our AWESSOME framework (A Word Embedding Sentiment Scorer Of Many Emotions) for sentiment intensity scoring, that capitalizes on pre-existing lexicons, does not require training and provides fine grained and accurate sentiment intensity scores of words, phrases and text. In our experiments, we used seven Sentiment Collections to evaluate the proposed approach, against lexicon based approaches (e.g., VADER), and supervised methods such as deep learning based approaches (e.g., SentiBERT). The results show that despite not surpassing supervised approaches, the AWESSOME unsupervised approach significantly outperforms existing lexicon approaches and therefore provides a simple and effective approach for sentiment analysis. The AWESSOME framework can be flexibly adapted to cater for different seed lexicons and different neural word embeddings models in order to produce corpus specific lexicons -- without the need for extensive supervised learning or retraining.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122760600","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Discriminative Semantic Ranker for Question Retrieval 问题检索的判别语义排序器

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472227

Yinqiong Cai, Yixing Fan, Jiafeng Guo, Ruqing Zhang, Yanyan Lan, Xueqi Cheng

{"title":"A Discriminative Semantic Ranker for Question Retrieval","authors":"Yinqiong Cai, Yixing Fan, Jiafeng Guo, Ruqing Zhang, Yanyan Lan, Xueqi Cheng","doi":"10.1145/3471158.3472227","DOIUrl":"https://doi.org/10.1145/3471158.3472227","url":null,"abstract":"Similar question retrieval is a core task in community-based question answering (CQA) services. To balance the effectiveness and efficiency, the question retrieval system is typically implemented as multi-stage rankers: The first-stage ranker aims to recall potentially relevant questions from a large repository, and the latter stages attempt to re-rank the retrieved results. Most existing works on question retrieval mainly focused on the re-ranking stages, leaving the first-stage ranker to some traditional term-based methods. However, term-based methods often suffer from the vocabulary mismatch problem, especially on short texts, which may block the re-rankers from relevant questions at the very beginning. An alternative is to employ embedding-based methods for the first-stage ranker, which compress texts into dense vectors to enhance the semantic matching. However, these methods often lose the discriminative power as term-based methods, thus introduce noise during retrieval and hurt the recall performance. In this work, we aim to tackle the dilemma of the first-stage ranker, and propose a discriminative semantic ranker, namely DenseTrans, for high-recall retrieval. Specifically, DenseTrans is a densely connected Transformer, which learns semantic embeddings for texts based on Transformer layers. Meanwhile, DenseTrans promotes low-level features through dense connections to keep the discriminative power of the learned representations. DenseTrans is inspired by DenseNet in computer vision (CV), but poses a new way to use the dense connectivity which is totally different from its original design purpose. Experimental results over two question retrieval benchmark datasets show that our model can obtain significant gain on recall against strong term-based methods as well as state-of-the-art embedding-based methods.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125728064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Effective Query Formulation in Conversation Contextualization: A Query Specificity-based Approach 会话上下文化中的有效查询公式:一种基于查询特殊性的方法

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472237

Dipasree Pal, Debasis Ganguly

{"title":"Effective Query Formulation in Conversation Contextualization: A Query Specificity-based Approach","authors":"Dipasree Pal, Debasis Ganguly","doi":"10.1145/3471158.3472237","DOIUrl":"https://doi.org/10.1145/3471158.3472237","url":null,"abstract":"Proactively retrieving relevant information to contextualize conversations has potential applications in better understanding the conversational content between communicating parties. Since in contrast to traditional IR, there is no explicitly formulated user-query, an important research challenge is to first identify the candidate segments of text that may require contextualization for a better comprehension of their content, and then make use of these identified segments to formulate a query and eventually retrieve the potentially relevant information to augment a conversation. In this paper, we propose a generic unsupervised framework that involves shifting overlapping windows of terms through a conversation and estimating scores indicating the likelihood of the existence of an information need within these segments. Within our proposed framework, we investigate a query performance prediction (QPP) based approach for scoring these candidate term windows with the hypothesis that a term window that indicates a higher specificity is likely to be indicative of a potential information need requiring contextualization. Our experiments revealed that the QPP approaches of scoring the term windows provide better contextualization than other term extraction approaches. A post-retrieval QPP approach was observed to yield better results than a pre-retrieval one.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130901598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Can Language Models Identify Wikipedia Articles with Readability and Style Issues? 语言模型可以识别维基百科文章的可读性和风格问题吗?

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472234

Yang Liu, A. Medlar, D. Glowacka

引用次数: 1

Session details: Session 3B - Evaluation 会议详情:会议3B -评估

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3484643

Mohammad Aliannejadi

引用次数: 0

Pareto Solutions vs Dataset Optima: Concepts and Methods for Optimizing Competing Objectives with Constraints in Retrieval 帕累托解与数据集最优:检索中具有约束的竞争目标优化的概念和方法

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval Pub Date : 2021-07-11 DOI: 10.1145/3471158.3472248

Soumyajit Gupta, Gurpreet Singh, Anubrata Das, Matthew Lease

{"title":"Pareto Solutions vs Dataset Optima: Concepts and Methods for Optimizing Competing Objectives with Constraints in Retrieval","authors":"Soumyajit Gupta, Gurpreet Singh, Anubrata Das, Matthew Lease","doi":"10.1145/3471158.3472248","DOIUrl":"https://doi.org/10.1145/3471158.3472248","url":null,"abstract":"When ranking search results for multiple objectives, such as maximizing the relevance and diversity of retrieved documents, competing objectives can induce a space of optimal solutions, each reflecting a different optimal trade-off over objectives. This raises several important questions. Firstly, what Pareto optimal solution set is induced by objective functions? Secondly, what are the best solutions achievable on a given dataset? Finally, how closely do the best dataset solutions approach the true Pareto optimal? We present a clear conceptual framing of these questions, with supporting terminology and visualizations, distinguishing Pareto vs. \"dataset-best\" solutions and providing strong intuition about how and why different optimization problems lead to different shapes and forms of solutions (regardless of optimization technique). We also provide benchmark problems for verifying the correctness of any Pareto machinery and show how existing multi-objective optimization (MOO) and filter methods can be used to provide accurate and interpretable answers to the above questions. Finally, we show how user-defined constraints imposed on the solution space can be effectively handled. In sum, we provide conceptual, translational, and practical contributions to solving MOO problems in retrieval.","PeriodicalId":289271,"journal":{"name":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116610259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0