发布求助
文献互助 智能选刊 最新文献

Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)最新文献

筛选
英文 中文
FaBULOUS: Fact-checking Based on Understanding of Language Over Unstructured and Structured information 神话:基于对非结构化和结构化信息的语言理解的事实核查
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.4
Mostafa Bouziane, Hugo Perrin, Amine Sadeq, Thanh-Tung Nguyen, Aurélien Cluzeau, Julien Mardas
{"title":"FaBULOUS: Fact-checking Based on Understanding of Language Over Unstructured and Structured information","authors":"Mostafa Bouziane, Hugo Perrin, Amine Sadeq, Thanh-Tung Nguyen, Aurélien Cluzeau, Julien Mardas","doi":"10.18653/v1/2021.fever-1.4","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.4","url":null,"abstract":"As part of the FEVEROUS shared task, we developed a robust and finely tuned architecture to handle the joint retrieval and entailment on text data as well as structured data like tables. We proposed two training schemes to tackle the hurdles inherent to multi-hop multi-modal datasets. The first one allows having a robust retrieval of full evidence sets, while the second one enables entailment to take full advantage of noisy evidence inputs. In addition, our work has revealed important insights and potential avenue of research for future improvement on this kind of dataset. In preliminary evaluation on the FEVEROUS shared task test set, our system achieves 0.271 FEVEROUS score, with 0.4258 evidence recall and 0.5607 entailment accuracy.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115513245","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German FANG-COVID:一个新的大规模德语假新闻检测基准数据集
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.9
Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann, M. Strohmaier
{"title":"FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German","authors":"Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann, M. Strohmaier","doi":"10.18653/v1/2021.fever-1.9","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.9","url":null,"abstract":"As the world continues to fight the COVID-19 pandemic, it is simultaneously fighting an ‘infodemic’ – a flood of disinformation and spread of conspiracy theories leading to health threats and the division of society. To combat this infodemic, there is an urgent need for benchmark datasets that can help researchers develop and evaluate models geared towards automatic detection of disinformation. While there are increasing efforts to create adequate, open-source benchmark datasets for English, comparable resources are virtually unavailable for German, leaving research for the German language lagging significantly behind. In this paper, we introduce the new benchmark dataset FANG-COVID consisting of 28,056 real and 13,186 fake German news articles related to the COVID-19 pandemic as well as data on their propagation on Twitter. Furthermore, we propose an explainable textual- and social context-based model for fake news detection, compare its performance to “black-box” models and perform feature ablation to assess the relative importance of human-interpretable features in distinguishing fake news from authentic news.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115464395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Team Papelo at FEVEROUS: Multi-hop Evidence Pursuit 狂热的帕佩洛小组:多跳证据追踪
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.5
Christopher Malon
{"title":"Team Papelo at FEVEROUS: Multi-hop Evidence Pursuit","authors":"Christopher Malon","doi":"10.18653/v1/2021.fever-1.5","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.5","url":null,"abstract":"We develop a system for the FEVEROUS fact extraction and verification task that ranks an initial set of potential evidence and then pursues missing evidence in subsequent hops by trying to generate it, with a “next hop prediction module” whose output is matched against page elements in a predicted article. Seeking evidence with the next hop prediction module continues to improve FEVEROUS score for up to seven hops. Label classification is trained on possibly incomplete extracted evidence chains, utilizing hints that facilitate numerical comparison. The system achieves .281 FEVEROUS score and .658 label accuracy on the development set, and finishes in second place with .259 FEVEROUS score and .576 label accuracy on the test set.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122923964","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Verdict Inference with Claim and Retrieved Elements Using RoBERTa 使用RoBERTa对索赔和检索元素进行判决推理
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.7
In-Zu Gi, Ting-Yu Fang, Richard Tzong-Han Tsai
{"title":"Verdict Inference with Claim and Retrieved Elements Using RoBERTa","authors":"In-Zu Gi, Ting-Yu Fang, Richard Tzong-Han Tsai","doi":"10.18653/v1/2021.fever-1.7","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.7","url":null,"abstract":"Automatic fact verification has attracted recent research attention as the increasing dissemination of disinformation on social media platforms. The FEVEROUS shared task introduces a benchmark for fact verification, in which a system is challenged to verify the given claim using the extracted evidential elements from Wikipedia documents. In this paper, we propose our 3rd place three-stage system consisting of document retrieval, element retrieval, and verdict inference for the FEVEROUS shared task. By considering the context relevance in the fact extraction and verification task, our system achieves 0.29 FEVEROUS score on the development set and 0.25 FEVEROUS score on the blind test set, both outperforming the FEVEROUS baseline.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"97 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130329938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Modeling Entity Knowledge for Fact Verification 为事实验证建模实体知识
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.6
Yang Liu, Chenguang Zhu, Michael Zeng
{"title":"Modeling Entity Knowledge for Fact Verification","authors":"Yang Liu, Chenguang Zhu, Michael Zeng","doi":"10.18653/v1/2021.fever-1.6","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.6","url":null,"abstract":"Fact verification is a challenging task of identifying the truthfulness of given claims based on the retrieval of relevant evidence texts. Many claims require understanding and reasoning over external entity information for precise verification. In this paper, we propose a novel fact verification model using entity knowledge to enhance its performance. We retrieve descriptive text from Wikipedia for each entity, and then encode these descriptions by a smaller lightweight network to be fed into the main verification model. Furthermore, we boost model performance by adopting and predicting the relatedness between the claim and each evidence as additional signals. We demonstrate experimentally on a large-scale benchmark dataset FEVER that our framework achieves competitive results with a FEVER score of 72.89% on the test set.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"345 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124263193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Automatic Fact-Checking with Document-level Annotations using BERT and Multiple Instance Learning 使用BERT和多实例学习的文档级注释的自动事实检查
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.11
Aalok Sathe, Joonsuk Park
{"title":"Automatic Fact-Checking with Document-level Annotations using BERT and Multiple Instance Learning","authors":"Aalok Sathe, Joonsuk Park","doi":"10.18653/v1/2021.fever-1.11","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.11","url":null,"abstract":"Automatic fact-checking is crucial for recognizing misinformation spreading on the internet. Most existing fact-checkers break down the process into several subtasks, one of which determines candidate evidence sentences that can potentially support or refute the claim to be verified; typically, evidence sentences with gold-standard labels are needed for this. In a more realistic setting, however, such sentence-level annotations are not available. In this paper, we tackle the natural language inference (NLI) subtask—given a document and a (sentence) claim, determine whether the document supports or refutes the claim—only using document-level annotations. Using fine-tuned BERT and multiple instance learning, we achieve 81.9% accuracy, significantly outperforming the existing results on the WikiFactCheck-English dataset.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"123 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127056115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
A Fact Checking and Verification System for FEVEROUS Using a Zero-Shot Learning Approach 使用零射击学习方法的fever事实检查和验证系统
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.13
Orkun Temiz, Oz Kilic, Arif Ozan Kızıldağ, Tugba Taskaya Temizel
{"title":"A Fact Checking and Verification System for FEVEROUS Using a Zero-Shot Learning Approach","authors":"Orkun Temiz, Oz Kilic, Arif Ozan Kızıldağ, Tugba Taskaya Temizel","doi":"10.18653/v1/2021.fever-1.13","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.13","url":null,"abstract":"In this paper, we propose a novel fact checking and verification system to check claims against Wikipedia content. Our system retrieves relevant Wikipedia pages using Anserini, uses BERT-large-cased question answering model to select correct evidence, and verifies claims using XLNET natural language inference model by comparing it with the evidence. Table cell evidence is obtained through looking for entity-matching cell values and TAPAS table question answering model. The pipeline utilizes zero-shot capabilities of existing models and all the models used in the pipeline requires no additional training. Our system got a FEVEROUS score of 0.06 and a label accuracy of 0.39 in FEVEROUS challenge.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114326270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) Shared Task 非结构化和结构化信息(fever)共享任务的事实提取与验证
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.1
Rami Aly, Zhijiang Guo, M. Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, O. Cocarascu, Arpit Mittal
{"title":"The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) Shared Task","authors":"Rami Aly, Zhijiang Guo, M. Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, O. Cocarascu, Arpit Mittal","doi":"10.18653/v1/2021.fever-1.1","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.1","url":null,"abstract":"The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) shared task, asks participating systems to determine whether human-authored claims are Supported or Refuted based on evidence retrieved from Wikipedia (or NotEnoughInfo if the claim cannot be verified). Compared to the FEVER 2018 shared task, the main challenge is the addition of structured data (tables and lists) as a source of evidence. The claims in the FEVEROUS dataset can be verified using only structured evidence, only unstructured evidence, or a mixture of both. Submissions are evaluated using the FEVEROUS score that combines label accuracy and evidence retrieval. Unlike FEVER 2018, FEVEROUS requires partial evidence to be returned for NotEnoughInfo claims, and the claims are longer and thus more complex. The shared task received 13 entries, six of which were able to beat the baseline system. The winning team was “Bust a move!”, achieving a FEVEROUS score of 27% (+9% compared to the baseline). In this paper we describe the shared task, present the full results and highlight commonalities and innovations among the participating systems.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132445825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 44
Evidence Selection as a Token-Level Prediction Task 证据选择作为标记级预测任务
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.2
Dominik Stammbach
{"title":"Evidence Selection as a Token-Level Prediction Task","authors":"Dominik Stammbach","doi":"10.18653/v1/2021.fever-1.2","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.2","url":null,"abstract":"In Automated Claim Verification, we retrieve evidence from a knowledge base to determine the veracity of a claim. Intuitively, the retrieval of the correct evidence plays a crucial role in this process. Often, evidence selection is tackled as a pairwise sentence classification task, i.e., we train a model to predict for each sentence individually whether it is evidence for a claim. In this work, we fine-tune document level transformers to extract all evidence from a Wikipedia document at once. We show that this approach performs better than a comparable model classifying sentences individually on all relevant evidence selection metrics in FEVER. Our complete pipeline building on this evidence selection procedure produces a new state-of-the-art result on FEVER, a popular claim verification benchmark.","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132794311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Neural Re-rankers for Evidence Retrieval in the FEVEROUS Task 热任务中证据检索的神经重排序
Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.fever-1.12
Mohammed Saeed, Giulio Alfarano, Khai Nguyen, Duc-Hong Pham, Raphael Troncy, Paolo Papotti
{"title":"Neural Re-rankers for Evidence Retrieval in the FEVEROUS Task","authors":"Mohammed Saeed, Giulio Alfarano, Khai Nguyen, Duc-Hong Pham, Raphael Troncy, Paolo Papotti","doi":"10.18653/v1/2021.fever-1.12","DOIUrl":"https://doi.org/10.18653/v1/2021.fever-1.12","url":null,"abstract":"Computational fact-checking has gained a lot of traction in the machine learning and natural language processing communities. A plethora of solutions have been developed, but methods which leverage both structured and unstructured information to detect misinformation are of particular relevance. In this paper, we tackle the FEVEROUS (Fact Extraction and VERification Over Unstructured and Structured information) challenge which consists of an open source baseline system together with a benchmark dataset containing 87,026 verified claims. We extend this baseline model by improving the evidence retrieval module yielding the best evidence F1 score among the competitors in the challenge leaderboard while obtaining an overall FEVEROUS score of 0.20 (5th best ranked system).","PeriodicalId":417162,"journal":{"name":"Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133818387","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
下一页 尾页
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整，为了账户安全，请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们：info@booksci.cn Book学术提供免费学术资源搜索服务，方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号：604180095
Book学术官方微信