Conference and Labs of the Evaluation Forum最新文献_第2页

Overview of BioASQ 2022: The Tenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering BioASQ 2022概述:大规模生物医学语义索引和问答的第十届BioASQ挑战

Conference and Labs of the Evaluation Forum Pub Date : 2022-10-13 DOI: 10.1007/978-3-031-13643-6_22

A. Nentidis, Georgios Katsimpras, Eirini Vandorou, Anastasia Krithara, Antonio Miranda-Escalada, Luis Gasco, Martin Krallinger, G. Paliouras

引用次数: 6

Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet Text 在CheckThat!实验室2022:推特文本的可靠性识别

Conference and Labs of the Evaluation Forum Pub Date : 2022-07-15 DOI: 10.48550/arXiv.2207.07308

Prerona Tarannum, Firoj Alam, Md. Arid Hasan, S. R. H. Noori

{"title":"Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet Text","authors":"Prerona Tarannum, Firoj Alam, Md. Arid Hasan, S. R. H. Noori","doi":"10.48550/arXiv.2207.07308","DOIUrl":"https://doi.org/10.48550/arXiv.2207.07308","url":null,"abstract":"The wide use of social media and digital technologies facilitates sharing various news and information about events and activities. Despite sharing positive information misleading and false information is also spreading on social media. There have been efforts in identifying such misleading information both manually by human experts and automatic tools. Manual effort does not scale well due to the high volume of information, containing factual claims, are appearing online. Therefore, automatically identifying check-worthy claims can be very useful for human experts. In this study, we describe our participation in Subtask-1A: Check-worthiness of tweets (English, Dutch and Spanish) of CheckThat! lab at CLEF 2022. We performed standard preprocessing steps and applied different models to identify whether a given text is worthy of fact checking or not. We use the oversampling technique to balance the dataset and applied SVM and Random Forest (RF) with TF-IDF representations. We also used BERT multilingual (BERT-m) and XLM-RoBERTa-base pre-trained models for the experiments. We used BERT-m for the official submissions and our systems ranked as 3rd, 5th, and 12th in Spanish, Dutch, and English, respectively. In further experiments, our evaluation shows that transformer models (BERT-m and XLM-RoBERTa-base) outperform the SVM and RF in Dutch and English languages where a different scenario is observed for Spanish.","PeriodicalId":232729,"journal":{"name":"Conference and Labs of the Evaluation Forum","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130497035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

A Late Fusion Framework with Multiple Optimization Methods for Media Interestingness 基于多优化方法的媒体兴趣度后期融合框架

Conference and Labs of the Evaluation Forum Pub Date : 2022-07-11 DOI: 10.48550/arXiv.2207.04762

M. Shoukat, Khubaib Ahmad, Naina Said, Nasir Ahmad, Mohammed Hasanuzzaman, Kashif Ahmad

{"title":"A Late Fusion Framework with Multiple Optimization Methods for Media Interestingness","authors":"M. Shoukat, Khubaib Ahmad, Naina Said, Nasir Ahmad, Mohammed Hasanuzzaman, Kashif Ahmad","doi":"10.48550/arXiv.2207.04762","DOIUrl":"https://doi.org/10.48550/arXiv.2207.04762","url":null,"abstract":"The recent advancement in Multimedia Analytical, Computer Vision (CV), and Artificial Intelligence (AI) algorithms resulted in several interesting tools allowing an automatic analysis and retrieval of multimedia content of users' interests. However, retrieving the content of interest generally involves analysis and extraction of semantic features, such as emotions and interestingness-level. The extraction of such meaningful information is a complex task and generally, the performance of individual algorithms is very low. One way to enhance the performance of the individual algorithms is to combine the predictive capabilities of multiple algorithms using fusion schemes. This allows the individual algorithms to complement each other, leading to improved performance. This paper proposes several fusion methods for the media interestingness score prediction task introduced in CLEF Fusion 2022. The proposed methods include both a naive fusion scheme, where all the inducers are treated equally and a merit-based fusion scheme where multiple weight optimization methods are employed to assign weights to the individual inducers. In total, we used six optimization methods including a Particle Swarm Optimization (PSO), a Genetic Algorithm (GA), Nelder Mead, Trust Region Constrained (TRC), and Limited-memory Broyden Fletcher Goldfarb Shanno Algorithm (LBFGSA), and Truncated Newton Algorithm (TNA). Overall better results are obtained with PSO and TNA achieving 0.109 mean average precision at 10. The task is complex and generally, scores are low. We believe the presented analysis will provide a baseline for future research in the domain.","PeriodicalId":232729,"journal":{"name":"Conference and Labs of the Evaluation Forum","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131254251","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Solutions for Fine-grained and Long-tailed Snake Species Recognition in SnakeCLEF 2022 蛇类识别中细纹和长尾蛇物种识别的解决方案

Conference and Labs of the Evaluation Forum Pub Date : 2022-07-04 DOI: 10.48550/arXiv.2207.01216

Cheng Zou, Furong Xu, Meng Wang, Wenqi Li, Yuan Cheng

引用次数: 3

Few-shot Long-Tailed Bird Audio Recognition 少射长尾鸟音频识别

Conference and Labs of the Evaluation Forum Pub Date : 2022-06-22 DOI: 10.48550/arXiv.2206.11260

Marcos V. Conde, Ui-Jin Choi

引用次数: 4

Motif Mining and Unsupervised Representation Learning for BirdCLEF 2022 Motif Mining and Unsupervised Representation Learning for BirdCLEF 2022

Conference and Labs of the Evaluation Forum Pub Date : 2022-06-08 DOI: 10.48550/arXiv.2206.04805

Anthony Miyaguchi, Jiangyue Yu, Bryan Cheungvivatpant, Dakota Dudley, Aniketh Swain

引用次数: 2

hmBERT: Historical Multilingual Language Models for Named Entity Recognition 命名实体识别的历史多语言语言模型

Conference and Labs of the Evaluation Forum Pub Date : 2022-05-31 DOI: 10.48550/arXiv.2205.15575

Stefan Schweter, Luisa März, Katharina Schmid, Erion cCano

{"title":"hmBERT: Historical Multilingual Language Models for Named Entity Recognition","authors":"Stefan Schweter, Luisa März, Katharina Schmid, Erion cCano","doi":"10.48550/arXiv.2205.15575","DOIUrl":"https://doi.org/10.48550/arXiv.2205.15575","url":null,"abstract":"Compared to standard Named Entity Recognition (NER), identifying persons, locations, and organizations in historical texts constitutes a big challenge. To obtain machine-readable corpora, the historical text is usually scanned and Optical Character Recognition (OCR) needs to be performed. As a result, the historical corpora contain errors. Also, entities like location or organization can change over time, which poses another challenge. Overall, historical texts come with several peculiarities that differ greatly from modern texts and large labeled corpora for training a neural tagger are hardly available for this domain. In this work, we tackle NER for historical German, English, French, Swedish, and Finnish by training large historical language models. We circumvent the need for large amounts of labeled data by using unlabeled data for pretraining a language model. We propose hmBERT, a historical multilingual BERT-based language model, and release the model in several versions of different sizes. Furthermore, we evaluate the capability of hmBERT by solving downstream NER as part of this year's HIPE-2022 shared task and provide detailed analysis and insights. For the Multilingual Classical Commentary coarse-grained NER challenge, our tagger HISTeria outperforms the other teams' models for two out of three languages.","PeriodicalId":232729,"journal":{"name":"Conference and Labs of the Evaluation Forum","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122978829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Overview of LiLAS 2021 - Living Labs for Academic Search (Extended Overview) LiLAS 2021概述-学术搜索生活实验室(扩展概述)

Conference and Labs of the Evaluation Forum Pub Date : 2022-03-10 DOI: 10.1007/978-3-030-85251-1_25

Philipp Schaer, Timo Breuer, L. J. Castro, Benjamin Wolff, Johann Schaible, Narges Tavakolpoursaleh

引用次数: 9

Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering BioASQ 2021概述:关于大规模生物医学语义索引和问答的第九次BioASQ挑战

Conference and Labs of the Evaluation Forum Pub Date : 2021-06-28 DOI: 10.1007/978-3-030-85251-1_18

A. Nentidis, Anastasia Krithara, K. Bougiatiotis, Martin Krallinger, C. R. Penagos, Marta Villegas, G. Paliouras

引用次数: 49

Self-Calibrating Neural-Probabilistic Model for Authorship Verification Under Covariate Shift 协变量移位下作者身份验证的自校正神经概率模型

Conference and Labs of the Evaluation Forum Pub Date : 2021-06-21 DOI: 10.1007/978-3-030-85251-1_12

Benedikt T. Boenninghoff, D. Kolossa, R. M. Nickel

引用次数: 4