Quality Assurance of a German COVID-19 Question Answering Systems using Component-based Microbenchmarking

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining Pub Date : 2022-02-11 DOI:10.1145/3488560.3502196

A. Both, Paul Heinze, A. Perevalov, Johannes Richard Bartsch, Rostislav Iudin, Johannes Rudolf Herkner, Tim Schrader, Jonas Wunsch, René Gürth, Ann Kristin Falkenhain

{"title":"Quality Assurance of a German COVID-19 Question Answering Systems using Component-based Microbenchmarking","authors":"A. Both, Paul Heinze, A. Perevalov, Johannes Richard Bartsch, Rostislav Iudin, Johannes Rudolf Herkner, Tim Schrader, Jonas Wunsch, René Gürth, Ann Kristin Falkenhain","doi":"10.1145/3488560.3502196","DOIUrl":null,"url":null,"abstract":"Question Answering (QA) has become an often used method to retrieve data as part of chatbots and other natural-language user interfaces. In particular, QA systems of official institutions have high expectations regarding the answers computed by the system, as the provided information might be critical. In this demonstration, we use the official COVID-19 QA system that was developed together with the German Federal government to provide German citizens access to data regarding incident values, number of deaths, etc. To ensure high quality, a component-based approach was used that enables exchanging data between QA components using RDF and validating the functionality of the QA system using SPARQL. Here, we will demonstrate how our solution enables developers of QA systems to use a descriptive approach to validate the quality of their implementation before the system's deployment and also within a live environment.","PeriodicalId":348686,"journal":{"name":"Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining","volume":"34 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-02-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3488560.3502196","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

Abstract

Question Answering (QA) has become an often used method to retrieve data as part of chatbots and other natural-language user interfaces. In particular, QA systems of official institutions have high expectations regarding the answers computed by the system, as the provided information might be critical. In this demonstration, we use the official COVID-19 QA system that was developed together with the German Federal government to provide German citizens access to data regarding incident values, number of deaths, etc. To ensure high quality, a component-based approach was used that enables exchanging data between QA components using RDF and validating the functionality of the QA system using SPARQL. Here, we will demonstrate how our solution enables developers of QA systems to use a descriptive approach to validate the quality of their implementation before the system's deployment and also within a live environment.

查看原文本刊更多论文

基于组件的微基准测试的德国COVID-19问答系统质量保证

作为聊天机器人和其他自然语言用户界面的一部分，问答(QA)已经成为一种常用的检索数据的方法。特别是，官方机构的QA系统对系统计算的答案有很高的期望，因为提供的信息可能是关键的。在本次演示中，我们使用了与德国联邦政府共同开发的官方COVID-19 QA系统，为德国公民提供有关事件值、死亡人数等数据。为了确保高质量，使用了一种基于组件的方法，该方法支持使用RDF在QA组件之间交换数据，并使用SPARQL验证QA系统的功能。在这里，我们将演示我们的解决方案如何使QA系统的开发人员能够在系统部署之前以及在活动环境中使用描述性方法来验证其实现的质量。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

自引率

0.00%

发文量