Journal of Web Semantics最新文献

{"title":"Schema-agnostic SPARQL-driven faceted search benchmark generation","authors":"Claus Stadler ,&nbsp;Simon Bin ,&nbsp;Lisa Wenige ,&nbsp;Lorenz Bühmann ,&nbsp;Jens Lehmann","doi":"10.1016/j.websem.2020.100614","DOIUrl":"10.1016/j.websem.2020.100614","url":null,"abstract":"<div><p>In this work, we present a schema-agnostic faceted browsing benchmark <em>generation framework</em><span><span> for RDF data and SPARQL engines. </span>Faceted search<span> is a technique that allows narrowing down sets of information items by applying constraints over their properties, whereas facets correspond to properties of these items. While our work can be used to realise real-world faceted search user interfaces, our focus lies on the construction and benchmarking of faceted search queries over knowledge graphs. The RDF model exhibits several traits that seemingly make it a natural foundation for faceted search: all information items are represented as RDF resources, property values typically already correspond to meaningful semantic classifications, and with SPARQL there is a standard language for uniformly querying instance and schema information.</span></span></p><p><span>However, although faceted search is ubiquitous today, it is typically not performed on the RDF model directly. Two major sources of concern are the complexity of query generation and the query performance. To overcome the former, our framework comes with an intermediate domain-specific language. Thereby our approach is </span><em>SPARQL-driven</em> which means that every faceted search information need is intensionally expressed as a single SPARQL query. In regard to the latter, we investigate the possibilities and limits of real-time SPARQL-driven faceted search on contemporary triple stores. We report on our findings by evaluating systems performance and correctness characteristics when executing a benchmark generated using our generation framework.</p><p>All components, namely the benchmark generator, the benchmark runners and the underlying faceted search framework, are published freely available as open source.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100614","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73858477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Schema aware iterative Knowledge Graph completion","authors":"Kemas Wiharja ,&nbsp;Jeff Z. Pan ,&nbsp;Martin J. Kollingbaum ,&nbsp;Yu Deng","doi":"10.1016/j.websem.2020.100616","DOIUrl":"10.1016/j.websem.2020.100616","url":null,"abstract":"<div><p>Recent success of Knowledge Graph has spurred widespread interests in methods for the problem of Knowledge Graph completion. However, efforts to understand the quality of the candidate triples from these methods, in particular from the schema aspect, have been limited. Indeed, most existing Knowledge Graph completion methods do not guarantee that the expanded Knowledge Graphs are consistent with the ontological schema of the initial Knowledge Graph. In this work, we challenge the silver<span> standard method, by proposing the notion of schema-correctness. A fundamental challenge is how to make use of different types of Knowledge Graph completion methods together to improve the production of schema-correct triples. To address this, we analyse the characteristics of different methods and propose a schema aware iterative approach to Knowledge Graph completion. Our main findings are: (i) Some popular Knowledge Graph completion methods have surprisingly low schema-correctness ratio; (ii) Different types of Knowledge Graph completion methods can work with each other to help overcame individual limitations; (iii) Some iterative sequential combinations of Knowledge Graph completion methods have significantly better schema-correctness and coverage ratios than other combinations; (iv) All the MapReduce based iterative methods outperform involved single-pass methods significantly over the tested Knowledge Graphs in terms of productivity of schema-correct triples. Our findings and infrastructure can help further work on evaluating Knowledge Graph completion methods, more fine-grained approaches for schema aware iterative knowledge graph completion, as well as new approximate reasoning approaches based Knowledge Graph completion methods.</span></p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100616","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90519334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"GTFS-Madrid-Bench: A benchmark for virtual knowledge graph access in the transport domain","authors":"David Chaves-Fraga,&nbsp;Freddy Priyatna,&nbsp;Andrea Cimmino,&nbsp;Jhon Toledo,&nbsp;Edna Ruckhaus,&nbsp;Oscar Corcho","doi":"10.1016/j.websem.2020.100596","DOIUrl":"10.1016/j.websem.2020.100596","url":null,"abstract":"<div><p>A large number of datasets are being made available on the Web using a variety of formats and according to diverse data models. Ontology Based Data Integration (OBDI) has been traditionally proposed as a mechanism to facilitate access to such heterogeneous datasets, providing a unified view over their data by means of ontologies. Recently, the term “Virtual Knowledge Graph Access” has begun to be used to refer to the mechanisms that provide query-based access to knowledge graphs virtually generated from heterogeneous data sources. Several OBDI engines exist in the state of the art, with overlapping capabilities but also clear differences among them (in terms of the data formats that they can deal with, mapping languages that they support, query expressivity that they allow, etc.). These engines have been evaluated with different testbeds and benchmarks. However, their heterogeneity has made it difficult to come up with a common comprehensive benchmark that allows for comparisons among them to facilitate their selection by practitioners, and more importantly, for their continuous improvement by the teams that maintain them. In this paper we present GTFS-Madrid-Bench, a benchmark to evaluate OBDI engines that can be used for the provision of access mechanisms to virtual knowledge graphs. Our proposal introduces several scenarios that aim at measuring the query capabilities, performance and scalability of all these engines, considering their heterogeneity. The data sources used in our benchmark are derived from the GTFS data files of the subway network of Madrid. They have been transformed into several formats (CSV, JSON, SQL and XML) and scaled up. The query set aims at addressing a representative number of SPARQL 1.1 features while covering usual queries that data consumers may be interested in.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100596","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72708559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fine-Grained Entity Linking","authors":"Henry Rosales-Méndez,&nbsp;Aidan Hogan,&nbsp;Barbara Poblete","doi":"10.1016/j.websem.2020.100600","DOIUrl":"10.1016/j.websem.2020.100600","url":null,"abstract":"<div><p><span>The Entity Linking (EL) task involves linking mentions of entities in a text with their identifier in a Knowledge Base (KB) such as Wikipedia, BabelNet, DBpedia, Freebase, Wikidata, YAGO, etc. Numerous techniques have been proposed to address this task down through the years. However, not all works adopt the same convention regarding the entities that the EL task should target; for example, while some EL works target common entities like “interview” appearing in the KB, others only target named entities like “Michael Jackson”. The lack of consensus on this issue (and others) complicates research on the EL task; for example, how can the performance of EL systems be evaluated and compared when systems may target different types of entities? In this work, we first design a questionnaire to understand what kinds of mentions and links the EL research community believes should be targeted by the task. Based on these results we propose a fine-grained categorization scheme for EL that distinguishes different types of mentions and links. We propose a vocabulary extension that allows to express such categories in EL benchmark datasets. We then relabel (subsets of) three popular EL datasets according to our novel categorization scheme, where we additionally discuss a tool used to semi-automate the labeling process. We next present the performance results of five EL systems for individual categories. We further extend EL systems with Word Sense Disambiguation and Coreference Resolution components, creating initial versions of what we call </span><em>Fine-Grained Entity Linking</em> (<em>FEL</em><span>) systems, measuring the impact on performance per category. Finally, we propose a configurable performance measure based on fuzzy sets that can be adapted for different application scenarios Our results highlight a lack of consensus on the goals of the EL task, show that the evaluated systems do indeed target different entities, and further reveal some open challenges for the (F)EL task regarding more complex forms of reference for entities.</span></p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100600","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90166175","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"No one is perfect: Analysing the performance of question answering components over the DBpedia knowledge graph","authors":"Kuldeep Singh ,&nbsp;Ioanna Lytra ,&nbsp;Arun Sethupat Radhakrishna ,&nbsp;Saeedeh Shekarpour ,&nbsp;Maria-Esther Vidal ,&nbsp;Jens Lehmann","doi":"10.1016/j.websem.2020.100594","DOIUrl":"10.1016/j.websem.2020.100594","url":null,"abstract":"<div><p><span>Question answering (QA) over knowledge graphs has gained significant momentum over the past five years due to the increasing availability of large knowledge graphs and the rising importance of Question Answering for user interaction. Existing QA systems have been extensively evaluated as black boxes and their performance has been characterised in terms of average results over all the questions of </span>benchmarking datasets (i.e. macro evaluation). Albeit informative, macro evaluation studies do not provide evidence about QA components’ strengths and concrete weaknesses. Therefore, the objective of this article is to analyse and micro evaluate available QA components in order to comprehend which question characteristics impact on their performance. For this, we measure at question level and with respect to different question features the accuracy of 29 components reused in QA frameworks for the DBpedia knowledge graph using state-of-the-art benchmarks. As a result, we provide a perspective on collective failure cases, study the similarities and synergies among QA components for different component types and suggest their characteristics preventing them from effectively solving the corresponding QA tasks. Finally, based on these extensive results, we present conclusive insights for future challenges and research directions in the field of Question Answering over knowledge graphs.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100594","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82872795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"FAT-RE: A faster dependency-free model for relation extraction","authors":"Lifang Ding ,&nbsp;Zeyang Lei ,&nbsp;Guangxu Xun ,&nbsp;Yujiu Yang","doi":"10.1016/j.websem.2020.100598","DOIUrl":"10.1016/j.websem.2020.100598","url":null,"abstract":"<div><p>Recent years have seen the dependency tree as effective information for relation extraction. Two problems still exist in previous methods: (1) dependency tree relies on external tools and needs to be carefully integrated with a trade-off between pruning noisy words and keeping semantic integrity; (2) dependency-based methods still have to encode sequential context as a supplement, which needs extra time. To tackle the two problems, we propose a faster dependency-free model in this paper: regarding the sentence as a fully-connected graph, we customize the vanilla transformer architecture to remove the irrelevant information via filtering mechanism and further aggregate the sentence information through the enhanced query. Our model yields comparable results on the SemEval2010 Task8 dataset and better results on the TACRED dataset, without requiring external information from the dependency tree but with improved time efficiency.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100598","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86122123","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Knowledge-driven joint posterior revision of named entity classification and linking","authors":"Marco Rospocher ,&nbsp;Francesco Corcoglioniti","doi":"10.1016/j.websem.2020.100617","DOIUrl":"10.1016/j.websem.2020.100617","url":null,"abstract":"<div><p>In this work we address the problem of extracting quality entity knowledge from natural language text, an important task for the automatic construction of knowledge graphs from unstructured content.</p><p><span><span>More in details, we investigate the benefit of performing a joint posterior revision, driven by ontological background knowledge, of the annotations resulting from </span>natural language processing<span> (NLP) entity analyses such as named entity recognition<span> and classification (NERC) and entity linking (EL). The revision is performed via a probabilistic model, called </span></span></span><span>jpark</span><span>, that given the candidate annotations independently identified by NERC and EL tools on the same textual entity mention, reconsiders the best annotation choice performed by the tools in light of the coherence of the candidate annotations with the ontological knowledge. The model can be explicitly instructed to handle the information that an entity can potentially be NIL (i.e., lacking a corresponding referent in the target linking knowledge base), exploiting it for predicting the best NERC and EL annotation combination.</span></p><p>We present a comprehensive evaluation of <span>jpark</span> along various dimensions, comparing its performances with and without exploiting NIL information, as well as the usage of three different background knowledge resources (YAGO, DBpedia, and Wikidata) to build the model. The evaluation, conducted using different tools (the popular Stanford NER and DBpedia Spotlight, as well as the more recent Flair NER and End-to-End Neural EL) with three reference datasets (AIDA, MEANTIME, and TAC-KBP), empirically confirms the capability of the model to improve the quality of the annotations of the given tools, and thus their performances on the tasks they are designed for.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100617","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90776390","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A benchmark for end-user structured data exploration and search user interfaces","authors":"Roberto García ,&nbsp;Rosa Gil ,&nbsp;Eirik Bakke ,&nbsp;David R. Karger","doi":"10.1016/j.websem.2020.100610","DOIUrl":"10.1016/j.websem.2020.100610","url":null,"abstract":"<div><p>During the years, it has been possible to assess significant improvements in the computational efficiency of Semantic Web search<span><span> and exploration systems. However, it has been much harder to assess how well different semantic systems’ user interfaces help their users. One of the key factors facilitating the advancement of research in a particular field is the ability to compare the performance of different approaches. Though there are many such benchmarks in Semantic Web fields that have experienced significant improvements, this is not the case for Semantic Web user interfaces for data exploration. We propose and demonstrate the use of a benchmark for evaluating such user interfaces, which includes a set of typical user tasks and a well-defined procedure for assigning a </span>measure of performance<span> on those tasks to a semantic system. We have applied the benchmark to four such systems. Moreover, all the required resources to apply the benchmark are openly available online. We intend to initiate a community conversation that will lead to a generally accepted framework for comparing systems and for measuring, and thus encouraging, progress towards better semantic search and exploration tools.</span></span></p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100610","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81148276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"IQA: Interactive query construction in semantic question answering systems","authors":"Hamid Zafar ,&nbsp;Mohnish Dubey ,&nbsp;Jens Lehmann ,&nbsp;Elena Demidova","doi":"10.1016/j.websem.2020.100586","DOIUrl":"10.1016/j.websem.2020.100586","url":null,"abstract":"<div><p>Semantic Question Answering (SQA) systems automatically interpret user questions expressed in a natural language in terms of semantic queries. This process involves uncertainty, such that the resulting queries do not always accurately match the user intent, especially for more complex and less common questions. In this article, we aim to empower users in guiding SQA systems towards the intended semantic queries through interaction. We introduce IQA — an interaction scheme for SQA pipelines. This scheme facilitates seamless integration of user feedback in the question answering process and relies on Option Gain — a novel metric that enables efficient and intuitive user interaction. Our evaluation shows that using the proposed scheme, even a small number of user interactions can lead to significant improvements in the performance of SQA systems.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100586","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87741624","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Benchmarking neural embeddings for link prediction in knowledge graphs under semantic and structural changes","authors":"Asan Agibetov ,&nbsp;Matthias Samwald","doi":"10.1016/j.websem.2020.100590","DOIUrl":"10.1016/j.websem.2020.100590","url":null,"abstract":"<div><p>Recently, link prediction algorithms based on neural embeddings have gained tremendous popularity in the Semantic Web community, and are extensively used for knowledge graph completion. While algorithmic advances have strongly focused on efficient ways of learning embeddings, fewer attention has been drawn to the different ways their performance and robustness can be evaluated. In this work we propose an open-source evaluation pipeline, which benchmarks the accuracy of neural embeddings in situations where knowledge graphs may experience semantic and structural changes. We define relation-centric connectivity measures that allow us to connect the link prediction capacity to the structure of the knowledge graph. Such an evaluation pipeline is especially important to simulate the accuracy of embeddings for knowledge graphs that are expected to be frequently updated.</p></div>","PeriodicalId":49951,"journal":{"name":"Journal of Web Semantics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.websem.2020.100590","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87784043","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}