{"title":"Look, Read and Enrich - Learning from Scientific Figures and their Captions","authors":"José Manuél Gómez-Pérez, Raúl Ortega","doi":"10.1145/3360901.3364420","DOIUrl":"https://doi.org/10.1145/3360901.3364420","url":null,"abstract":"Compared to natural images, understanding scientific figures is particularly hard for machines. However, there is a valuable source of information in scientific literature that until now has remained untapped: the correspondence between a figure and its caption. In this paper we investigate what can be learnt by looking at a large number of figures and reading their captions, and introduce a figure-caption correspondence learning task that makes use of our observations. Training visual and language networks without supervision other than pairs of unconstrained figures and captions is shown to successfully solve this task. We also show that transferring lexical and semantic knowledge from a knowledge graph significantly enriches the resulting features. Finally, we demonstrate the positive impact of such features in other tasks involving scientific text and figures, like multi-modal classification and machine comprehension for question answering, outperforming supervised baselines and ad-hoc approaches.","PeriodicalId":116830,"journal":{"name":"Proceedings of the 10th International Conference on Knowledge Capture","volume":"128 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127970180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Y. Jaradeh, A. Oelen, K. Farfar, Manuel Prinz, Jennifer D'Souza, G. Kismihók, M. Stocker, S. Auer
{"title":"Open Research Knowledge Graph: Next Generation Infrastructure for Semantic Scholarly Knowledge","authors":"M. Y. Jaradeh, A. Oelen, K. Farfar, Manuel Prinz, Jennifer D'Souza, G. Kismihók, M. Stocker, S. Auer","doi":"10.1145/3360901.3364435","DOIUrl":"https://doi.org/10.1145/3360901.3364435","url":null,"abstract":"Despite improved digital access to scholarly knowledge in recent decades, scholarly communication remains exclusively document-based. In this form, scholarly knowledge is hard to process automatically. We present the first steps towards a knowledge graph based infrastructure that acquires scholarly knowledge in machine actionable form thus enabling new possibilities for scholarly knowledge curation, publication and processing. The primary contribution is to present, evaluate and discuss multi-modal scholarly knowledge acquisition, combining crowdsourced and automated techniques. We present the results of the first user evaluation of the infrastructure with the participants of a recent international conference. Results suggest that users were intrigued by the novelty of the proposed infrastructure and by the possibilities for innovative scholarly knowledge processing it could enable.","PeriodicalId":116830,"journal":{"name":"Proceedings of the 10th International Conference on Knowledge Capture","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131782017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 10th International Conference on Knowledge Capture","authors":"D. Sleeman, Ken Barker","doi":"10.1145/3360901","DOIUrl":"https://doi.org/10.1145/3360901","url":null,"abstract":"We are pleased to welcome you to the Fourth International Conference on Knowledge Capture -- K-Cap 2007. The intellectual focus of the meeting remains the Capture of Knowledge from a variety of sources including human experts, on-line texts, as well as a variety of documents available on the web. This Conference, together with the corresponding European Conference, EKAW, remains the primary meeting at which the knowledge capture research community comes together to discuss recent results and future research directions. \u0000 \u000060 papers were submitted in response to the call for papers; these came from Africa, Asia, Europe, North America, and South America. The Program Committee accepted 20 papers covering a wide range of views and perspectives; a further call for posters resulted in 18 being accepted, these are also featured in the proceedings as short papers. Thirdly, the proceedings include extended abstracts from our invited speakers: Oren Etzioni who is Professor of Computer Science at the University of Washington's Computer Science Department, and the founder and director of the university's Turing Center, and Luis von Ahn who is an Assistant Professor in the Computer Science Department at Carnegie Mellon University. Both contributions are very thought provoking.","PeriodicalId":116830,"journal":{"name":"Proceedings of the 10th International Conference on Knowledge Capture","volume":"256 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132417319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}