Annotations as Knowledge Practices in Image Archives: Application of Linked Open Usable Data and Machine Learning

IF 2.1 3区计算机科学 Q3 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

ACM Journal on Computing and Cultural Heritage Pub Date : 2023-09-26 DOI:10.1145/3625301

Murielle Cornut, Julien Antoine Raemy, Florian Spiess

{"title":"Annotations as Knowledge Practices in Image Archives: Application of Linked Open Usable Data and Machine Learning","authors":"Murielle Cornut, Julien Antoine Raemy, Florian Spiess","doi":"10.1145/3625301","DOIUrl":null,"url":null,"abstract":"We reflect on some of the preliminary findings of the Participatory Knowledge Practices in Analogue and Digital Image Archives (PIA) research project around annotations of photographic archives from the Swiss Society for Folklore Studies (SSFS) as knowledge practices, the underlying technological decisions, and their impact. The aim is not only to seek more information but to find new approaches of understanding the way in which people’s memory relate to the collective, public form of archival memory and ultimately how users figure in and shape the digital archive. We provide a proof-of-concept workflow based on automatically generated annotations comprising 53,481 photos that were subjected to object detection using Faster R-CNN Inception ResNet V2. Of the detected objects, 184,609 have a detection score greater than 0.5, 123,529 have a score greater than 0.75, and 88,442 have a score greater than 0.9. A threshold of 0.75 was set for the dissemination of our annotations, compatible with the W3C Web Annotation Data Model (WADM) and embedded in our IIIF Manifests. In the near future, the workflow will be upgraded to allow for the co-existence of various, and occasionally conflicting, assertions made by both human and machine users. We believe that Linked Open Usable Data (LOUD) standards should be used to improve the sustainability of such an ecosystem and to foster collaboration between actors in cultural heritage.","PeriodicalId":54310,"journal":{"name":"ACM Journal on Computing and Cultural Heritage","volume":"89 1","pages":"0"},"PeriodicalIF":2.1000,"publicationDate":"2023-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACM Journal on Computing and Cultural Heritage","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3625301","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}

引用次数: 0

Abstract

We reflect on some of the preliminary findings of the Participatory Knowledge Practices in Analogue and Digital Image Archives (PIA) research project around annotations of photographic archives from the Swiss Society for Folklore Studies (SSFS) as knowledge practices, the underlying technological decisions, and their impact. The aim is not only to seek more information but to find new approaches of understanding the way in which people’s memory relate to the collective, public form of archival memory and ultimately how users figure in and shape the digital archive. We provide a proof-of-concept workflow based on automatically generated annotations comprising 53,481 photos that were subjected to object detection using Faster R-CNN Inception ResNet V2. Of the detected objects, 184,609 have a detection score greater than 0.5, 123,529 have a score greater than 0.75, and 88,442 have a score greater than 0.9. A threshold of 0.75 was set for the dissemination of our annotations, compatible with the W3C Web Annotation Data Model (WADM) and embedded in our IIIF Manifests. In the near future, the workflow will be upgraded to allow for the co-existence of various, and occasionally conflicting, assertions made by both human and machine users. We believe that Linked Open Usable Data (LOUD) standards should be used to improve the sustainability of such an ecosystem and to foster collaboration between actors in cultural heritage.

查看原文本刊更多论文

注释作为图像档案中的知识实践:链接开放可用数据和机器学习的应用

我们反思了模拟和数字图像档案(PIA)研究项目中参与式知识实践的一些初步发现，这些研究项目围绕瑞士民俗研究协会(SSFS)的摄影档案注释作为知识实践，潜在的技术决策及其影响。其目的不仅是寻找更多的信息，而且是找到新的方法来理解人们的记忆与档案记忆的集体、公共形式之间的关系，以及最终用户如何参与和塑造数字档案。我们提供了一个基于自动生成注释的概念验证工作流，其中包含53,481张照片，这些照片使用Faster R-CNN Inception ResNet V2进行对象检测。检测到的对象中，检测得分大于0.5的有184,609个，大于0.75的有123,529个，大于0.9的有88,442个。我们为注释的传播设置了0.75的阈值，与W3C Web注释数据模型(WADM)兼容，并嵌入到IIIF manifest中。在不久的将来，工作流将被升级，以允许人类和机器用户所做的各种断言(偶尔会发生冲突)共存。我们认为，应使用关联开放可用数据(LOUD)标准来提高这种生态系统的可持续性，并促进文化遗产参与者之间的合作。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

ACM Journal on Computing and Cultural Heritage Arts and Humanities-Conservation

CiteScore

4.60

自引率

8.30%

发文量

期刊介绍： ACM Journal on Computing and Cultural Heritage (JOCCH) publishes papers of significant and lasting value in all areas relating to the use of information and communication technologies (ICT) in support of Cultural Heritage. The journal encourages the submission of manuscripts that demonstrate innovative use of technology for the discovery, analysis, interpretation and presentation of cultural material, as well as manuscripts that illustrate applications in the Cultural Heritage sector that challenge the computational technologies and suggest new research opportunities in computer science.