Migrating 120,000 Legacy Publications from Several Systems into a Current Research Information System Using Advanced Data Wrangling Techniques

IF 4.6 Q1 INFORMATION SCIENCE & LIBRARY SCIENCE

Publications Pub Date : 2023-11-14 DOI:10.3390/publications11040049

Yrjö Lappalainen, Matti Lassila, Tanja Heikkilä, Jani Nieminen, Tapani Lehtilä

{"title":"Migrating 120,000 Legacy Publications from Several Systems into a Current Research Information System Using Advanced Data Wrangling Techniques","authors":"Yrjö Lappalainen, Matti Lassila, Tanja Heikkilä, Jani Nieminen, Tapani Lehtilä","doi":"10.3390/publications11040049","DOIUrl":null,"url":null,"abstract":"This article describes a complex CRIS (current research information system) implementation project involving the migration of around 120,000 legacy publication records from three different systems. The project, undertaken by Tampere University, encountered several challenges in data diversity, data quality, and resource allocation. To handle the extensive and heterogenous dataset, innovative approaches such as machine learning techniques and various data wrangling tools were used to process data, correct errors, and merge information from different sources. Despite significant delays and unforeseen obstacles, the project was ultimately successful in achieving its goals. The project served as a valuable learning experience, highlighting the importance of data quality and standardized practices, and the need for dedicated resources in handling complex data migration projects in research organizations. This study stands out for its comprehensive documentation of the data wrangling and migration process, which has been less explored in the context of CRIS literature.","PeriodicalId":37551,"journal":{"name":"Publications","volume":null,"pages":null},"PeriodicalIF":4.6000,"publicationDate":"2023-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Publications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3390/publications11040049","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"INFORMATION SCIENCE & LIBRARY SCIENCE","Score":null,"Total":0}

引用次数: 0

Abstract

This article describes a complex CRIS (current research information system) implementation project involving the migration of around 120,000 legacy publication records from three different systems. The project, undertaken by Tampere University, encountered several challenges in data diversity, data quality, and resource allocation. To handle the extensive and heterogenous dataset, innovative approaches such as machine learning techniques and various data wrangling tools were used to process data, correct errors, and merge information from different sources. Despite significant delays and unforeseen obstacles, the project was ultimately successful in achieving its goals. The project served as a valuable learning experience, highlighting the importance of data quality and standardized practices, and the need for dedicated resources in handling complex data migration projects in research organizations. This study stands out for its comprehensive documentation of the data wrangling and migration process, which has been less explored in the context of CRIS literature.

查看原文本刊更多论文

使用先进的数据整理技术将12万份遗留出版物从几个系统迁移到当前的研究信息系统

本文描述了一个复杂的CRIS(当前研究信息系统)实现项目，涉及从三个不同系统迁移大约120,000个遗留出版记录。该项目由坦佩雷大学承担，在数据多样性、数据质量和资源分配方面遇到了一些挑战。为了处理广泛而异构的数据集，使用了机器学习技术和各种数据整理工具等创新方法来处理数据，纠正错误，并合并来自不同来源的信息。尽管有重大的延误和不可预见的障碍，该项目最终成功地实现了其目标。该项目提供了宝贵的学习经验，突出了数据质量和标准化实践的重要性，以及在研究组织中处理复杂数据迁移项目时需要专用资源。本研究因其对数据整理和迁移过程的全面记录而脱颖而出，这在CRIS文献的背景下探索较少。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Publications Social Sciences-Library and Information Sciences

CiteScore

6.50

自引率

1.90%

发文量

审稿时长

11 weeks

期刊介绍： The scope of Publications includes: Theory and practice of scholarly communication Digitisation and innovations in scholarly publishing technologies Metadata, infrastructure, and linking the scholarly record Publishing policies and editorial/peer-review workflows Financial models for scholarly publishing Copyright, licensing and legal issues in publishing Research integrity and publication ethics Issues and best practices in the publication of non-traditional research outputs (e.g., data, software/code, protocols, data management plans, grant proposals, etc.) Issues in the transition to open access and open science Inclusion and participation of traditionally excluded actors Language issues in publication processes and products Traditional and alternative models of peer review Traditional and alternative means of assessment and evaluation of research and its impact, including bibliometrics and scientometrics The place of research libraries, scholarly societies, funders and others in scholarly communication.