HFCommunity: An extraction process and relational database to analyze Hugging Face Hub data

IF 1.5 4区计算机科学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Science of Computer Programming Pub Date : 2024-01-10 DOI:10.1016/j.scico.2024.103079

Adem Ait , Javier Luis Cánovas Izquierdo , Jordi Cabot

{"title":"HFCommunity: An extraction process and relational database to analyze Hugging Face Hub data","authors":"Adem Ait , Javier Luis Cánovas Izquierdo , Jordi Cabot","doi":"10.1016/j.scico.2024.103079","DOIUrl":null,"url":null,"abstract":"<div>Social coding platforms such as GitHub or GitLab have become the de facto standard for developing Open-Source Software (OSS) projects. With the emergence of Machine Learning (ML), platforms specifically designed for hosting and developing ML-based projects have appeared, being Hugging Face Hub (HFH) one of the most popular ones. HFH aims at sharing datasets, pre-trained ML models and the applications built with them. With over 400 K repositories, and growing fast, HFH is becoming a promising source of empirical data on all aspects of ML project development. However, apart from the API provided by the platform, there are no easy-to-use solutions to collect the data, nor prepackaged datasets to explore the different facets of HFH. We present HFCommunity, an extraction process for HFH data and a relational database to facilitate an empirical analysis on the growing number of ML projects.</div>","PeriodicalId":49561,"journal":{"name":"Science of Computer Programming","volume":"234 ","pages":"Article 103079"},"PeriodicalIF":1.5000,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0167642324000029/pdfft?md5=bb0c43422124d50d91f987a6ab598504&pid=1-s2.0-S0167642324000029-main.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science of Computer Programming","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167642324000029","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

Social coding platforms such as GitHub or GitLab have become the de facto standard for developing Open-Source Software (OSS) projects. With the emergence of Machine Learning (ML), platforms specifically designed for hosting and developing ML-based projects have appeared, being Hugging Face Hub (HFH) one of the most popular ones. HFH aims at sharing datasets, pre-trained ML models and the applications built with them. With over 400 K repositories, and growing fast, HFH is becoming a promising source of empirical data on all aspects of ML project development. However, apart from the API provided by the platform, there are no easy-to-use solutions to collect the data, nor prepackaged datasets to explore the different facets of HFH. We present HFCommunity, an extraction process for HFH data and a relational database to facilitate an empirical analysis on the growing number of ML projects.

查看原文本刊更多论文

HFCommunity：分析拥抱脸枢纽数据的提取过程和关系数据库

GitHub 或 GitLab 等社交编码平台已成为开发开源软件 (OSS) 项目的事实标准。随着机器学习（ML）的出现，出现了专门用于托管和开发基于 ML 的项目的平台，Hugging Face Hub（HFH）就是其中最受欢迎的平台之一。HFH 旨在共享数据集、预训练的 ML 模型以及使用这些模型构建的应用程序。HFH 拥有超过 400 K 个存储库，并且还在快速增长，它正在成为有关 ML 项目开发各个方面的经验数据的一个有前途的来源。然而，除了该平台提供的应用程序接口（API）外，还没有易于使用的解决方案来收集数据，也没有预先打包的数据集来探索 HFH 的不同方面。我们介绍了 HFCommunity，这是一个提取 HFH 数据的流程，也是一个关系数据库，有助于对日益增多的 ML 项目进行实证分析。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Science of Computer Programming 工程技术-计算机：软件工程

CiteScore

3.80

自引率

0.00%

发文量

审稿时长

67 days

期刊介绍： Science of Computer Programming is dedicated to the distribution of research results in the areas of software systems development, use and maintenance, including the software aspects of hardware design. The journal has a wide scope ranging from the many facets of methodological foundations to the details of technical issues andthe aspects of industrial practice. The subjects of interest to SCP cover the entire spectrum of methods for the entire life cycle of software systems, including • Requirements, specification, design, validation, verification, coding, testing, maintenance, metrics and renovation of software; • Design, implementation and evaluation of programming languages; • Programming environments, development tools, visualisation and animation; • Management of the development process; • Human factors in software, software for social interaction, software for social computing; • Cyber physical systems, and software for the interaction between the physical and the machine; • Software aspects of infrastructure services, system administration, and network management.