Boa Meets Python: A Boa Dataset of Data Science Software in Python Language

2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) Pub Date : 2019-05-26 DOI:10.1109/MSR.2019.00086

Sumon Biswas, Md Johirul Islam, Yijia Huang, Hridesh Rajan

{"title":"Boa Meets Python: A Boa Dataset of Data Science Software in Python Language","authors":"Sumon Biswas, Md Johirul Islam, Yijia Huang, Hridesh Rajan","doi":"10.1109/MSR.2019.00086","DOIUrl":null,"url":null,"abstract":"The popularity of Python programming language has surged in recent years due to its increasing usage in Data Science. The availability of Python repositories in Github presents an opportunity for mining software repository research, e.g., suggesting the best practices in developing Data Science applications, identifying bug-patterns, recommending code enhancements, etc. To enable this research, we have created a new dataset that includes 1,558 mature Github projects that develop Python software for Data Science tasks. By analyzing the metadata and code, we have included the projects in our dataset which use a diverse set of machine learning libraries and managed by a variety of users and organizations. The dataset is made publicly available through Boa infrastructure both as a collection of raw projects as well as in a processed form that could be used for performing large scale analysis using Boa language. We also present two initial applications to demonstrate the potential of the dataset that could be leveraged by the community.","PeriodicalId":6706,"journal":{"name":"2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)","volume":"35 1","pages":"577-581"},"PeriodicalIF":0.0000,"publicationDate":"2019-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"21","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/MSR.2019.00086","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 21

Abstract

The popularity of Python programming language has surged in recent years due to its increasing usage in Data Science. The availability of Python repositories in Github presents an opportunity for mining software repository research, e.g., suggesting the best practices in developing Data Science applications, identifying bug-patterns, recommending code enhancements, etc. To enable this research, we have created a new dataset that includes 1,558 mature Github projects that develop Python software for Data Science tasks. By analyzing the metadata and code, we have included the projects in our dataset which use a diverse set of machine learning libraries and managed by a variety of users and organizations. The dataset is made publicly available through Boa infrastructure both as a collection of raw projects as well as in a processed form that could be used for performing large scale analysis using Boa language. We also present two initial applications to demonstrate the potential of the dataset that could be leveraged by the community.

查看原文本刊更多论文

蟒蛇遇见Python: Python语言数据科学软件的蟒蛇数据集

近年来，由于Python编程语言在数据科学中的使用越来越多，它的受欢迎程度激增。Github中Python存储库的可用性为挖掘软件存储库研究提供了机会，例如，建议开发数据科学应用程序的最佳实践，识别错误模式，推荐代码增强等。为了进行这项研究，我们创建了一个新的数据集，其中包括1,558个成熟的Github项目，这些项目为数据科学任务开发Python软件。通过分析元数据和代码，我们将项目包含在我们的数据集中，这些项目使用了一组不同的机器学习库，并由各种用户和组织管理。数据集通过Boa基础设施公开提供，既可以作为原始项目的集合，也可以以处理后的形式用于使用Boa语言执行大规模分析。我们还提出了两个初步的应用程序来展示数据集的潜力，这些潜力可以被社区利用。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)

自引率

0.00%

发文量