Bot or not?: Detecting bots in GitHub pull request activity based on comment similarity

M. Golzadeh, Damien Legay, Alexandre Decan, T. Mens
{"title":"Bot or not?: Detecting bots in GitHub pull request activity based on comment similarity","authors":"M. Golzadeh, Damien Legay, Alexandre Decan, T. Mens","doi":"10.1145/3387940.3391503","DOIUrl":null,"url":null,"abstract":"Many empirical studies focus on socio-technical activity in social coding platforms such as GitHub, for example to study the onboarding, abandonment, productivity and collaboration among team members. Such studies face the difficulty that GitHub activity can also be generated automatically by bots of a different nature. It therefore becomes imperative to distinguish such bots from human users. We propose an automated approach to detect bots in GitHub pull request (PR) activity. Relying on the assumption that bots contain repetitive message patterns in their PR comments, we analyse the similarity between multiple messages from the same GitHub identity, using a clustering method that combines the Jaccard and Levenshtein distance. We empirically evaluate our approach by analysing 20,090 PR comments of 250 users and 42 bots in 1,262 GitHub repositories. Our results show that the method is able to clearly separate bots from human users.","PeriodicalId":309659,"journal":{"name":"Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2020-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"23","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3387940.3391503","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 23

Abstract

Many empirical studies focus on socio-technical activity in social coding platforms such as GitHub, for example to study the onboarding, abandonment, productivity and collaboration among team members. Such studies face the difficulty that GitHub activity can also be generated automatically by bots of a different nature. It therefore becomes imperative to distinguish such bots from human users. We propose an automated approach to detect bots in GitHub pull request (PR) activity. Relying on the assumption that bots contain repetitive message patterns in their PR comments, we analyse the similarity between multiple messages from the same GitHub identity, using a clustering method that combines the Jaccard and Levenshtein distance. We empirically evaluate our approach by analysing 20,090 PR comments of 250 users and 42 bots in 1,262 GitHub repositories. Our results show that the method is able to clearly separate bots from human users.
是还是不是?:基于评论相似度检测GitHub拉请求活动中的机器人
许多实证研究关注社交编码平台(如GitHub)中的社会技术活动,例如研究团队成员之间的入职、放弃、生产力和协作。这类研究面临的困难是,GitHub活动也可以由不同性质的机器人自动生成。因此,必须将这些机器人与人类用户区分开来。我们提出了一种自动化的方法来检测GitHub拉请求(PR)活动中的机器人。基于机器人在其PR评论中包含重复消息模式的假设,我们使用结合Jaccard和Levenshtein距离的聚类方法,分析来自相同GitHub身份的多个消息之间的相似性。我们通过分析1262个GitHub存储库中250个用户和42个机器人的20,090条PR评论来对我们的方法进行实证评估。我们的结果表明,该方法能够清楚地将机器人与人类用户区分开来。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信