How Well Static Type Checkers Work with Gradual Typing? A Case Study on Python

Wenjie Xu, Lin Chen, Chenghao Su, Yimeng Guo, Yanhui Li, Yuming Zhou, Baowen Xu
{"title":"How Well Static Type Checkers Work with Gradual Typing? A Case Study on Python","authors":"Wenjie Xu, Lin Chen, Chenghao Su, Yimeng Guo, Yanhui Li, Yuming Zhou, Baowen Xu","doi":"10.1109/ICPC58990.2023.00039","DOIUrl":null,"url":null,"abstract":"Python has become increasingly popular and widely used in many fields. Dynamic features of Python provide much convenience for developers. However, they can also cause many type-related bugs undetected until runtime, which increases the cost of maintenance. Static type checking is essential to find bugs early, and the introduction of gradual typing and type annotations makes it easier to perform static type analysis. However, it remains to be investigated how well gradual typing improves real bug detection. Therefore, we conducted a comprehensive study on three widely used checkers: MyPy, PyRight, and PyType. We used a benchmark containing 10 popular Python projects with 40 real type-related bugs. First, we performed static type checking on the projects with and without type annotations to evaluate the effectiveness of finding real bugs. Second, we manually analyzed the missing bugs and investigated the reasons. The results show that the three tools can detect 29 of the 40 studied bugs after annotating, while only 14 bugs are detected before annotating. We also found that type annotations can substantially improve the ability of static type checkers to detect real bugs. A detailed analysis of bugs missed by the checkers shows that: (i) the accuracy of type analysis is challenged when it comes to programs with complicated dynamic features, such as dynamically changing object’s attributes, even with annotations; (ii) the inaccurate type annotations can undermine the ability of static type checkers to detect real bugs; (iii) static type checkers have different checking strategies in some cases, which has an impact on real bug detection. Our study can not only enable developers to better understand static type checking and make better use of them but also guide future research.","PeriodicalId":376593,"journal":{"name":"2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICPC58990.2023.00039","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Python has become increasingly popular and widely used in many fields. Dynamic features of Python provide much convenience for developers. However, they can also cause many type-related bugs undetected until runtime, which increases the cost of maintenance. Static type checking is essential to find bugs early, and the introduction of gradual typing and type annotations makes it easier to perform static type analysis. However, it remains to be investigated how well gradual typing improves real bug detection. Therefore, we conducted a comprehensive study on three widely used checkers: MyPy, PyRight, and PyType. We used a benchmark containing 10 popular Python projects with 40 real type-related bugs. First, we performed static type checking on the projects with and without type annotations to evaluate the effectiveness of finding real bugs. Second, we manually analyzed the missing bugs and investigated the reasons. The results show that the three tools can detect 29 of the 40 studied bugs after annotating, while only 14 bugs are detected before annotating. We also found that type annotations can substantially improve the ability of static type checkers to detect real bugs. A detailed analysis of bugs missed by the checkers shows that: (i) the accuracy of type analysis is challenged when it comes to programs with complicated dynamic features, such as dynamically changing object’s attributes, even with annotations; (ii) the inaccurate type annotations can undermine the ability of static type checkers to detect real bugs; (iii) static type checkers have different checking strategies in some cases, which has an impact on real bug detection. Our study can not only enable developers to better understand static type checking and make better use of them but also guide future research.
静态类型检查器如何与渐进类型工作?Python的案例研究
Python已经变得越来越流行,并广泛应用于许多领域。Python的动态特性为开发人员提供了很多便利。然而,它们也可能导致许多直到运行时才被发现的与类型相关的错误,这增加了维护成本。静态类型检查对于早期发现bug至关重要,渐进类型和类型注释的引入使执行静态类型分析变得更加容易。然而,渐进式类型在多大程度上改善了真正的bug检测,这仍有待研究。因此,我们对三种广泛使用的检查器:MyPy, PyRight和PyType进行了全面的研究。我们使用了一个包含10个流行Python项目的基准测试,其中包含40个与类型相关的bug。首先,我们对带有和不带有类型注释的项目执行静态类型检查,以评估发现真正bug的有效性。其次,我们手工分析缺失的bug并调查其原因。结果表明,这三种工具在注释后可以检测到所研究的40个bug中的29个,而在注释前仅检测到14个。我们还发现,类型注释可以大大提高静态类型检查器检测真正bug的能力。对检查器遗漏的错误的详细分析表明:(i)当涉及到具有复杂动态特性的程序时,类型分析的准确性受到挑战,例如动态更改对象的属性,即使有注释;(ii)不准确的类型注释会破坏静态类型检查器检测真正bug的能力;(iii)静态类型检查器在某些情况下有不同的检查策略,这对真正的bug检测有影响。我们的研究不仅可以使开发人员更好地理解和使用静态类型检查,而且可以指导未来的研究。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信