API peer reviews: a method for evaluating usability of application programming interfaces

Abstract

API usability tests in the lab are time and resource intensive, thus allowing a relatively small percentage of the API namespace to be evaluated. We describe a group-based usability inspection method -- API Peer Reviews -- to evaluate API usability. Based on an analysis of usability breakdowns from API Peer Reviews and API usability tests, results show that API Peer Reviews identified breakdowns across several cognitive dimensions, some of which were different than what was identified by API usability tests. We reflect on the adoption of API Peer Reviews as a collaborative practice in organizations for evaluating API usability.