Automatic Generation of Test Cases for REST APIs: A Specification-Based Approach

The REpresentation State Transfer (REST) has gained momentum as the preferred technique to design Web APIs. REST allows building loosely coupled systems by relying on HTTP and the Web-friendly format JSON. However, REST is not backed by any standard or specification to describe how to create/consume REST APIs, thus creating new challenges for their integration, testing and verification. To face this situation, several specification formats have been proposed (e.g., OpenAPI, RAML, and API Blueprint), which can help automate tasks in REST API development (e.g., testing) and consumption (e.g., SDKs generation). In this paper we focus on automated REST API testing relying on API specifications, and particularly the OpenAPI one. We propose an approach to generate specification-based test cases for REST APIs to make sure that such APIs meet the requirements defined in their specifications. We provide a proof-of-concept tool implementing our approach, which we have validated with 91 OpenAPI definitions. Our experiments show that the generated test cases cover on average 76.5% of the elements included in the OpenAPI definitions. Furthermore, our experiments also reveal that 40% of the tested APIs fail.