Development and application of a heuristic to assess trends in API documentation

Computer technology has made amazing advances in the past few decades; however, the software documentation of today still looks strikingly similar to the software documentation used 30 years ago. If this continues into the 21st century, more and more software developers could be using 20th-century-style documentation to solve 21st-century problems with 21st-century technologies. Is 20th-century-style documentation up to the challenge? How can that be measured? This paper seeks to answer those questions by developing a heuristic to identify whether the documentation set for an application programming interface (API) contains the key elements of API reference documentation that help software developers learn an API. The resulting heuristic was tested on a collection of software documentation that was chosen to provide a diverse set of examples with which to validate the heuristic. In the course of testing the heuristic, interesting patterns in the API documentation were observed. For example, twenty-five percent of the documentation sets studied did not have any overview information, which, according to studies, is one of the most basic elements an API documentation set needs to help software developers learn to use the API. The heuristic produced by this research can be used to evaluate large sets of API documentation, track trends in API documentation, and facilitate additional research.