Towards a measurement framework for tools' ceiling and threshold

Abstract

Software development tools are not catching up with the requirements of increasingly complex interactive software products and services. Successful tools are claimed to either be low-threshold/low-ceiling or high-threshold/high-ceiling, however no research to date addressed how to define and measure these concepts. This is increasingly important as these tools undergo an evaluation and adoption process by end-users. Here we hypothesized that the evaluation and adoption of tools is associated with the threshold (learnability). To assess this we conducted a learnability and usability study using three commercial Platform-as-a-Service tools. In this study we used an augmented think-aloud protocol with question asking where ten subjects were asked to create a simple web application. Our data shows that most learnability issues fall into two categories: understanding or locating. No evidence was found that usability defects correlate with the tools learnability score. Though we found an inverse correlation between the amount of issues and the learnability score.