Usability comparison scoring of video and direct laryngoscopes

Lotlikar recently demonstrated that the perceived usability of Macintosh-shaped videolaryngoscopes was inferior to direct laryngoscopes of the same profile [1]. While I fully agree that the design of tools used by our speciality is commonly overlooked and provides substantial opportunity to improve safety and efficiency, the comparison in this instance needs further context and is only a small part of the story.

The very nature of the two laryngoscopes compared are of different complexities with similar but not identical modes of use. A direct laryngoscope is a simple device with a handle, blade, batteries and light source. In comparison, a videolaryngoscope has complex electronics including a fibreoptic camera and image processor in addition to the familiar physical components of its ‘analogue’ partner.

Arguably, a similar comparison could be made, for example, between physically typed letters and emails – one requiring a typewriter and paper, the other a computer with an email user interface, internet connection, screens and a multitude of complicated components. In this case, both require similar physical steps of pressing keys to communicate an idea, but the broader task is potentially much more complex when the functioning of electronic devices is included.

The interpretation of the system usability scale should be undertaken cautiously [2]. It was initially derived in the 1980s to aid comparison of similar computer hardware and software products and through the development cycle. However, it has since become ubiquitous and valuable as a ‘quick and dirty’, adaptable human factors assessment tool. Scores are graded on a sigmoid-shaped curve (Fig. 1) with nearly all scores being found in the 40–80 point range and little perceived improvement over the upper 20 points [3]. Given the simplicity and duration of use of the direct laryngoscope (first introduced into practice in 1944), it is unsurprising that it scored exceptionally in the top 5% (A+ or Best imaginable), whereas the videolaryngoscope, despite an apparently high mark, scored barely in the top 50% (C+ or Good). Asking the question “I found the system unnecessarily complex” when comparing a simple device with a complicated one that achieves the same outcome is perhaps unfair. Indeed, in a previous study, a comparison between desk and mobile (cellular) phones showed statistically significantly higher system usability scale scores with the analogue versions. This outlines the imbalance of such direct score comparisons with similar devices of different technological eras [4].

The introduction of videolaryngoscopes has been a leap forward in the safe management of airways and in the last few years there has been a frenzy of comparisons between direct- and videolaryngoscopes [5, 6]. While we should always strive to improve the design, usability and safety of our tools, this simple comparison of one device against another belies the opportunities afforded by the technology and improved success rate. Merely boiling perceived usability down to a score fails to recognise these benefits. Future usability comparisons between videolaryngoscopes rather than with direct laryngoscopes will provide useful information but it should not be the only method to assess the utility of these devices. Usability needs to be understood in context and include effectiveness, efficiency and (as measured by the system usability scale) satisfaction. Success rates, time taken and additional patient-reported measures must remain the primary metrics for evaluation of airway devices.