Human Versus Machine

Abstract

This article implements a method for classifying adverse incidents involving reusable medical devices according to their underlying cause and assesses the level of agreement between different raters. To achieve this, the adverse incidents were classified into 1 or more of 62 separate categories, and the level of agreement between 3 experienced human raters was established. Moreover, the ChatGPT artificial intelligence tool was used to replicate the classification process used by human raters. The results showed that there was a fair level of agreement between human raters and a slight agreement between human raters and ChatGPT. This suggests that, although ChatGPT can intelligently classify adverse incidents, it was not able to replicate the performance of experienced human raters when given access to only the limited incident details and classification categories as provided for in this study.