Assessing Trustworthiness

Measuring the more than thirty attributes of HCAI systems, like reliability, safety, and trustworthiness is very difficult. Objective measures seem unlikely, but subjective impressions from stakeholders such as researchers, developers, managers, and users may help to clarify whether the trustworthiness is increased or decreased after changes have been made, or to compare two competing products, such as is done by winetasting. The HCAI Trustworthiness Assessment Process suggests a way of developing a subjective evaluation scale. The proposed twelve-item HCAI Trustworthiness Scale relies on questions about whether a system has an audit trail, has passed software and fairness tests, and includes explainable user interfaces. It also has questions about internal and external review processes.