Software documentation is not enough! Requirements for the documentation of AI

Purpose Artificial intelligence (AI) is currently one of the most disruptive technologies and can be applied in many different use cases. However, applying AI in regulated environments is challenging, as it is currently not clear how to achieve and assess the fairness, accountability and transparency (FAT) of AI. Documentation is one promising governance mechanism to ensure that AI is FAT when it is applied in practice. However, due to the nature of AI, documentation standards from software engineering are not suitable to collect the required evidence. Even though FAT AI is called for by lawmakers, academics and practitioners, suitable guidelines on how to document AI are not available. This interview study aims to investigate the requirements for AI documentations. Design/methodology/approach A total of 16 interviews were conducted with senior employees from companies in the banking and IT industry as well as with consultants. The interviews were then analyzed using an informed-inductive coding approach. Findings The authors found five requirements for AI documentation, taking the specific nature of AI into account. The interviews show that documenting AI is not a purely technical task, but also requires engineers to present information on how the AI is understandably integrated into the business process. Originality/value This paper benefits from the unique insights of senior employees into the documentation of AI.