Toward a Level Playing Field: An Analysis of the Language and Transparency of IT Risk

Abstract

We employ both manual and automated content analysis to update an existing dictionary (Boritz, Hayes, and Lim 2013) used to classify types of information technology weakness (ITW). Through analysis of auditors’ reports on internal control filed under Section 404 of the Sarbanes-Oxley Act from 2011 through 2020, we provide a robust tool for textual analytics. Additionally, we apply OpenAI’s large-language model neural networks for classification as a comparison point. We expand identification of ITW categories such as Design, Security, and Outsourcing and identify Governance as a critical new category in auditor reporting. We further expand the dictionary, capturing categories of IT risk found within current IT control frameworks (e.g., COBIT and AICPA TSCs), which we compare with content of auditor reporting. Evidence suggests stakeholders are subject to significant information asymmetry in assessing and reporting IT risk. We provide direction for future IT governance and improve application of advanced textual analysis.