The AILA Methodology for Automated and Intelligent Likelihood Assignment

Abstract

Risk assessment is core to any institution's evaluation of risk, notably for what concerns people's privacy. The assessment often relies on information stated in a policy shaped as a text document. The risk assessor, or analyst in brief, is called to understand documentation that can be long, unclear or incomplete, hence subjectivity or distraction may strongly influence the process, particularly for identifying each relevant asset and for the assignment of the likelihood value of a given threat to an identified asset. The aim of this paper is to reduce the influence of subjectivity and distraction through risk assessment by means of our methodology for the Automated and Intelligent Likelihood Assignment (AILA). While the analyst's role cannot be emptied, it is facilitated through entities identification and likelihood assignment to threats for assets. The methodology adopts Natural Language Processing for summarisation and entity recognition, it tailors fully-supervised Machine Learning over policy documents and it leverages an existing tool supporting risk assessment, PILAR, in order to gain a more objective likelihood assignment. The paper demonstrates AILA over three real-world case studies from the automotive domain, culminating with the risk assessment exercises over the privacy policies of Toyota, Mercedes and Tesla. The executable components of AILA, the AILA Entity Extractor and the AILA Classifier are released as open source.