The application of explainable artificial intelligence methods to models for automatic creativity assessment.

Abstract

Objective: The study is devoted to comparing various models based on Artificial Intelligence to determine the level of creativity based on drawings performed using the Urban test, as well as analyzing the results of applying explainable artificial intelligence methods to a trained model to identify the most relevant features in drawings that influence the model's prediction.

Methods: The dataset is represented by a set of 1,823 scanned forms of drawings of participants performed according to the Urban test. The test results of each participant were assessed by an expert. Preprocessed images were used for fine-tuning pre-trained models such as MobileNet, ResNet18, AlexNet, DenseNet, ResNext, EfficientNet, ViT with additional linear layers to predict the participant's score. Visualization of the areas that are of greatest importance from the point of view of the model was carried out using the Gradient-weighted Class Activation Mapping (Grad-CAM) method.

Results: Trained models based on MobileNet showed the highest prediction accuracy rate of 76%. The results of the application of explainable artificial intelligence demonstrated areas of interest that correlated with the criteria for expert assessment according to the Urban test. Analysis of erroneous predictions of the model in terms of interpretation of areas of interest made it possible to clarify the features of the drawing on which the model relies, contrary to the expert.

Conclusion: The study demonstrated the possibility of using neural network methods for automated diagnosis of the level of creativity according to the Urban test based on the respondents' drawings. The application of explainable artificial intelligence methods to the trained model demonstrated the compliance of the identified activation zones with the rules of expert assessment according to the Urban test.