Integration patterns in the use of metadata for data sense-making during relevance evaluation: An interpretable deep learning-based prediction

Abstract

Integrating diverse cues from metadata to make sense of retrieved data during relevance evaluation is a crucial yet challenging task for data searchers. However, this integrative task remains underexplored, impeding the development of effective strategies to address metadata's shortcomings in supporting this task. To address this issue, this study proposes the “Integrative Use of Metadata for Data Sense-Making” (IUM-DSM) model. This model provides an initial framework for understanding the integrative tasks performed by data searchers, focusing on their integration patterns and associated challenges. Experimental data were analyzed using an interpretable deep learning-based prediction approach to validate this model. The findings offer preliminary support for the model, revealing that data searchers engage in integrative tasks to utilize metadata effectively for data sense-making during relevance evaluation. They construct coherent mental representations of retrieved data by integrating systematic and heuristic cues from metadata through two distinct patterns: within-category integration and across-category integration. This study identifies key challenges: within-category integration entails comparing, classifying, and connecting systematic or heuristic cues, while across-category integration necessitates considerable effort to integrate cues from both categories. To support these integrative tasks, this study proposes strategies for mitigating these challenges by optimizing metadata layouts and developing intelligent data retrieval systems.