Decoding virtual chats: NLP insights into academic library services.

Abstract

Assessing unstructured data from virtual reference chats is complex. Full-text reveals nuances but is time-consuming, while transcript metadata gives an overview but may miss important details in the conversation. This research applies a machine learning (ML) tool to the complete set of transcripts from a research university's chat reference service (2017–2022) to examine evolving trends and patron needs in the library reference service. The study has two key objectives: 1) demonstrating ML's effectiveness in the academic library setting, and 2) assessing the impact of COVID-19 on chat reference needs. A text classification model, trained on 1.5 % of the sample, achieves a 75 % accuracy match with human annotations. Findings indicate a marked rise in circulation-related inquiries as libraries transitioned to fully online services during the pandemic. Notably, user behaviors remain consistent even after the pandemic. This study highlights ML's potential to analyze large-scale unstructured data effectively in the academic library setting.