Beyond the lab: An in-depth analysis of real-world practices in government-to-citizen software user documentation

Abstract

Context:

Governments, including Switzerland through its Digital Switzerland Strategy, are using new technologies to improve public services. However, unclear user guides often lead people to prefer expensive help desk services. Current research on software documentation is limited by small-scale surveys that do not reflect real-world challenges. This paper addresses these gaps by examining the limitations of user guides in a more practical context.

Objective:

Building on the identified need for a more comprehensive understanding of user documentation in real-world applications, this study aims to critically analyse user documentation in government-to-citizen (G2C) interactions within Switzerland. We intend to identify both common and critical issues in existing documentation to direct future research towards substantial improvements. By doing so, this research will contribute to the development of more effective user guides, ultimately improving the digital experience for citizens and reducing reliance on costly help desk support.

Methods:

Our research methodology involved a thorough analysis of user documentation in German-speaking Swiss cantons. We began with around 5’000 links from official cantonal websites and narrowed it down to nearly 600 user guides relevant to G2C applications. The study progressed in phases: we first assessed the content to identify real-world documentation characteristics, then compared these with common issues from academic research to pinpoint frequent problems. Finally, we analysed the data to identify overarching trends in the documentation characteristics and issues.

Results:

Our analyses, which linked guide features to documentation issues, uncovered prevalent real-world issue trends, characterized by significant statistical correlations ($p<.05$) with the socioeconomic status of the cantons, such as their wealth and population size.

Conclusions:

Identifying these trends will help researchers and practitioners concentrate on the most common and critical issues encountered in practice. This, in turn, holds the potential to drive the development of more effective technology for documenting software. Data and Materials: https://doi.org/10.5281/zenodo.10592871