UX Testing of Developer Documentation - A pilot study of OceanBase Database Documentation

This pilot UX study aims to establish a user experience (UX) approach for assessing the quality of developer documentation, specifically for the OceanBase database company. A usability test was designed to examine task completion rates, times, errors, and satisfaction. Additionally, biometric equipment, such as eye-tracking, EEG, facial expression recognition, and measures of visual and mental fatigue, were utilized to analyze participants’ experiences. The PANAS scale was employed to collect self-reported emotions. The hybrid evaluation method revealed that the OceanBase database documentation suffers from poor usability, understanding, and findability. The root causes of these problems include weak functionality, subpar interaction design, lack of conciseness and explanation, and inadequate structure and legibility. Participants experienced negative emotions and increased mental fatigue after using the documentation, indicating a substantial cognitive load. These findings will inform future improvements to the documentation and provide a foundational UX model for developer documentation. The UX study process may also be a reference for practitioners or researchers conducting similar research on technical documents.