Exploring user engagement by diagnosing visual guides in onboarding screens with linear regression and XGBoost

Onboarding screens are regarded as the first service point when a user experiences a new application, which presents the key functions and features of such an application. The User Interface (UI) walkthroughs, product tours, and tooltips are three common categories of visual guides (VGs) in the onboarding screens for users to get familiar with the app. It is important to offer first-time users appropriate VG to explain the key functions in the app interface. In this paper, we study the effective VG elements that help users adapt to the app UI. We first crowd-sourced user engagement (UE) assessments, and collected 7,080 responses reflecting user cognitive preferences to 114 collected apps containing 1,194 visual guides. Our analytics of the responses shows the improvement of VG following the analysis in three perspectives (types of UI elements, semantic, and spatial analysis). Accordingly, the proposed Parallel Boosted Regression Trees resulted in a highly accurate rating (85%) of the VGs into a three-level UE score, providing app designers useful hints on designing VGs for high levels of user retention and user engagement.