Choice Consistency in Discrete Choice Experiments: Does Numeracy Skill Matter? (VIH-2023-0494.R2).

Abstract

Objectives: This study investigated the relationship between numeracy skills and choice consistency in discrete choice experiments (DCEs).

Methods: A DCE was conducted to explore patients' preferences for kidney transplantation in Italy. Patients completed the DCE and answered three-item numeracy questions. A Heteroskedastic Multinomial Logit (HMNL) model was used to investigate the effect of numeracy on choice consistency.

Results: Higher numeracy skills were associated with greater choice consistency, increasing the scale to 1.63 (p<0.001), 1.39 (p<0.001), and 1.18 (p<0.001) for patients answering 3/3, 2/3, and 1/3 questions correctly, respectively, compared to those with no correct answers. This corresponded to 63%, 39%, and 18% more consistent choices, respectively. Accounting for choice consistency resulted in varying willingness-to-wait (WTW) estimates for kidney transplant attributes. Patients with the lowest numeracy (0/3) were willing to wait approximately 42 months [95% CI: 29.37, 54.68] for standard infectious risk, compared to 33 months [95% CI: 28.48, 38.09] for 1/3, 28 months [95% CI: 25.13, 30.32] for 2/3, and 24 months [95% CI: 20.51, 27.25] for 3/3 correct answers. However, WTW differences for an additional year of graft survival and neoplastic risk were not statistically significant across numeracy levels. Supplementary analyses of two additional DCEs on COVID-19 vaccinations and rheumatoid arthritis, conducted online, supported these findings: higher numeracy skills were associated with more consistent choices across different disease contexts and survey formats.

Conclusions: The findings suggested that combining patients with varying numeracy skills could bias WTW estimates, highlighting the need to consider numeracy in DCE data analysis and interpretation.