Crowdsourcing and machine learning contests in Parkinson's disease research - when do they work?

Abstract

ObjectiveTo review the application of crowdsourcing and machine learning contests in Parkinson's disease (PD) research, identify best practices for successful implementation, and highlight future opportunities.MethodsThis paper analyzes the landscape of crowdsourcing in PD research through a literature survey and a comparative case study of two major machine learning contests: the MJFF Freezing of Gait (FOG) Challenge and the AMP PD Proteomics Challenge. We also describe a taxonomy of crowdsourcing projects and a framework of success characteristics for machine learning contest design.ResultsThe analysis of previous crowdsourcing and machine learning contests revealed that contest success is highly dependent on specific design factors. The FOG challenge, which addressed a "solvable but not yet solved" problem with a suitable scoring metric, successfully produced a high-performing algorithm with real-world clinical value. In contrast, the Proteomics challenge did not yield biologically meaningful results, as winning models bypassed the core proteomic data, highlighting issues of data signal and metric selection. The review also identified underutilized crowdsourcing approaches in PD research, including gamification and community-based open-source development.ConclusionsMachine learning contests offer a powerful, open-science-aligned method to address complex problems in PD. Success requires careful design, particularly a solvable problem and an appropriate scoring metric. There is significant potential to expand the use of diverse crowdsourcing techniques to accelerate progress in PD research and clinical care.