Maximilian U. Friedrich, Anna-Julia Roenn, Chiara Palmisano, Jane Alty, Steffen Paschen, Guenther Deuschl, Chi Wang Ip, Jens Volkmann, Muthuraman Muthuraman, Robert Peach, Martin M. Reich
{"title":"Validation and application of computer vision algorithms for video-based tremor analysis","authors":"Maximilian U. Friedrich, Anna-Julia Roenn, Chiara Palmisano, Jane Alty, Steffen Paschen, Guenther Deuschl, Chi Wang Ip, Jens Volkmann, Muthuraman Muthuraman, Robert Peach, Martin M. Reich","doi":"10.1038/s41746-024-01153-1","DOIUrl":"10.1038/s41746-024-01153-1","url":null,"abstract":"Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman’s ρ = 0.55–0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [−3.13, 8.23]) and ≤0.21 Hz (95% CI [−0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01153-1.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141436166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cédric Walker, Tasneem Talawalla, Robert Toth, Akhil Ambekar, Kien Rea, Oswin Chamian, Fan Fan, Sabina Berezowska, Sven Rottenberg, Anant Madabhushi, Marie Maillard, Laura Barisoni, Hugo Mark Horlings, Andrew Janowczyk
{"title":"PatchSorter: a high throughput deep learning digital pathology tool for object labeling","authors":"Cédric Walker, Tasneem Talawalla, Robert Toth, Akhil Ambekar, Kien Rea, Oswin Chamian, Fan Fan, Sabina Berezowska, Sven Rottenberg, Anant Madabhushi, Marie Maillard, Laura Barisoni, Hugo Mark Horlings, Andrew Janowczyk","doi":"10.1038/s41746-024-01150-4","DOIUrl":"10.1038/s41746-024-01150-4","url":null,"abstract":"The discovery of patterns associated with diagnosis, prognosis, and therapy response in digital pathology images often requires intractable labeling of large quantities of histological objects. Here we release an open-source labeling tool, PatchSorter, which integrates deep learning with an intuitive web interface. Using >100,000 objects, we demonstrate a >7x improvement in labels per second over unaided labeling, with minimal impact on labeling accuracy, thus enabling high-throughput labeling of large datasets.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11190251/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141432450","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Constance Dubois, David Eigen, François Simon, Vincent Couloigner, Michael Gormish, Martin Chalumeau, Laurent Schmoll, Jérémie F. Cohen
{"title":"Development and validation of a smartphone-based deep-learning-enabled system to detect middle-ear conditions in otoscopic images","authors":"Constance Dubois, David Eigen, François Simon, Vincent Couloigner, Michael Gormish, Martin Chalumeau, Laurent Schmoll, Jérémie F. Cohen","doi":"10.1038/s41746-024-01159-9","DOIUrl":"10.1038/s41746-024-01159-9","url":null,"abstract":"Middle-ear conditions are common causes of primary care visits, hearing impairment, and inappropriate antibiotic use. Deep learning (DL) may assist clinicians in interpreting otoscopic images. This study included patients over 5 years old from an ambulatory ENT practice in Strasbourg, France, between 2013 and 2020. Digital otoscopic images were obtained using a smartphone-attached otoscope (Smart Scope, Karl Storz, Germany) and labeled by a senior ENT specialist across 11 diagnostic classes (reference standard). An Inception-v2 DL model was trained using 41,664 otoscopic images, and its diagnostic accuracy was evaluated by calculating class-specific estimates of sensitivity and specificity. The model was then incorporated into a smartphone app called i-Nside. The DL model was evaluated on a validation set of 3,962 images and a held-out test set comprising 326 images. On the validation set, all class-specific estimates of sensitivity and specificity exceeded 98%. On the test set, the DL model achieved a sensitivity of 99.0% (95% confidence interval: 94.5–100) and a specificity of 95.2% (91.5–97.6) for the binary classification of normal vs. abnormal images; wax plugs were detected with a sensitivity of 100% (94.6–100) and specificity of 97.7% (95.0–99.1); other class-specific estimates of sensitivity and specificity ranged from 33.3% to 92.3% and 96.0% to 100%, respectively. We present an end-to-end DL-enabled system able to achieve expert-level diagnostic accuracy for identifying normal tympanic aspects and wax plugs within digital otoscopic images. However, the system’s performance varied for other middle-ear conditions. Further prospective validation is necessary before wider clinical deployment.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11189910/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141432448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Varun K. Viswanath, Wendy Hartogenesis, Stephan Dilchert, Leena Pandya, Frederick M. Hecht, Ashley E. Mason, Edward J. Wang, Benjamin L. Smarr
{"title":"Five million nights: temporal dynamics in human sleep phenotypes","authors":"Varun K. Viswanath, Wendy Hartogenesis, Stephan Dilchert, Leena Pandya, Frederick M. Hecht, Ashley E. Mason, Edward J. Wang, Benjamin L. Smarr","doi":"10.1038/s41746-024-01125-5","DOIUrl":"10.1038/s41746-024-01125-5","url":null,"abstract":"Sleep monitoring has become widespread with the rise of affordable wearable devices. However, converting sleep data into actionable change remains challenging as diverse factors can cause combinations of sleep parameters to differ both between people and within people over time. Researchers have attempted to combine sleep parameters to improve detecting similarities between nights of sleep. The cluster of similar combinations of sleep parameters from a night of sleep defines that night’s sleep phenotype. To date, quantitative models of sleep phenotype made from data collected from large populations have used cross-sectional data, which preclude longitudinal analyses that could better quantify differences within individuals over time. In analyses reported here, we used five million nights of wearable sleep data to test (a) whether an individual’s sleep phenotype changes over time and (b) whether these changes elucidate new information about acute periods of illness (e.g., flu, fever, COVID-19). We found evidence for 13 sleep phenotypes associated with sleep quality and that individuals transition between these phenotypes over time. Patterns of transitions significantly differ (i) between individuals (with vs. without a chronic health condition; chi-square test; p-value < 1e−100) and (ii) within individuals over time (before vs. during an acute condition; Chi-Square test; p-value < 1e−100). Finally, we found that the patterns of transitions carried more information about chronic and acute health conditions than did phenotype membership alone (longitudinal analyses yielded 2–10× as much information as cross-sectional analyses). These results support the use of temporal dynamics in the future development of longitudinal sleep analyses.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01125-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141430533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Eric Hyndman, Robert J. Paproski, Adam Kinnaird, Adrian Fairey, Leonard Marks, Christian P. Pavlovich, Sean A. Fletcher, Roman Zachoval, Vanda Adamcova, Jiri Stejskal, Armen Aprikian, Christopher J. D. Wallis, Desmond Pink, Catalina Vasquez, Perrin H. Beatty, John D. Lewis
{"title":"Development of an effective predictive screening tool for prostate cancer using the ClarityDX machine learning platform","authors":"M. Eric Hyndman, Robert J. Paproski, Adam Kinnaird, Adrian Fairey, Leonard Marks, Christian P. Pavlovich, Sean A. Fletcher, Roman Zachoval, Vanda Adamcova, Jiri Stejskal, Armen Aprikian, Christopher J. D. Wallis, Desmond Pink, Catalina Vasquez, Perrin H. Beatty, John D. Lewis","doi":"10.1038/s41746-024-01167-9","DOIUrl":"10.1038/s41746-024-01167-9","url":null,"abstract":"The current prostate cancer (PCa) screen test, prostate-specific antigen (PSA), has a high sensitivity for PCa but low specificity for high-risk, clinically significant PCa (csPCa), resulting in overdiagnosis and overtreatment of non-csPCa. Early identification of csPCa while avoiding unnecessary biopsies in men with non-csPCa is challenging. We built an optimized machine learning platform (ClarityDX) and showed its utility in generating models predicting csPCa. Integrating the ClarityDX platform with blood-based biomarkers for clinically significant PCa and clinical biomarker data from a 3448-patient cohort, we developed a test to stratify patients’ risk of csPCa; called ClarityDX Prostate. When predicting high risk cancer in the validation cohort, ClarityDX Prostate showed 95% sensitivity, 35% specificity, 54% positive predictive value, and 91% negative predictive value, at a ≥ 25% threshold. Using ClarityDX Prostate at this threshold could avoid up to 35% of unnecessary prostate biopsies. ClarityDX Prostate showed higher accuracy for predicting the risk of csPCa than PSA alone and the tested model-based risk calculators. Using this test as a reflex test in men with elevated PSA levels may help patients and their healthcare providers decide if a prostate biopsy is necessary.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11190196/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141432449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robert Peach, Maximilian Friedrich, Lara Fronemann, Muthuraman Muthuraman, Sebastian R. Schreglmann, Daniel Zeller, Christoph Schrader, Joachim K. Krauss, Alfons Schnitzler, Matthias Wittstock, Ann-Kristin Helmers, Steffen Paschen, Andrea Kühn, Inger Marie Skogseid, Wilhelm Eisner, Joerg Mueller, Cordula Matthies, Martin Reich, Jens Volkmann, Chi Wang Ip
{"title":"Head movement dynamics in dystonia: a multi-centre retrospective study using visual perceptive deep learning","authors":"Robert Peach, Maximilian Friedrich, Lara Fronemann, Muthuraman Muthuraman, Sebastian R. Schreglmann, Daniel Zeller, Christoph Schrader, Joachim K. Krauss, Alfons Schnitzler, Matthias Wittstock, Ann-Kristin Helmers, Steffen Paschen, Andrea Kühn, Inger Marie Skogseid, Wilhelm Eisner, Joerg Mueller, Cordula Matthies, Martin Reich, Jens Volkmann, Chi Wang Ip","doi":"10.1038/s41746-024-01140-6","DOIUrl":"10.1038/s41746-024-01140-6","url":null,"abstract":"Dystonia is a neurological movement disorder characterised by abnormal involuntary movements and postures, particularly affecting the head and neck. However, current clinical assessment methods for dystonia rely on simplified rating scales which lack the ability to capture the intricate spatiotemporal features of dystonic phenomena, hindering clinical management and limiting understanding of the underlying neurobiology. To address this, we developed a visual perceptive deep learning framework that utilizes standard clinical videos to comprehensively evaluate and quantify disease states and the impact of therapeutic interventions, specifically deep brain stimulation. This framework overcomes the limitations of traditional rating scales and offers an efficient and accurate method that is rater-independent for evaluating and monitoring dystonia patients. To evaluate the framework, we leveraged semi-standardized clinical video data collected in three retrospective, longitudinal cohort studies across seven academic centres. We extracted static head angle excursions for clinical validation and derived kinematic variables reflecting naturalistic head dynamics to predict dystonia severity, subtype, and neuromodulation effects. The framework was also applied to a fully independent cohort of generalised dystonia patients for comparison between dystonia sub-types. Computer vision-derived measurements of head angle excursions showed a strong correlation with clinically assigned scores. Across comparisons, we identified consistent kinematic features from full video assessments encoding information critical to disease severity, subtype, and effects of neural circuit interventions, independent of static head angle deviations used in scoring. Our visual perceptive machine learning framework reveals kinematic pathosignatures of dystonia, potentially augmenting clinical management, facilitating scientific translation, and informing personalized precision neurology approaches.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01140-6.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141334435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reply to: Association of accelerometer-measured physical activity intensity, sedentary time, and exercise time with incident Parkinson’s disease: need more evidence","authors":"Mengyi Liu, Xianhui Qin","doi":"10.1038/s41746-024-01155-z","DOIUrl":"10.1038/s41746-024-01155-z","url":null,"abstract":"","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11189527/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141420092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Paola Daniore, Vasileios Nittas, Christina Haag, Jürgen Bernard, Roman Gonzenbach, Viktor von Wyl
{"title":"From wearable sensor data to digital biomarker development: ten lessons learned and a framework proposal","authors":"Paola Daniore, Vasileios Nittas, Christina Haag, Jürgen Bernard, Roman Gonzenbach, Viktor von Wyl","doi":"10.1038/s41746-024-01151-3","DOIUrl":"10.1038/s41746-024-01151-3","url":null,"abstract":"Wearable sensor technologies are becoming increasingly relevant in health research, particularly in the context of chronic disease management. They generate real-time health data that can be translated into digital biomarkers, which can provide insights into our health and well-being. Scientific methods to collect, interpret, analyze, and translate health data from wearables to digital biomarkers vary, and systematic approaches to guide these processes are currently lacking. This paper is based on an observational, longitudinal cohort study, BarKA-MS, which collected wearable sensor data on the physical rehabilitation of people living with multiple sclerosis (MS). Based on our experience with BarKA-MS, we provide and discuss ten lessons we learned in relation to digital biomarker development across key study phases. We then summarize these lessons into a guiding framework (DACIA) that aims to informs the use of wearable sensor data for digital biomarker development and chronic disease management for future research and teaching.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11189504/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141420037","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rebecca Mathias, Peter McCulloch, Anastasia Chalkidou, Stephen Gilbert
{"title":"Digital health technologies need regulation and reimbursement that enable flexible interactions and groupings","authors":"Rebecca Mathias, Peter McCulloch, Anastasia Chalkidou, Stephen Gilbert","doi":"10.1038/s41746-024-01147-z","DOIUrl":"10.1038/s41746-024-01147-z","url":null,"abstract":"Digital Health Technologies (DHTs) are being applied in a widening range of scenarios in medicine. We describe the emerging phenomenon of the grouping of individual DHTs, with a clinical use case and regulatory approval in their own right, into packages to perform specific clinical tasks in defined settings. Example groupings include suites of devices for remote monitoring, or for smart clinics. In this first article of a two-article series, we describe challenges in implementation and limitations in frameworks for the regulation, health technology assessment, and reimbursement of these device suites and linked novel care pathways.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01147-z.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141334122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}