JMIR Formative Research最新文献

{"title":"Correction: \"Smartwatch-Based Ecological Momentary Assessment for High-Temporal-Density, Longitudinal Measurement of Alcohol Use (AlcoWatch): Feasibility Evaluation\".","authors":"Chris Stone, Sally Adams, Robyn E Wootton, Andy Skinner","doi":"10.2196/82194","DOIUrl":"https://doi.org/10.2196/82194","url":null,"abstract":"<p><p>[This corrects the article DOI: 10.2196/63184.].</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e82194"},"PeriodicalIF":2.0,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145199419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Impact of Dose in an mHealth Intervention to Support Parents and Carers Via Healthy Beginnings for Hunter New England Kids Program: Pragmatic Randomized Controlled Trial.","authors":"Alison L Brown, Nayerra Hudson, Jessica Pinfold, Rebecca Sewter, Lynda Davies, Christophe Lecathelinais, Jacklyn K Jackson, Tessa Delaney, Sienna Kavalec, Rachel Sutherland","doi":"10.2196/70158","DOIUrl":"10.2196/70158","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;The dose of mobile health (mHealth) interventions can influence participant engagement, acceptability, and overall impact. However, few mHealth interventions have explored this dose-response relationship.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aims to explore how dose influences the acceptability, engagement, cost, and impact on infant feeding status of a parent-targeted mHealth text messaging program which aims to enhance child health, including breastfeeding exclusivity and duration.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;This pragmatic randomized controlled trial was conducted from October 2021 to May 2024. The Healthy Beginnings for Hunter New England Kids (HB4HNEKids) program provides- text messages aimed to support parents and carers and their children by providing evidence-based preventive health information across the first 2000 days. Participants were enrolled in HB4HNEKids from 5 Child and Family Health Services in the Hunter New England region of New South Wales, Australia, and randomized into either a high-dose or low-dose text message group for the first 2 years of the pilot program. Dose refers to the quantity and frequency of text messages sent to participants. Participants in the high-dose text message group received an average of 111-121 text messages, and the low-dose text message group received 80-82 text messages across the 2 years. Outcomes of interest included acceptability, engagement, cost, and infant feeding status in relation to dose. Engagement with the messages was determined using click rates and program opt-out rates. Participant acceptability was assessed via a brief survey. Impact on infant feeding status (ie, breastfeeding, formula feeding, or mixed feeding) was determined by participants reporting their feeding status at several time points across the program. Cost was determined by assessing the per participant and total cost of sending text messages for each dose group across the 2-year period.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;There were no statistically significant differences in click rates between high or low-dose text message groups. In the first 6 months, significantly more participants opted out of the high-dose text message group (191/2724; 7%) compared to the low-dose (108/2812; 3.8%; P&lt;.001). In terms of program acceptability, 183 out of 214 (85.5%) participants of the high-dose and 228 out of 252 (90.5%) participants of the low-dose text message group were satisfied with the frequency of text messages. In addition, 188 out of 215 (87%) participants of high-dose and 220 out of 255 (86%) participants of low-dose text message group indicated they would recommend the program to other caregivers. The average per participant and total cost to the health service for sending messages was lower in the low-dose group (A$9.32 per participant and A$15,271.48 total; A$1 is approximately equal to US $0.68) compared to the high-dose text message group (A$12.96 per participant and A$21,2","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e70158"},"PeriodicalIF":2.0,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12488034/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145206582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Differential Diagnosis Assessment in Ambulatory Care With a Digital Health History Device: Pseudorandomized Study.","authors":"Beth Healey, Adrien Schwitzguebel, Herve Spechbach","doi":"10.2196/56384","DOIUrl":"10.2196/56384","url":null,"abstract":"<p><strong>Background: </strong>Digital health history devices represent a promising wave of digital tools with the potential to enhance the quality and efficiency of medical consultations. They achieve this by providing physicians with standardized, high-quality patient history summaries and facilitating the development of differential diagnoses (DDs) before consultation, while also engaging patients in the diagnostic process.</p><p><strong>Objective: </strong>This study evaluates the efficacy of one such digital health history device, diagnosis and anamnesis (DIANNA), in assisting with the formulation of appropriate DDs in an outpatient setting.</p><p><strong>Methods: </strong>A pseudorandomized controlled trial was conducted with 101 patients seeking care at the University Hospital Geneva emergency outpatient department. Participants presented with various conditions affecting the limbs, back, and chest. The first 51 patients were assigned to the control group, while the subsequent 50 formed the intervention group. In the control group, physicians developed DD lists based on traditional history-taking and clinical examination. In the intervention group, physicians reviewed DIANNA-generated DD reports before interacting with the patient. In both groups, a senior physician independently formulated a DD list, serving as the gold standard for comparison.</p><p><strong>Results: </strong>The study findings indicate that DIANNA use was associated with a notable improvement in DD accuracy (mean 79.3%, SD 24%) compared with the control group (mean 70.5%, SD 33%; P=.01). Subgroup analysis revealed variations in effectiveness based on case complexity: low-complexity cases (1-2 possible DDs) showed 8% improvement in the intervention group (P=.08), intermediate-complexity cases (3 possible DDs) showed 17% improvement (P=.03), and high-complexity cases (4-5 possible DDs) showed 15% improvement (P=.92). The intervention was not superior to the control in low-complexity cases (P=.08) or high-complexity cases (P=.92). Overall, DIANNA successfully determined appropriate DDs in 81.6% of cases, and physicians reported that it helped establish the correct DD in 26% of cases.</p><p><strong>Conclusions: </strong>The study suggests that DIANNA has the potential to support physicians in formulating more precise DDs, particularly in intermediate-complexity cases. However, its effectiveness varied by case complexity and further validation is needed to assess its full clinical impact. These findings highlight the potential role of digital health history devices such as DIANNA in improving clinical decision-making and diagnostic accuracy in medical practice.</p><p><strong>Trial registration: </strong>ClinicalTrials.gov NCT03901495; https://clinicaltrials.gov/study/NCT03901495.</p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":" ","pages":"e56384"},"PeriodicalIF":2.0,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143993601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Machine Learning-Enhanced Surveillance for Surgical Site Infections in Patients Undergoing Colon Surgery: Model Development and Evaluation Study.","authors":"Ugur Celik, Feifan Liu, Kimiyoshi Kobayashi, Richard T Ellison Iii, Yurima Guilarte-Walker, Deborah Ann Mack, Qiming Shi, Adrian Zai","doi":"10.2196/75121","DOIUrl":"10.2196/75121","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Surgical site infections (SSIs) are one of the most common health care-associated infections, accounting for nearly 20% of all health care-associated infections in hospitalized patients. SSIs are associated with longer hospital stays, increased readmission rates, higher health care costs, and a mortality rate twice that of patients without infections.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aimed to develop and evaluate machine learning (ML) models for augmenting SSI surveillance after colon surgery with the goal of improving the efficiency of infection control practices by prioritizing patients at high risk.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We conducted a retrospective study using data from 1508 patients undergoing colon surgery treated between 2018 and 2023 at a single academic medical center. Of these 1508 patients, 66 (4.4%) developed SSIs as adjudicated by infection control practitioners following Centers for Disease Control and Prevention National Healthcare Safety Network criteria. Data included 78 structured variables (eg, demographics, comorbidities, vital signs, laboratory tests, medications, and operative details) and 2 features derived from unstructured clinical notes using natural language processing. ML models&lt;strong&gt;-&lt;/strong&gt;logistic regression, random forest, and Extreme Gradient Boosting (XGBoost)&lt;strong&gt;-&lt;/strong&gt;were trained using stratified 80/20 train-test splits. Class imbalance was addressed using cost-sensitive learning and the synthetic minority oversampling technique. Model performance was evaluated using precision, recall, F&lt;sub&gt;1&lt;/sub&gt;-score, area under the receiver operating characteristic curve, and Brier scores for calibration.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Of the 1508 patients, those who developed SSIs had longer hospital stays (mean 8.1, SD 6.8 days vs mean 6.3, SD 10.5 days; P&lt;.001), higher rates of an American Society of Anesthesiologists score of 3 (52/66, 79% vs 653/1442, 45.3%; P&lt;.001), and elevated white blood cell counts (51/66, 77% vs 734/1442, 50.9%; P&lt;.001). XGBoost achieved the best overall performance with an area under the receiver operating characteristic curve of 0.788, precision of 50%, recall of 38%, and Brier score of 0.035. Random forest yielded perfect precision (100%) but lower recall (23%), with a Brier score of 0.034. Logistic regression showed the highest recall (46%) but the lowest precision (10%), with a Brier score of 0.139. Feature importance analysis using Shapley additive explanations (SHAP) values revealed that the top predictors included recovery duration (SHAP=1.18), SSI keyword frequency (SHAP=1.12), patient age (SHAP=1.12), and American Society of Anesthesiologists score (SHAP=0.94), with natural language processing-derived features ranking among the top 10.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;ML models can augment traditional SSI surveillance by improving early identification of patients at high risk. The XGBoost model offered the best trad","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e75121"},"PeriodicalIF":2.0,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145199425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Application of Large Language Models in Data Analysis and Medical Education for Assisted Reproductive Technology: Comparative Study.","authors":"Noriyuki Okuyama, Mika Ishii, Yuriko Fukuoka, Hiromitsu Hattori, Yuta Kasahara, Tai Toshihiro, Koki Yoshinaga, Tomoko Hashimoto, Koichi Kyono","doi":"10.2196/70107","DOIUrl":"10.2196/70107","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Recent studies have demonstrated that large language models exhibit exceptional performance in medical examinations. However, there is a lack of reports assessing their capabilities in specific domains or their application in practical data analysis using code interpreters. Furthermore, comparative analyses across different large language models have not been extensively conducted.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;The purpose of this study was to evaluate whether advanced artificial intelligence (AI) models can analyze data from template-based input and demonstrate basic knowledge of reproductive medicine. Four AI models (GPT-4, GPT-4o, Claude 3.5 Sonnet, and Gemini Pro 1.5) were evaluated for their data analytical capabilities through numerical calculations and graph rendering. Their knowledge of infertility treatment was assessed using 10 examination questions developed by experts.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;First, we uploaded data to the AI models and furnished instruction templates using the chat interface. The study investigated whether the AI models could perform pregnancy rate analysis and graph rendering, based on blastocyst grades according to Gardner criteria. Second, we assessed model diagnostic capabilities based on specialized knowledge. This evaluation used 10 questions derived from the Japanese Fertility Specialist Examination and the Embryologist Certification Exam, along with chromosome imaging. These materials were curated under the supervision of certified embryologists and fertility specialists. All procedures were repeated 10 times per AI model.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;GPT-4o achieved grade A output (defined as achieving the objective with a single output attempt) in 9 out of 10 trials, outperforming GPT-4, which achieved grade A in 7 out of 10. The average processing times for data analysis were 26.8 (SD 3.7) seconds for GPT-4o and 36.7 (SD 3) seconds for GPT-4, whereas Claude failed in all 10 attempts. Gemini achieved an average processing time of 23 (SD 3) seconds and received grade A in 6 out of 10 trials, though occasional manual corrections were needed. Embryologists required an average of 358.3 (SD 9.7) seconds for the same tasks. In the knowledge-based assessment, GPT-4o, Claude, and Gemini achieved perfect scores (9/9) on multiple-choice questions, while GPT-4 showed a 60% (6/10) success rate on 1 question. None of the AI models could reliably diagnose chromosomal abnormalities from karyotype images, with the highest image diagnostic accuracy being 70% (7/10) for Claude and Gemini.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;This rapid processing demonstrates the potential for these AI models to significantly expedite data-intensive tasks in clinical settings. This performance underscores their potential utility as educational tools or decision support systems in reproductive medicine. However, none of the models were able to accurately interpret and diagnose using medical images","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e70107"},"PeriodicalIF":2.0,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12488165/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145206545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bedtime App-Guided Mindfulness Meditation in Patients With Insomnia: Mixed Methods Feasibility and Acceptability Pilot Study.","authors":"Yan Ma, Peter M Wayne, Janet M Mullington, Suzanne Bertisch, Gloria Y Yeh","doi":"10.2196/67366","DOIUrl":"10.2196/67366","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;While mindfulness meditation (MM) apps have gained popularity as a tool for promoting sleep, research focusing on bedtime mindfulness practice and app usage is limited.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;As the first step toward understanding the efficacy and mechanisms of such bedtime practice and to inform future investigations, the goal of this pilot study was to explore the feasibility of app-guided bedtime MM practice with both in-lab and at-home physiological and self-report sleep remote assessments.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;We conducted a single-arm, prospective mixed methods pilot study that included both standard in-lab sleep studies and remote at-home assessments of individuals with insomnia disorder with self-reported difficulty falling asleep. Participants practiced MM guided by a commercially available smartphone app at bedtime for 4 weeks. Pre-post assessments included a battery of sleep-related and psychological health questionnaires, objective physiological sleep measures (polysomnography and actigraphy), and daily sleep logs. We also conducted qualitative exit interviews to further assess feasibility and acceptability. Transcripts were analyzed for dominant themes using inductive and deductive qualitative methods.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;We recruited 13 participants with chronic insomnia (symptoms ≥3 nights weekly for ≥3 months) to complete the study protocol within 8 months (retention rate 77%). We were able to collect analyzable physiological and psychometric data with overall completion rates of more than 90%. The study was deemed feasible, meeting a priori benchmarks including recruitment, retention, completion, and adherence. The 10 participants retained in the program had excellent engagement (95% completion of in-lab studies, 100% completion of questionnaires, and 91% compliance with use of the app). Our preliminary analysis of subjective measures indicated improvement in sleep quality, insomnia severity, and presleep arousal, including Pittsburgh Sleep Quality Index change of -3.7 (95% CI -6.7 to -0.7), Insomnia Severity Index change of -4.5 (95% CI -7.7 to -1.4), Pre-Sleep Arousal Scale change of -7.7 (95% CI -13.1 to -2.3), and trend toward improvement in the Ford Insomnia Response to Stress Test indicated by a change of -2.5 (95% CI -5.9 to 0.9). From qualitative data, we identified domains that inform the feasibility and acceptability of the study, including (1) barriers to sleep prior to the study, (2) benefits and skills imparted by mindfulness, and (3) feedback on app use. Benefits and skills imparted by mindfulness included decreased catastrophizing, acceptance and nonreactivity, body awareness and relaxation, self-kindness, awareness of sleep hygiene and bedtime routine, earlier defusing of stress, increased focus and presence, and calm throughout the day.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;Bedtime app-guided MM as an intervention in patients with insomnia and the hy","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e67366"},"PeriodicalIF":2.0,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12483474/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145199422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"AI-Enabled, Text-Based Health Coaching and Navigation for Employees to Support Health Outcomes: Pre-Post Observational Study.","authors":"Paula Wilbourne, Susan Mirch-Kretschmann, Denise Walker, Michael Varghese, Roberto Arnetoli","doi":"10.2196/64553","DOIUrl":"10.2196/64553","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Limited, timely access to quality mental health treatment harms well-being and quality of life while costing individuals and organizations millions in increased medical spending and reduced productivity. Too few qualified professionals, inconsistent quality, and stigma thwart traditional solutions, creating the need for scalable, science-based solutions.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This report provides an overview of a novel digital health coaching service that consists of artificial intelligence (AI)-assisted, human-delivered, text-based health coaching. This report provides data evaluating the efficacy of this service for delivering mental health support, improving well-being, and enhancing workplace productivity.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;This observational study analyzed operational and self-reported health data from employees of subscribing organizations who used Sibly's digital health coaching service. Data included response times, changes in expressed member sentiment, conversation topics, and adherence to motivational interviewing. A subset of members (n=38) provided pre-post self-reported assessment measures of distress, unhealthy days, and presenteeism, having engaged in at least 4 coaching conversations over a minimum of 14 days. Sentiment was evaluated using a natural language processing tool.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;Sibly provided quick access to interactive human coaching, with a median response time of 132 seconds. Sentiment analysis showed that 57% (878/1540) of conversations increased in positive emotions. The coaches maintained strong fidelity to the techniques of motivational interviewing, with adherence exceeding 90% (387/430). The proportion of users reporting severe distress declined from 33.3% (10/30) at baseline to 6.7% (2/30) at follow-up, representing a 79% relative reduction (P&lt;.001). Participants also reported a reduction in the number of unhealthy days per month, decreasing from 19.57 to 15.87 per month (P=.02). Self-reported productivity improved by 18% during the study period (P&lt;.001). Additionally, 61% (47/77) of users who received referrals to additional employer-sponsored benefits engaged with those resources, demonstrating effective care navigation to relevant support services.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;This report provides an overview of novel mental health support and navigation services that use AI-enabled, text-based health coaching and care navigation. Data suggest that the services provide effective, scalable mental health support in workplace settings. The platform helped reduce distress, improve well-being, and boost productivity by offering immediate access to trained coaches and personalized guidance. These results are consistent with existing research on digital mental health services. They highlight the potential of AI-assisted coaching to improve access to care. Future research should include larger, diverse populations and more rigorous ","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e64553"},"PeriodicalIF":2.0,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12500221/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145199416","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Professional Support Through a Tailor-Made Mobile App to Reduce Stress and Depressive Symptoms Among Family Caregivers of People With Dementia: Mixed Methods Pilot Study.","authors":"Aber Sharon Kagwa, Jessica Longhini, Muhammed Nazmul Islam, Sofia Vikström, Åsa Dorell, Hanne Konradsen, Zarina Nahar Kabir","doi":"10.2196/75113","DOIUrl":"10.2196/75113","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Providing informal care to people with dementia living at home can be challenging and may cause caregiver stress and depression. Interventions delivered through mobile apps provide innovative solutions for community-based social care professionals to address the increasing support needs of family caregivers (FCs) of people with dementia.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;This study aimed to examine, among FCs of people with dementia living at home, (1) the potential association between professional support provided through a mobile app and caregiver stress and depressive symptoms, (2) types of support provided through chat interactions between FCs and social care professionals, and (3) how support provided through a mobile app relates to changes in caregiver stress and depressive symptoms.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;A mixed methods pilot study integrated quantitative pre- and postintervention data with qualitative logged chat data. FCs of people with dementia living at home (n=35) were recruited to test a tailor-made mobile app over 8 weeks. The primary and secondary outcome measures were caregiver stress and depressive symptoms, respectively. Descriptive statistics were used to summarize sociodemographic factors; inferential statistics were used to analyze mean differences in outcomes pre- and postintervention. FCs were divided into 3 groups based on changes in caregiver stress scores between pre- and postintervention. Generalized linear model analyses determined the association between participation in the intervention and caregiver stress and depressive symptoms, adjusting for age, gender, and relationship to the person with dementia. Logged chat data were analyzed using summative content analysis to identify types of support provided and received. Changes in caregiver stress were integrated with chat data to determine patterns in types of support received.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;The mean age of FCs was 69.4 (SD 11.9) years, with most being women (28/35, 80%), partners (24/35, 68.6%), and living with the person with dementia (26/35, 74%). The mean score of caregiver stress was marginally higher postintervention (24.1, SD 9.3) than preintervention (23.9, SD 9.2), whereas the mean score of depressive symptoms decreased from pre- (6.5, SD 5.1) to postintervention (6.2, SD 5.2). These differences were not statistically significant. Regression analyses showed that participation in the intervention was not statistically significantly associated with caregiver stress (β=0.171, α=.05; P=.86) or depressive symptoms (β=-0.293, α=.05; P=.75) after adjusting for age, gender, and relationship to the person with dementia. However, mixed methods analysis at the subgroup level suggested that frequent tailored support by social care professionals delivered through a mobile app may reduce caregiver stress among FCs of people with dementia living at home.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;The study highlights the i","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":"9 ","pages":"e75113"},"PeriodicalIF":2.0,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12483335/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145199397","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Impact of Detailed Versus Generic Instructions on Fine-Tuned Language Models for Patient Discharge Instructions Generation: Comparative Statistical Analysis.","authors":"Muneerah Alqahtani, Abdullah Al-Barakati, Fahd Alotaibi, Mohammed Al Shibli, Saad Almousa","doi":"10.2196/80917","DOIUrl":"https://doi.org/10.2196/80917","url":null,"abstract":"<p><strong>Background: </strong>Discharge instructions are essential for patient post-hospital care, but are time-consuming to write. With the rise of large language models (LLMs), there is strong potential to automate this process. This study explores the use of open-source LLMs for generating discharge instructions.</p><p><strong>Objective: </strong>We investigated whether a Mistral model can reliably generate patient-oriented discharge instructions. Two distinct instruction-tuning paradigms were compared, each using a different mechanism for embedding guidance during fine-tuning.</p><p><strong>Methods: </strong>In our experiment, we applied Mistral-NeMo-Instruct, a large language model, in combination with two distinct instruction strategies for fine-tuning. The first were detailed instructions tailored to the task of discharge instruction generation. The second was a basic instruction with minimal guidance and no task-specific detail. The independent variable in this study is the instruction strategy (detailed vs. generic), while the dependent variables are the evaluation scores of the generated discharge instructions. The generated discharge instructions were evaluated against 3,621 ground-truth references. We used BLEU-1 to BLEU-4, ROUGE (ROUGE-1, ROUGE-2, ROUGE-L), SentenceTransformer similarity, and BERTScore as evaluation metrics to assess the quality of the generated outputs in comparison to the corresponding ground-truth instructions for the same discharge summaries.</p><p><strong>Results: </strong>The detailed instruction model demonstrated superior performance across all automated evaluation metrics compared with the generic instruction model. BERTScore increased from 78.92% to 87.05%, while structural alignment measured by ROUGE-L improved from 8.59% to 26.52%. N-gram precision (BLEU-4) increased from 0.81% to 21.24%, and METEOR scores rose from 15.33% to 18.47%. Additional metrics showed consistent gains: ROUGE-1 improved from 16.59% to 42.72%, and ROUGE-2 increased from 1.97% to 45.84%. All improvements were statistically significant (P < .001), indicating that detailed, task-specific instruction design substantially enhances model performance.</p><p><strong>Conclusions: </strong>The use of detailed, task-specific instruction strategies significantly enhances the effectiveness of open-source large language models in generating discharge instructions. These findings indicate that carefully designed instructions during fine-tuning substantially improve model performance.</p><p><strong>Clinicaltrial: </strong></p>","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":" ","pages":""},"PeriodicalIF":2.0,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145206534","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Integration of an artificial intelligence-based autism diagnostic into the ECHO Autism Primary Care Early Diagnostic workflow: results of a prospective observational study.","authors":"Kristin Sohl, Eric Linstead, Kelianne Heinz, Elia Eiroa Lledo, Alicia Brewer Curran, Melissa Mahurin, Valeria Nanclares-Nogués, Carmela Salomon, Minda Seal, Sharief Taraman","doi":"10.2196/80733","DOIUrl":"https://doi.org/10.2196/80733","url":null,"abstract":"&lt;p&gt;&lt;strong&gt;Background: &lt;/strong&gt;Pediatric specialist shortages and rapidly rising autism prevalence rates have compelled primary care clinicians to consider playing a greater role in the autism diagnostic process. The ECHO Autism: Early Diagnosis Program (EDx) prepares clinicians to screen, evaluate, differentiate, diagnose and provide longitudinal care for autistic children in primary care settings. Canvas Dx is a prescription-only Software as a Medical Device designed to support clinical diagnosis or rule out of autism, including in primary care settings. It is FDA authorized for use, in conjunction with clinical judgement, in 18-72-month-olds with indicators of developmental delay.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Objective: &lt;/strong&gt;To assess the feasibility and impact of integrating the Device into the ECHO Autism: EDx workflow. Time from the first clinical question of developmental delay to autism diagnosis is the primary endpoint. Secondary endpoints explore clinician and caregiver experience of device use.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Methods: &lt;/strong&gt;Children aged 18-72-months-olds with concern for developmental delay indicated by either a caregiver or health professionals were eligible to participate in this prospective observational study. Experienced ECHO Autism: EDx Clinicians were recruited to evaluate the inclusion of the Device as part of their diagnostic evaluations. Outcome data was collected via a combination of electronic questionnaires, standard clinical care record reviews and analysis of Device outputs. Institutional Review Board Approval was provided by the University of Missouri-Columbia (IRB assigned project number 2075722).&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Results: &lt;/strong&gt;80 children and seven clinicians completed the study. On average, time from clinical concern at study enrollment to final autism diagnosis was 39.22 days, compared to 180-264 day waits at adjacent specialist referral centers. The vast majority (93%) of caregivers reported being satisfied with the ECHO Autism: EDx plus Device evaluation their child received and endorsed that they would recommend it to others and that they felt comfortable using the Device. The Device produced determinate autism predictions or rule outs for 52.50% of participants, and in all cases these were consistent with the final clinical determination. Participating clinicians reported Device use was feasible and reduced several challenges associated with their previous diagnostic process, however, they noted it did not obviate the need for additional structured observation in every case.&lt;/p&gt;&lt;p&gt;&lt;strong&gt;Conclusions: &lt;/strong&gt;The ECHO Autism: EDx plus Device workflow offers considerable time savings compared to specialty center referral and was strongly endorsed by caregiver participants. Embedding the Device into the ECHO Autism: EDx workflow was feasible and helped streamline several workflow efficiencies. Clinicians still utilized their training and application and interpretation of DSM-5 criteria when formulating the diagnosis fo","PeriodicalId":14841,"journal":{"name":"JMIR Formative Research","volume":" ","pages":""},"PeriodicalIF":2.0,"publicationDate":"2025-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145199507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}