Journal of evaluation in clinical practice最新文献

{"title":"Primary outcomes and characteristics of clinical trial registries (up to October 2021) on COVID-19 vaccines.","authors":"Yuri Yokoyama do Nascimento, Matheus Aparecido de Toledo, Daniel Maringelli Pasqui, Rafael L Pacheco, Rachel Riera","doi":"10.1111/jep.14082","DOIUrl":"https://doi.org/10.1111/jep.14082","url":null,"abstract":"<p><strong>Objectives: </strong>To analyse the general and primary outcome-related characteristics of clinical trials protocols on COVID-19 vaccines.</p><p><strong>Study design and setting: </strong>A meta-research study. A search for clinical trial protocols on COVID-19 vaccines was conducted on the ClinicalTrials.gov platform. We considered all protocols of comparative trials registered up to October 26, 2021.</p><p><strong>Results: </strong>Two hundred and eighty-two trials were analysed. The median expected trial duration was 445 days (interquartile range [IQR] = 225), and the median target sample size was 420 participants (IQR = 1638). A retrospective registry (after the start date) was observed for 42.55% of the trials. Randomization procedures were planned by 84.75% and full-blinding procedures by 34.75% of the 282 trials. Most trials were labelled as active or still recruiting, and 14 trials (5%) were completed. None of the 14 trials labelled as completed on our search date had results available. Industry funding was reported by 198 trials (70.2%). Most studies declared more than one primary outcome, usually a safety or immunogenicity outcome, and 59 studies (20.9%) had at least one primary efficacy outcome. The description of the primary efficacy outcomes was limited in most cases, referred to as a non-specified 'efficacy' outcome (18.6%) or described as 'COVID-19 cases' (32.2%).</p><p><strong>Conclusion: </strong>the primary outcomes of clinical trials on COVID-19 vaccines are poorly described, and the registers provide insufficient information about them. The registry was retrospectively fulfilled for many trials, which may lead to bias and research waste. Outcomes were generically described and did not provide transparent information for replication in practice, further trials or meta-analyses.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498236","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"People with type 2 diabetes experiences of using WhatsApp-based diabetes self-management education and support: The process evaluation.","authors":"Esmaeel Yaagoob, Regina Lee, Michelle Stubbs, Mohammed Hakami, Sally Chan","doi":"10.1111/jep.14083","DOIUrl":"https://doi.org/10.1111/jep.14083","url":null,"abstract":"<p><strong>Rational: </strong>Online Diabetes Self-Management Education and Support (DSMES) offers people with type 2 diabetes mellitus (T2DM) accessible and tailored education, utilising innovative and interactive tools such as social media to enhance engagement and outcomes. Despite the demonstrated effectiveness of social media-based DSMES in improving health outcomes, there remains a significant gap in qualitative insights regarding participants' experiences.</p><p><strong>Aim: </strong>This study aims to explore the experiences of people with T2DM who are using a newly developed WhatsApp-based DSMES.</p><p><strong>Methods: </strong>A qualitative descriptive approach was adopted. Data consisted of 23 semi-structured phone interviews with people with T2DM who had received the WhatsApp-based DSMES. Interviews were analysed using qualitative content analysis. The present study adheres to the COREQ guidelines.</p><p><strong>Results: </strong>Four themes emerged from the data: (1) acceptability of the programme, (2) flexible accessibility of the programme, (3) promoting healthy lifestyle and (4) future preferences for the programme use.</p><p><strong>Conclusion: </strong>This study explored the experiences of people with T2DM participating in a 6-week WhatsApp-based DSMES. The findings indicated that the programme was acceptable, accessible, effectively revealing necessary self-management knowledge and skills, and provided essential support from professional and peer. The study also indicated that WhatsApp-based programmes could be feasibly implemented in various populations, healthcare settings and communities to support people with T2DM globally.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141534551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Mental health clinical pathways for children and young people with long-term health conditions: A systematic review","authors":"Thomas King MSc,&nbsp;Roz Shafran PhD,&nbsp;Dougal S. Hargreaves PhD,&nbsp;Luke Muschialli BSc,&nbsp;Daniela Linton BSc,&nbsp;Sophie Bennett PhD","doi":"10.1111/jep.14018","DOIUrl":"10.1111/jep.14018","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Rationale</h3>\u0000 \u0000 <p>Clinical pathways (CPWs) are structured care plans that set out essential steps in the care of patients with a specific clinical problem. Amidst calls for the prioritisation of integrated mental and physical health care for young people, multidisciplinary CPWs have been proposed as a step towards closer integration. There is very limited evidence around CPWs for young people with mental and physical health needs, necessitating a review of the literature.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Aims and Objectives</h3>\u0000 \u0000 <p>The aim of this review is to understand how clinical pathways have been used to deliver mental health support to children and young people with long-term physical health conditions and their effectiveness across a range of outcomes.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>The databases MEDLINE, CENTRAL, PsycINFO and CINAHL were searched from inception to 6 September 2023. Keywords linked to children and young people, mental health, long-term physical health conditions and CPWs were used. Studies using either quantitative or qualitative research designs were included. All studies must have evaluated a CPW to provide mental health support to children and young people (up to 25 years old) with long-term health physical conditions. Both mental and physical health outcomes were considered. Pathways were grouped by integration ‘model’ as described in the wider literature.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>The initial search returned 4082 studies after deduplication. A total of eight studies detailing six distinct care pathways (232 participants [170 children and young people; 50 caregivers; 12 healthcare professionals]) met eligibility criteria and were included in the analysis. Four pathways were conducted within an ‘integrated model’; two were a combination of ‘integrated’ and ‘colocated’; and none within a ‘co-ordinated model’. Only pathways within an integrated model reported quantitative health outcomes, with improvements across a range of mental health measures. One negative physical health outcome was reported from an integrated diabetes pathway, but this should be interpreted with caution.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>This review identified a range of CPW designs but most fell under an integrated model. The results suggest that calls for integrated mental health pathways in this population may be appropriate; however, conclusions are limited by a paucity of eviden","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/jep.14018","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141534550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Combating COVID-19 health disparities in Black, Indigenous, and People of Colour Communities—A call for critical systems thinking","authors":"Michele Battle-Fisher MPH, MA","doi":"10.1111/jep.14031","DOIUrl":"10.1111/jep.14031","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Rationale</h3>\u0000 \u0000 <p>The Black, Indigenous, and People of Colour populations in the United States are disproportionately affected by the emerging health threat SARS-CoV-2, which causes COVID-19.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Aims and Objectives</h3>\u0000 \u0000 <p>This paper aims to demonstrate the usefulness of critical systems thinking by using scenario planning based on epidemiological data and tying epidemiology with soft systems methodology to investigate COVID-19 disparities among disproportionately affected Black, Indigenous, and People of Colour populations.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>Using a review of the COVID-19 literature and publicly available US COVID-19 data, critical systems thinking is applied in a scenario planning example and a call to link soft systems methodology with epidemiology.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>According to the four plausible Endgame scenarios, levels of community transmission as well as the current state transmission are based on the driving forces of the scenarios. In addition, soft systems methodology explores the effect on stakeholders and strengthens the picture of disease burden beyond sole reliance on traditional data sources.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>This analysis underscores employing critical systems thinking to critically assess diverse methods appropriate for the ongoing complexity of global crises. It is argued that critically engaged subjectivity should be given space alongside data-dependent objectivity. COVID-19 disparities are reliant on the social determinants of health's effects as driving forces on disease transmission in Black, Indigenous, and People of Colour populations. It is moreover argued that critical systems thinking is demonstrated by linking epidemiological evidence with scenario planning and soft systems methodology. This in turn supports a critical systems thinking approach to uncover the state of health disparities among minoritized communities under COVID-19.</p>\u0000 </section>\u0000 </div>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The effect of practice environments and psychological empowerment perceptions of nurses on their care behaviours: A cross-sectional study","authors":"Aysegul Yilmaz PhD, RN, BSN,&nbsp;Mine Yilmaz Kocak PhD, RN, BSN","doi":"10.1111/jep.14076","DOIUrl":"10.1111/jep.14076","url":null,"abstract":"<div>\u0000 \u0000 \u0000 <section>\u0000 \u0000 <h3> Aim</h3>\u0000 \u0000 <p>The aim of this study was to examine the effect of professional and individual characteristics, practice environments and psychological empowerment perceptions of nurses on their care behaviours.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Design</h3>\u0000 \u0000 <p>The study used a cross-sectional research design following the STROBE guidelines.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Methods</h3>\u0000 \u0000 <p>The sample of this descriptive and correlational study consisted of 584 nurses working in a university hospital in Turkey. The data were collected between January and June 2023 using the Nursing Information Form, the Practice Environment Scale of the Nursing Work Index (PES-NWI), the Psychological Empowerment Scale (PES), and the Caring Behaviours Scale-30, and analysed using descriptive statistics, Spearman's correlation and multiple linear regression.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Results</h3>\u0000 \u0000 <p>It was determined that PES-NWI, PES and Caring Behaviours Scale (CBI-30) scores of nurses were higher than the average. The study revealed that certain professional and individual characteristics of nurses, practice environments and psychological empowerment perceptions had an effect on their care behaviours.</p>\u0000 </section>\u0000 \u0000 <section>\u0000 \u0000 <h3> Conclusion</h3>\u0000 \u0000 <p>The results of the study showed that nurses' PES-NWI, PES and CBI-30 scores were higher than the average, and that certain professional and individual characteristics of nurses, practice environments and psychological empowerment perceptions had an effect on care behaviours. Nursing service managers, in particular, are required to understand the importance of this relationship and create appropriate working conditions which are integrated with psychological empowerment to improve caring behaviours of nurses.</p>\u0000 </section>\u0000 </div>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Explicit inference: A meta-replication of SEER cancer registry research evaluating the Affordable Care Act's Medicaid expansion.","authors":"Jason Semprini","doi":"10.1111/jep.14055","DOIUrl":"https://doi.org/10.1111/jep.14055","url":null,"abstract":"<p><strong>Objectives: </strong>Among the provisions within the Affordable Care Act (ACA), expanding Medicaid was arguably the greatest contributor to increasing access to care. For over a decade, researchers have investigated how Medicaid expansion impacted cancer outcomes. Over this same decade, statistical theory illuminated how state-based policy research could be compromised by invalid inference. After reviewing the literature to identify the inference strategies of state-based cancer registry Medicaid expansion research, this study aimed to assess how inference decisions could change the interpretation of Medicaid expansion's impact on staging, treatment, and mortality in cancer patients.</p><p><strong>Data sources: </strong>Cancer case data (2000-2019) was obtained from the Surveillance, Epidemiology, End Results (SEER) programme. Cases included all cancer sites combined, top 10 cancer sites combined, and three screening amenable cancers (colorectal, female breast, female cervical).</p><p><strong>Study design: </strong>A Difference-in-Differences design estimated the association between Medicaid expansion and four binary outcomes: distant stage, initiating treatment >1 month after diagnosis, no surgery recommendation, and death. Three inference techniques were compared: (1) traditional, (2) cluster, and (3) Wild Cluster Bootstrap.</p><p><strong>Data collection: </strong>Data was accessed via SEER*Stat.</p><p><strong>Principal findings: </strong>Estimating standard errors via traditional inference would suggest that Medicaid expansion was associated with delayed treatment initiation and surgery recommendations. Traditional and clustered inference also suggested that Medicaid expansion reduced mortality. Inference using Wild Cluster Bootstrap techniques never rejected the null hypotheses.</p><p><strong>Conclusions: </strong>This study reiterates the importance of explicit inference. Future state-based, cancer policy research can be improved by incorporating emerging techniques. These findings warrant caution when interpreting prior SEER research reporting significant effects of Medicaid expansion on cancer outcomes, especially studies that did not explicitly define their inference strategy.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498233","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing evidence-based medication therapy management tools for psychiatric nurses: An evaluative qualitative case study.","authors":"Melisa Bulut, Çiğdem Yüksel","doi":"10.1111/jep.14050","DOIUrl":"https://doi.org/10.1111/jep.14050","url":null,"abstract":"<p><strong>Rationale, aims and objectives: </strong>Clinical use of psychotropic medications involves diverse risks, addressable by nursing interventions. The research had a dual purpose: developing an \"Evidence-Based Medication Therapy Management Guideline\" and a \"Medication Administration-Tracking Chart\" and evaluating their use through an evaluative case study.</p><p><strong>Methods: </strong>Evidence-based guideline and chart development and evaluative case study. Initially, Evidence-Based Medication Therapy Management Guideline and Medication Administration Tracking Chart for managing medication in a psychiatric unit were developed. Subsequently, their efficacy was evaluated in a case study involving 10 participating nurses used in the psychiatric unit with 123-bed of a training and research hospital in Turkey. Data was collected through personal forms, interviews, medication charts, and researcher observations, and the analysis employed Merriam's case study method.</p><p><strong>Results: </strong>Three themes (inception, implementation, termination, and sustainers) and 12 sub-themes emerged. Nurses stated that the research tools filled their information gaps, enhancing the medication therapy management process's effectiveness and safety, improving nursing care quality and continuity, and benefiting patient outcomes. Nurses expressed a desire to consistently use the tools in the unit and provided suggestions.</p><p><strong>Conclusion: </strong>Nurses highlighted the tools' potential to enhance medication safety, psychiatric care, and patient outcomes. However, their stance on using evidence-based tools revealed an approach/avoidance conflict, balancing benefits and barriers. Experience emerged as a hindrance in embracing evidence-based clinical tools. This study is among the first to comprehensively develop evidence-based medication management guideline and administration-tracking chart for psychiatric nurses globally and in our country. Routine use of the tools is expected to enhance nurses' expertise in psychotropic medication management, leading to improved patient outcomes in medication-related aspects.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Factors influencing the reimbursement of cancer drugs in Europe: A scoping review.","authors":"Pelayo Nieto-Gómez, Celia Castaño-Amores","doi":"10.1111/jep.14080","DOIUrl":"https://doi.org/10.1111/jep.14080","url":null,"abstract":"<p><strong>Rationale: </strong>Reimbursement process of oncology drugs in Europe occurs within a complex decision-making process that varies between Member States. Distinctions between the States trigger societal debates since it is necessary to balance access to medicines and health systems sustainability.</p><p><strong>Aims and objectives: </strong>We aimed to review the evidence concerning factors associated with the reimbursement decision or Health Technology Agency recommendation of oncology drugs in Europe.</p><p><strong>Methods: </strong>A systematic literature search was performed in two databases from inception to august 2023. Screening and data extraction were performed by pairs.</p><p><strong>Results: </strong>Thirteen articles were included and encompassed data from 11 nations. Seven articles showed that cost-effective (C-E) drugs and lower Incremental Cost-Effectiveness Ratios (ICERs) had higher likelihood of reimbursement. Disease severity might influence the reimbursement decision with financial agreements. Improvement in clinical outcomes, substantial clinical benefit (p < 0.01) or overall survival gains (p < 0.05) were positively associated. Orphan drug designation impact varies between countries but positive decisions are usually achieved under specific conditions. Clinical and C-E uncertainty frequently led to reimbursement with financial agreements or outcomes-based conditions. Sociodemographic factors as: social health insurance system, higher Gross Domestic Product and larger elderly population were positively associated with reimbursement (p < 0.01).</p><p><strong>Conclusion: </strong>There is a need for further research into key determinants of reimbursement decisions in Europe and the development of drug access models that can effectively address and overcome costs and effectiveness uncertainties.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Diversity, equity and inclusiveness in healthcare: A primary care perspective.","authors":"Scott Worman","doi":"10.1111/jep.14073","DOIUrl":"https://doi.org/10.1111/jep.14073","url":null,"abstract":"<p><p>Addressing Diversity, Equity and Inclusion in health care is a multidimensional challenge. From a US perspective, the third-party payment system has disempowered and depersonalized health-care delivery. The net result is wasteful and inefficient use of human and financial resources, burnout among providers, as well as care inequities. Financial integration at the point of patient care is essential to aligning the needs of patients with advances in medical technology. Complexity theory offers valuable insights into the roles of government, intermediaries and patients. The government must focus on equity as a rule compiler and referee of the system. Patient and providers who are actively engaged in shared decision-making will naturally address the diverse needs of multitudinous communities. Intermediaries address inclusion by connecting resources with the point of care. In a dynamic, emerging health-care system that serves diverse communities, patient and community-based financing, vouchers and defined contributions are necessary first steps in addressing cultural diversity, inclusion and equity.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Can large language models provide accurate and quality information to parents regarding chronic kidney diseases?","authors":"Rüya Naz, Okan Akacı, Hakan Erdoğan, Ayfer Açıkgöz","doi":"10.1111/jep.14084","DOIUrl":"https://doi.org/10.1111/jep.14084","url":null,"abstract":"<p><strong>Rationale: </strong>Artificial Intelligence (AI) large language models (LLM) are tools capable of generating human-like text responses to user queries across topics. The use of these language models in various medical contexts is currently being studied. However, the performance and content quality of these language models have not been evaluated in specific medical fields.</p><p><strong>Aims and objectives: </strong>This study aimed to compare the performance of AI LLMs ChatGPT, Gemini and Copilot in providing information to parents about chronic kidney diseases (CKD) and compare the information accuracy and quality with that of a reference source.</p><p><strong>Methods: </strong>In this study, 40 frequently asked questions about CKD were identified. The accuracy and quality of the answers were evaluated with reference to the Kidney Disease: Improving Global Outcomes guidelines. The accuracy of the responses generated by LLMs was assessed using F1, precision and recall scores. The quality of the responses was evaluated using a five-point global quality score (GQS).</p><p><strong>Results: </strong>ChatGPT and Gemini achieved high F1 scores of 0.89 and 1, respectively, in the diagnosis and lifestyle categories, demonstrating significant success in generating accurate responses. Furthermore, ChatGPT and Gemini were successful in generating accurate responses with high precision values in the diagnosis and lifestyle categories. In terms of recall values, all LLMs exhibited strong performance in the diagnosis, treatment and lifestyle categories. Average GQ scores for the responses generated were 3.46 ± 0.55, 1.93 ± 0.63 and 2.02 ± 0.69 for Gemini, ChatGPT 3.5 and Copilot, respectively. In all categories, Gemini performed better than ChatGPT and Copilot.</p><p><strong>Conclusion: </strong>Although LLMs provide parents with high-accuracy information about CKD, their use is limited compared with that of a reference source. The limitations in the performance of LLMs can lead to misinformation and potential misinterpretations. Therefore, patients and parents should exercise caution when using these models.</p>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":null,"pages":null},"PeriodicalIF":2.1,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141498229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}