{"title":"Comparison of the Usability and Reliability of Answers to Clinical Questions: AI-Generated ChatGPT versus a Human-Authored Resource.","authors":"Farrin A Manian, Katherine Garland, Jimin Ding","doi":"10.14423/SMJ.0000000000001715","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001715","url":null,"abstract":"<p><strong>Objectives: </strong>Our aim was to compare the usability and reliability of answers to clinical questions posed of Chat-Generative Pre-Trained Transformer (ChatGPT) compared to those of a human-authored Web source (www.Pearls4Peers.com) in response to \"real-world\" clinical questions raised during the care of patients.</p><p><strong>Methods: </strong>Two domains of clinical information quality were studied: usability, based on organization/readability, relevance, and usefulness, and reliability, based on clarity, accuracy, and thoroughness. The top 36 most viewed real-world questions from a human-authored Web site (www.Pearls4Peers.com [P4P]) were posed to ChatGPT 3.5. Anonymized answers by ChatGPT and P4P (without literature citations) were separately assessed for usability by 18 practicing physicians (\"clinician users\") in triplicate and for reliability by 21 expert providers (\"content experts\") on a Likert scale (\"definitely yes,\" \"generally yes,\" or \"no\") in duplicate or triplicate. Participants also directly compared the usability and reliability of paired answers.</p><p><strong>Results: </strong>The usability and reliability of ChatGPT answers varied widely depending on the question posed. ChatGPT answers were not considered useful or accurate in 13.9% and 13.1% of cases, respectively. In within-individual rankings for usability, ChatGPT was inferior to P4P in organization/readability, relevance, and usefulness in 29.6%, 28.3%, and 29.6% of cases, respectively, and for reliability, inferior to P4P in clarity, accuracy, and thoroughness in 38.1%, 34.5%, and 31% of cases, respectively.</p><p><strong>Conclusions: </strong>The quality of ChatGPT responses to real-world clinical questions varied widely, with nearly one-third or more answers considered inferior to a human-authored source in several aspects of usability and reliability. Caution is advised when using ChatGPT in clinical decision making.</p>","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 8","pages":"467-473"},"PeriodicalIF":1.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141879496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robert W Alexander, Shengping Yang, Christopher J Peterson, Kenneth Nugent
{"title":"Analysis of the Types of Retracted COVID-19 Articles Published in PubMed-Listed Journals.","authors":"Robert W Alexander, Shengping Yang, Christopher J Peterson, Kenneth Nugent","doi":"10.14423/SMJ.0000000000001708","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001708","url":null,"abstract":"<p><strong>Objectives: </strong>Periodically, medical publications are retracted. The reasons vary from minor situations, such as author attributions, which do not undermine the validity of the data or the analysis in the article, to serious reasons, such as fraud. Understanding the reasons for retraction can provide important information for clinicians, educators, researchers, journals, and editorial boards.</p><p><strong>Methods: </strong>The PubMed database was searched using the term \"COVID-19\" (coronavirus disease 2019) and the term limitation \"retracted publication.\" The characteristics of the journals with retracted articles, the types of article, and the reasons for retraction were analyzed.</p><p><strong>Results: </strong>This search recovered 196 articles that had been retracted. These retractions were published in 179 different journals; 14 journals had >1 retracted article. The mean impact factor of these journals was 8.4, with a range of 0.32-168.9. The most frequent reasons for retractions were duplicate publication, concerns about data validity and analysis, concerns about peer review, author request, and the lack of permission or ethical violation. There were significant differences between the types of article and the reasons for retraction but no consistent pattern. A more detailed analysis of two particular retractions demonstrates the complexity and the effort required to make decisions about article retractions.</p><p><strong>Conclusions: </strong>The retraction of published articles presents a significant challenge to journals, editorial boards, peer reviewers, and authors. This process has the potential to provide important benefits; it also has the potential to undermine confidence in both research and the editorial process.</p>","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"358-363"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
William J Healy, Ali Musani, David J Fallaw, Shaheen U Islam
{"title":"Emerging Role of Artificial Intelligence in Academic Pulmonary Medicine.","authors":"William J Healy, Ali Musani, David J Fallaw, Shaheen U Islam","doi":"10.14423/SMJ.0000000000001704","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001704","url":null,"abstract":"","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"369-370"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Does Insurance Provide Adequate Coverage for Physical Therapy Visits for Common Orthopedic Diagnoses? A Survey Study.","authors":"Kaitlin Pyrz, Mathew Hargreaves, Audria Wood, Abdias Girardi, Gerald McGwin, Justin Kirk, Bolton Patton, Aaron Casp, Amit Momaya","doi":"10.14423/SMJ.0000000000001707","DOIUrl":"10.14423/SMJ.0000000000001707","url":null,"abstract":"<p><strong>Objectives: </strong>This study aimed to analyze the association between physical therapists' recommended number of visits for a full recovery from common orthopedic injuries/surgeries and the extent of insurance coverage for these visits.</p><p><strong>Methods: </strong>A prospective observational study was conducted with board-certified physical therapists. A qualitative questionnaire was used to gather physical therapists' demographics and the recommended number of physical therapy visits to achieve a full recovery after 11 common orthopedic diagnoses. Physical therapists also were asked to report whether they believe that insurance provides an adequate number of visits overall. In addition to the qualitative survey, insurance coverage details of major Alabama companies were obtained for comparison. Descriptive statistics of the participating therapists were analyzed for sex, age, degree/training, and years of experience. Kruskal-Wallis statistics were used to analyze variance between the aforementioned groupings when compared with the reported average number of sessions.</p><p><strong>Results: </strong>The survey (N = 251) collected data on the average number of physical therapy sessions that are necessary for a complete recovery as recommended by physical therapists for 11 common orthopedic diagnoses. From this survey, the average number of necessary visits ranged from 11.3 visits (ankle sprains) to 37.3 visits (anterior cruciate ligament reconstruction), with the overall average number of visits being 23.8. Only 24% of physical therapists believed that insurance companies provided enough coverage. Insurance coverage varied but often required additional procedures to allocate the adequate number of visits for the studied orthopedic pathologies.</p><p><strong>Conclusions: </strong>The majority of practicing physical therapists in Alabama perceive insufficient insurance coverage for physical therapy visits for most orthopedic diagnoses. This study has implications for healthcare decision making and patient-centered rehabilitation goals. Physicians and physical therapists can use this information to optimize treatment decisions and rehabilitation goals. Patients will benefit from improved physical and economic well-being. This study has the potential to drive further research and influence national insurance policies to better serve patients' needs.</p>","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"353-357"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499041","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Narrative Medicine and Overcoming Implicit Bias.","authors":"Joseph Abraham Kavian","doi":"10.14423/SMJ.0000000000001701","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001701","url":null,"abstract":"","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"364"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Devon Mills, Mary Nelson Robertson, Brian E Mills, David R Buys
{"title":"A Proportionate Mortality Study of Mississippi's Agricultural Industry, 2017-2021.","authors":"Devon Mills, Mary Nelson Robertson, Brian E Mills, David R Buys","doi":"10.14423/SMJ.0000000000001702","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001702","url":null,"abstract":"<p><strong>Objectives: </strong>Individuals employed in the agricultural industry encounter hazards in their work that could lead to injury or illness. Furthermore, the mental stress of being involved in the agricultural industry could lead to negative health-related outcomes for workers. This study evaluates the causes of deaths among employees in Mississippi's agricultural industry from 2017 to 2021.</p><p><strong>Methods: </strong>Data are provided by the Mississippi Department of Health. Proportionate mortality ratios (PMRs) are calculated to determine if agricultural industry employees show an elevated mortality in comparison to the general population for any cause of death.</p><p><strong>Results: </strong>Agricultural industry employees show a statistically significant elevated mortality for circulatory disease (PMR 107, 95% confidence interval [CI] 103-110) and coronavirus disease 2019 (PMR 122, 95% CI 111-134). They also show a significant excess mortality for deaths caused by transport accidents (PMR 117, 95% CI 101-136) and exposure to inanimate mechanical forces (PMR 274, 95% CI 183-396).</p><p><strong>Conclusions: </strong>The causes of death for which agricultural employees show an excess mortality can be explained by the hazards associated with working in the agricultural industry. These findings can be used to create targeted future public health programs for individuals who are employed in agriculture.</p>","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"379-382"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using a Chat-Based Trivia Activity to Foster Community in Residency.","authors":"Orhue Odaro, Jonathan Lim","doi":"10.14423/SMJ.0000000000001705","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001705","url":null,"abstract":"<p><strong>Objectives: </strong>For residency programs rotating at multiple sites, building a strong community can be challenging when house staff are geographically separated. Medical educators have had widespread use of technology to create virtual classrooms, discussion boards, and other activities. Less is known, however, about smaller-scale use of technology such as longitudinal use of chat to engage learners. We developed a chat-based trivia activity using social media tools to promote learning, community, and belonging in a large multisite residency program.</p><p><strong>Methods: </strong>Residents in our large academic program were invited to participate in a question-based activity called Internal Medicine Trivia Thursdays (IMTT) via the chat application GroupMe. Three to five questions were asked of all of the participants using a multimedia format. Question content included topics from the residency didactic curriculum and trivia about program leadership. A voluntary, anonymous survey on the effect of the activity on learning and belonging was sent to all of the residents at the end of the academic year.</p><p><strong>Results: </strong>Of the 224 residents, there were 48 survey respondents (21.4% response rate). When asked about overall satisfaction with the program, 43.8% (21/48) of all of the respondents reported feeling \"somewhat satisfied\" or \"very satisfied.\" Residents who frequently participated in Internal Medicine Trivia Thursdays experienced greater excitement about learning and a greater sense of community compared with those with infrequent to no participation.</p><p><strong>Conclusions: </strong>Our intervention used a theoretical framework of connectivism to design a virtual learning activity to engage residents, as well as to foster community among residents and between residents and program leadership. We believe this virtual learning experience is low cost and feasible, requiring mostly facilitator time. This study also contributes to the literature by evaluating outcomes related to social belonging and engagement. Future iterations should aim to optimize the methods of delivery by considering user-friendliness and the ability to opt out of the activity.</p>","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"365-368"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Margaux Meilhac, Suzanne Nesbit, Lindsay A Bowman, Rosalyn W Stewart
{"title":"Know Your Guidelines Series: The CDC Clinical Practice Guideline for Prescribing Opioids for Pain.","authors":"Margaux Meilhac, Suzanne Nesbit, Lindsay A Bowman, Rosalyn W Stewart","doi":"10.14423/SMJ.0000000000001703","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001703","url":null,"abstract":"","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"371-373"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Martin G McCandless, Madyson I Brown, James M Shiflett, Kristin J Weaver, Ian C Hoppe, Laura S Humphries
{"title":"Epidemiologic Assessment of Craniosynostosis in Mississippi's Pediatric Population from 2015 to 2020.","authors":"Martin G McCandless, Madyson I Brown, James M Shiflett, Kristin J Weaver, Ian C Hoppe, Laura S Humphries","doi":"10.14423/SMJ.0000000000001706","DOIUrl":"https://doi.org/10.14423/SMJ.0000000000001706","url":null,"abstract":"<p><strong>Objectives: </strong>This study aimed to report geographic and demographic patterns of patients with craniosynostosis (CS) treated at Children's of Mississippi, the state's only American Cleft Palate-Craniofacial Association-approved craniofacial team.</p><p><strong>Methods: </strong>Patients with CS were treated at a tertiary pediatric hospital cared for by craniofacial surgeons and neurosurgeons from 2015 to 2020. Demographic, geographic, and CS diagnosis details, including sex, gestational age, race, ethnicity, insurance status, and affected cranial suture type(s), number, and associated syndromic diagnosis were collected, including birth county and total live births from state data. Significant differences between prevalence of CS in four regions of Mississippi were examined using two-tailed <i>t</i> tests (<i>P</i> < 0.05).</p><p><strong>Results: </strong>Among 222,819 live births in Mississippi between 2015 and 2020, 79 pediatric patients presented to Children's of Mississippi with CS, with an overall incidence of 0.355/1000 live births. Most cases were nonsyndromic CS (82%, n = 65) affecting a single major cranial suture (81%, n = 64). The overall incidence of CS was higher in the coastal and central regions compared with northeast Mississippi, at 0.333 and 0.527 vs 0.132/1000 live births (<i>P =</i> 0.012 and <i>P</i> = 0.004), respectively.</p><p><strong>Conclusions: </strong>Results from this study suggest regional patterns of CS in Mississippi, which may reflect actual incidence patterns or proximity to Children's of Mississippi. Further study could reveal regional differences in risk factors underlying CS incidence or access to specialized CS care for different regions in the state. This will lead to opportunities for institutional outreach to decrease the burden of CS care in Mississippi.</p>","PeriodicalId":22043,"journal":{"name":"Southern Medical Journal","volume":"117 7","pages":"383-388"},"PeriodicalIF":1.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141499043","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}