Comparative Efficacy of AI LLMs in Clinical Social Work: ChatGPT-4, Gemini, Copilot

IF 1.7 4区社会学 Q1 SOCIAL WORK

Research on Social Work Practice Pub Date : 2025-01-17 DOI:10.1177/10497315241313071

Hacer Taşkıran Tepe, Hüsnünur Aslantürk

引用次数: 0

Abstract

PurposeThis study examines the comparative efficacy of three AI large language models (LLMs)—ChatGPT-4, Gemini, and Microsoft Copilot—in clinical social work.MethodBy presenting scenarios of varying complexities, the study assessed their performance using the Ateşman Readability Index and a Likert-type accuracy scale.ResultsResults showed that Gemini had the highest accuracy, while Microsoft Copilot excelled in readability. Significant differences were found in accuracy scores ( p = .003), although readability differences were not statistically significant ( p = .054). No correlation was found between case complexity and either accuracy or readability.DiscussionDespite the differences, none of the models fully met all accuracy standards, indicating areas for further improvement. The findings suggest that while LLMs offer promise in social work, they require refinement to better meet the field's needs.

查看原文本刊更多论文

AI llm在临床社会工作中的比较疗效：ChatGPT-4， Gemini, Copilot

目的本研究探讨了三种人工智能大型语言模型（LLMs）--ChatGPT-4、Gemini 和 Microsoft Copilot 在临床社会工作中的功效比较。方法本研究通过呈现不同复杂程度的情景，使用阿特斯曼可读性指数和李克特式准确性量表评估了它们的性能。结果研究结果表明，Gemini 的准确性最高，而 Microsoft Copilot 的可读性更出色。虽然可读性差异没有统计学意义（p = .054），但准确性得分存在显著差异（p = .003）。讨论尽管存在差异，但没有一个模型完全符合所有准确性标准，这表明还有待进一步改进。研究结果表明，尽管 LLM 在社会工作领域大有可为，但仍需对其进行改进，以更好地满足该领域的需求。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Research on Social Work Practice SOCIAL WORK-

CiteScore

3.50

自引率

11.10%

发文量

105

期刊介绍： Research on Social Work Practice, sponsored by the Society for Social Work and Research, is a disciplinary journal devoted to the publication of empirical research concerning the methods and outcomes of social work practice. Social work practice is broadly interpreted to refer to the application of intentionally designed social work intervention programs to problems of societal and/or interpersonal importance, including behavior analysis or psychotherapy involving individuals; case management; practice involving couples, families, and small groups; community practice education; and the development, implementation, and evaluation of social policies.