How to evaluate the cognitive abilities of LLMs

IF 21.4 1区心理学 Q1 MULTIDISCIPLINARY SCIENCES

Nature Human Behaviour Pub Date : 2025-01-15 DOI:10.1038/s41562-024-02096-z

Anna A. Ivanova

引用次数: 0

Abstract

Language models have become an essential part of the burgeoning field of artificial intelligence (AI) psychology. I discuss 14 methodological considerations that can be used to design more robust, generalizable studies that evaluate the cognitive abilities of language-based AI systems, as well as to accurately interpret the results of these studies.

查看原文本刊更多论文

如何评价法学硕士的认知能力

语言模型已经成为新兴的人工智能（AI）心理学领域的重要组成部分。我讨论了14种方法上的考虑，可以用来设计更健壮的、可推广的研究，以评估基于语言的人工智能系统的认知能力，并准确地解释这些研究的结果。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Nature Human Behaviour Psychology-Social Psychology

CiteScore

36.80

自引率

1.00%

发文量

227

期刊介绍： Nature Human Behaviour is a journal that focuses on publishing research of outstanding significance into any aspect of human behavior.The research can cover various areas such as psychological, biological, and social bases of human behavior.It also includes the study of origins, development, and disorders related to human behavior.The primary aim of the journal is to increase the visibility of research in the field and enhance its societal reach and impact.