Kanishk Gandhi, Zoe Lynch, Jan-Philipp Fränken, Kayla Patterson, Sharon Wambu, Tobias Gerstenberg, Desmond C. Ong, Noah D. Goodman
{"title":"基金会模型中的类人情感认知","authors":"Kanishk Gandhi, Zoe Lynch, Jan-Philipp Fränken, Kayla Patterson, Sharon Wambu, Tobias Gerstenberg, Desmond C. Ong, Noah D. Goodman","doi":"arxiv-2409.11733","DOIUrl":null,"url":null,"abstract":"Understanding emotions is fundamental to human interaction and experience.\nHumans easily infer emotions from situations or facial expressions, situations\nfrom emotions, and do a variety of other \\emph{affective cognition}. How adept\nis modern AI at these inferences? We introduce an evaluation framework for\ntesting affective cognition in foundation models. Starting from psychological\ntheory, we generate 1,280 diverse scenarios exploring relationships between\nappraisals, emotions, expressions, and outcomes. We evaluate the abilities of\nfoundation models (GPT-4, Claude-3, Gemini-1.5-Pro) and humans (N = 567) across\ncarefully selected conditions. Our results show foundation models tend to agree\nwith human intuitions, matching or exceeding interparticipant agreement. In\nsome conditions, models are ``superhuman'' -- they better predict modal human\njudgements than the average human. All models benefit from chain-of-thought\nreasoning. This suggests foundation models have acquired a human-like\nunderstanding of emotions and their influence on beliefs and behavior.","PeriodicalId":501030,"journal":{"name":"arXiv - CS - Computation and Language","volume":"76 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Human-like Affective Cognition in Foundation Models\",\"authors\":\"Kanishk Gandhi, Zoe Lynch, Jan-Philipp Fränken, Kayla Patterson, Sharon Wambu, Tobias Gerstenberg, Desmond C. Ong, Noah D. Goodman\",\"doi\":\"arxiv-2409.11733\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Understanding emotions is fundamental to human interaction and experience.\\nHumans easily infer emotions from situations or facial expressions, situations\\nfrom emotions, and do a variety of other \\\\emph{affective cognition}. How adept\\nis modern AI at these inferences? We introduce an evaluation framework for\\ntesting affective cognition in foundation models. Starting from psychological\\ntheory, we generate 1,280 diverse scenarios exploring relationships between\\nappraisals, emotions, expressions, and outcomes. We evaluate the abilities of\\nfoundation models (GPT-4, Claude-3, Gemini-1.5-Pro) and humans (N = 567) across\\ncarefully selected conditions. Our results show foundation models tend to agree\\nwith human intuitions, matching or exceeding interparticipant agreement. In\\nsome conditions, models are ``superhuman'' -- they better predict modal human\\njudgements than the average human. All models benefit from chain-of-thought\\nreasoning. This suggests foundation models have acquired a human-like\\nunderstanding of emotions and their influence on beliefs and behavior.\",\"PeriodicalId\":501030,\"journal\":{\"name\":\"arXiv - CS - Computation and Language\",\"volume\":\"76 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Computation and Language\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.11733\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Computation and Language","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.11733","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Human-like Affective Cognition in Foundation Models
Understanding emotions is fundamental to human interaction and experience.
Humans easily infer emotions from situations or facial expressions, situations
from emotions, and do a variety of other \emph{affective cognition}. How adept
is modern AI at these inferences? We introduce an evaluation framework for
testing affective cognition in foundation models. Starting from psychological
theory, we generate 1,280 diverse scenarios exploring relationships between
appraisals, emotions, expressions, and outcomes. We evaluate the abilities of
foundation models (GPT-4, Claude-3, Gemini-1.5-Pro) and humans (N = 567) across
carefully selected conditions. Our results show foundation models tend to agree
with human intuitions, matching or exceeding interparticipant agreement. In
some conditions, models are ``superhuman'' -- they better predict modal human
judgements than the average human. All models benefit from chain-of-thought
reasoning. This suggests foundation models have acquired a human-like
understanding of emotions and their influence on beliefs and behavior.