{"title":"用 \"分叉的舌头 \"说话--法律硕士的用户评价与文本情感之间的错位","authors":"Yixing Yang, Jianxiong Huang","doi":"10.1108/k-06-2024-1458","DOIUrl":null,"url":null,"abstract":"<h3>Purpose</h3>\n<p>The study aims to provide concrete service remediation and enhancement for LLM developers such as getting user forgiveness and breaking through perceived bottlenecks. It also aims to improve the efficiency of app users' usage decisions.</p><!--/ Abstract__block -->\n<h3>Design/methodology/approach</h3>\n<p>This paper takes the user reviews of the app stores in 21 countries and 10 languages as the research data, extracts the potential factors by LDA model, exploratively takes the misalignment between user ratings and textual emotions as user forgiveness and perceived bottleneck and uses the Word2vec-SVM model to analyze the sentiment. Finally, attributions are made based on empathy.</p><!--/ Abstract__block -->\n<h3>Findings</h3>\n<p>The results show that AI-based LLMs are more likely to cause bias in user ratings and textual content than regular APPs. Functional and economic remedies are effective in awakening empathy and forgiveness, while empathic remedies are effective in reducing perceived bottlenecks. Interestingly, empathetic users are “pickier”. Further social network analysis reveals that problem solving timeliness, software flexibility, model updating and special data (voice and image) analysis capabilities are beneficial in breaking perceived bottlenecks. Besides, heterogeneity analysis show that eastern users are more sensitive to the price factor and are more likely to generate forgiveness through economic remedy, and there is a dual interaction between basic attributes and extra boosts in the East and West.</p><!--/ Abstract__block -->\n<h3>Originality/value</h3>\n<p>The “gap” between negative (positive) user reviews and ratings, that is consumer forgiveness and perceived bottlenecks, is identified in unstructured text; the study finds that empathy helps to awaken user forgiveness and understanding, while it is limited to bottleneck breakthroughs; the dataset includes a wide range of countries and regions, findings are tested in a cross-language and cross-cultural perspective, which makes the study more robust, and the heterogeneity of users' cultural backgrounds is also analyzed.</p><!--/ Abstract__block -->","PeriodicalId":49930,"journal":{"name":"Kybernetes","volume":"154 1","pages":""},"PeriodicalIF":2.5000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Speaking with a “forked tongue” – misalignment between user ratings and textual emotions in LLMs\",\"authors\":\"Yixing Yang, Jianxiong Huang\",\"doi\":\"10.1108/k-06-2024-1458\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<h3>Purpose</h3>\\n<p>The study aims to provide concrete service remediation and enhancement for LLM developers such as getting user forgiveness and breaking through perceived bottlenecks. It also aims to improve the efficiency of app users' usage decisions.</p><!--/ Abstract__block -->\\n<h3>Design/methodology/approach</h3>\\n<p>This paper takes the user reviews of the app stores in 21 countries and 10 languages as the research data, extracts the potential factors by LDA model, exploratively takes the misalignment between user ratings and textual emotions as user forgiveness and perceived bottleneck and uses the Word2vec-SVM model to analyze the sentiment. Finally, attributions are made based on empathy.</p><!--/ Abstract__block -->\\n<h3>Findings</h3>\\n<p>The results show that AI-based LLMs are more likely to cause bias in user ratings and textual content than regular APPs. Functional and economic remedies are effective in awakening empathy and forgiveness, while empathic remedies are effective in reducing perceived bottlenecks. Interestingly, empathetic users are “pickier”. Further social network analysis reveals that problem solving timeliness, software flexibility, model updating and special data (voice and image) analysis capabilities are beneficial in breaking perceived bottlenecks. Besides, heterogeneity analysis show that eastern users are more sensitive to the price factor and are more likely to generate forgiveness through economic remedy, and there is a dual interaction between basic attributes and extra boosts in the East and West.</p><!--/ Abstract__block -->\\n<h3>Originality/value</h3>\\n<p>The “gap” between negative (positive) user reviews and ratings, that is consumer forgiveness and perceived bottlenecks, is identified in unstructured text; the study finds that empathy helps to awaken user forgiveness and understanding, while it is limited to bottleneck breakthroughs; the dataset includes a wide range of countries and regions, findings are tested in a cross-language and cross-cultural perspective, which makes the study more robust, and the heterogeneity of users' cultural backgrounds is also analyzed.</p><!--/ Abstract__block -->\",\"PeriodicalId\":49930,\"journal\":{\"name\":\"Kybernetes\",\"volume\":\"154 1\",\"pages\":\"\"},\"PeriodicalIF\":2.5000,\"publicationDate\":\"2024-09-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Kybernetes\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1108/k-06-2024-1458\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, CYBERNETICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Kybernetes","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1108/k-06-2024-1458","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}
Speaking with a “forked tongue” – misalignment between user ratings and textual emotions in LLMs
Purpose
The study aims to provide concrete service remediation and enhancement for LLM developers such as getting user forgiveness and breaking through perceived bottlenecks. It also aims to improve the efficiency of app users' usage decisions.
Design/methodology/approach
This paper takes the user reviews of the app stores in 21 countries and 10 languages as the research data, extracts the potential factors by LDA model, exploratively takes the misalignment between user ratings and textual emotions as user forgiveness and perceived bottleneck and uses the Word2vec-SVM model to analyze the sentiment. Finally, attributions are made based on empathy.
Findings
The results show that AI-based LLMs are more likely to cause bias in user ratings and textual content than regular APPs. Functional and economic remedies are effective in awakening empathy and forgiveness, while empathic remedies are effective in reducing perceived bottlenecks. Interestingly, empathetic users are “pickier”. Further social network analysis reveals that problem solving timeliness, software flexibility, model updating and special data (voice and image) analysis capabilities are beneficial in breaking perceived bottlenecks. Besides, heterogeneity analysis show that eastern users are more sensitive to the price factor and are more likely to generate forgiveness through economic remedy, and there is a dual interaction between basic attributes and extra boosts in the East and West.
Originality/value
The “gap” between negative (positive) user reviews and ratings, that is consumer forgiveness and perceived bottlenecks, is identified in unstructured text; the study finds that empathy helps to awaken user forgiveness and understanding, while it is limited to bottleneck breakthroughs; the dataset includes a wide range of countries and regions, findings are tested in a cross-language and cross-cultural perspective, which makes the study more robust, and the heterogeneity of users' cultural backgrounds is also analyzed.
期刊介绍:
Kybernetes is the official journal of the UNESCO recognized World Organisation of Systems and Cybernetics (WOSC), and The Cybernetics Society.
The journal is an important forum for the exchange of knowledge and information among all those who are interested in cybernetics and systems thinking.
It is devoted to improvement in the understanding of human, social, organizational, technological and sustainable aspects of society and their interdependencies. It encourages consideration of a range of theories, methodologies and approaches, and their transdisciplinary links. The spirit of the journal comes from Norbert Wiener''s understanding of cybernetics as "The Human Use of Human Beings." Hence, Kybernetes strives for examination and analysis, based on a systemic frame of reference, of burning issues of ecosystems, society, organizations, businesses and human behavior.