Exploring the neural landscape: Visual analytics of neuron activation in large language models with NeuronautLLM

IF 2.5 4区计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Graphical Models Pub Date : 2024-10-19 DOI:10.1016/j.gmod.2024.101238

Ollie Woodman , Zhen Wen , Hui Lu , Yiwen Ren , Minfeng Zhu , Wei Chen

{"title":"Exploring the neural landscape: Visual analytics of neuron activation in large language models with NeuronautLLM","authors":"Ollie Woodman , Zhen Wen , Hui Lu , Yiwen Ren , Minfeng Zhu , Wei Chen","doi":"10.1016/j.gmod.2024.101238","DOIUrl":null,"url":null,"abstract":"<div><div>Large language models (LLMs) like those that power OpenAI’s ChatGPT and Google’s Gemini have played a major part in the recent wave of machine learning and artificial intelligence advancements. However, interpreting LLMs and visualizing their components is extremely difficult due to the incredible scale and high dimensionality of model data. NeuronautLLM introduces a visual analysis system for identifying and visualizing influential neurons in transformer-based language models as they relate to user-defined prompts. Our approach combines simple, yet information-dense visualizations as well as neuron explanation and classification data to provide a wealth of opportunities for exploration. NeuronautLLM was reviewed by two experts to verify its efficacy as a tool for practical model interpretation. Interviews and usability tests with five LLM experts demonstrated NeuronautLLM’s exceptional usability and its readiness for real-world application. Furthermore, two in-depth case studies on model reasoning and social bias highlight NeuronautLLM’s versatility in aiding the analysis of a wide range of LLM research problems.</div></div>","PeriodicalId":55083,"journal":{"name":"Graphical Models","volume":"136 ","pages":"Article 101238"},"PeriodicalIF":2.5000,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Graphical Models","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1524070324000262","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

Large language models (LLMs) like those that power OpenAI’s ChatGPT and Google’s Gemini have played a major part in the recent wave of machine learning and artificial intelligence advancements. However, interpreting LLMs and visualizing their components is extremely difficult due to the incredible scale and high dimensionality of model data. NeuronautLLM introduces a visual analysis system for identifying and visualizing influential neurons in transformer-based language models as they relate to user-defined prompts. Our approach combines simple, yet information-dense visualizations as well as neuron explanation and classification data to provide a wealth of opportunities for exploration. NeuronautLLM was reviewed by two experts to verify its efficacy as a tool for practical model interpretation. Interviews and usability tests with five LLM experts demonstrated NeuronautLLM’s exceptional usability and its readiness for real-world application. Furthermore, two in-depth case studies on model reasoning and social bias highlight NeuronautLLM’s versatility in aiding the analysis of a wide range of LLM research problems.

查看原文本刊更多论文

探索神经景观：利用 NeuronautLLM 对大型语言模型中的神经元激活进行可视化分析

大型语言模型（LLM）（如 OpenAI 的 ChatGPT 和谷歌的 Gemini）在最近的机器学习和人工智能进步浪潮中发挥了重要作用。然而，由于模型数据规模大、维度高，解释 LLM 和可视化其组成部分极其困难。NeuronautLLM 引入了一个可视化分析系统，用于识别和可视化基于变压器的语言模型中与用户定义的提示相关的有影响力的神经元。我们的方法结合了简单但信息密集的可视化以及神经元解释和分类数据，为探索提供了大量机会。两位专家对 NeuronautLLM 进行了审查，以验证其作为实用模型解释工具的有效性。与五位 LLM 专家的访谈和可用性测试表明，NeuronautLLM 具有出色的可用性，并可在现实世界中应用。此外，关于模型推理和社会偏见的两个深入案例研究突出了 NeuronautLLM 在帮助分析各种 LLM 研究问题方面的多功能性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Graphical Models 工程技术-计算机：软件工程

CiteScore

3.60

自引率

5.90%

发文量

审稿时长

47 days

期刊介绍： Graphical Models is recognized internationally as a highly rated, top tier journal and is focused on the creation, geometric processing, animation, and visualization of graphical models and on their applications in engineering, science, culture, and entertainment. GMOD provides its readers with thoroughly reviewed and carefully selected papers that disseminate exciting innovations, that teach rigorous theoretical foundations, that propose robust and efficient solutions, or that describe ambitious systems or applications in a variety of topics. We invite papers in five categories: research (contributions of novel theoretical or practical approaches or solutions), survey (opinionated views of the state-of-the-art and challenges in a specific topic), system (the architecture and implementation details of an innovative architecture for a complete system that supports model/animation design, acquisition, analysis, visualization?), application (description of a novel application of know techniques and evaluation of its impact), or lecture (an elegant and inspiring perspective on previously published results that clarifies them and teaches them in a new way). GMOD offers its authors an accelerated review, feedback from experts in the field, immediate online publication of accepted papers, no restriction on color and length (when justified by the content) in the online version, and a broad promotion of published papers. A prestigious group of editors selected from among the premier international researchers in their fields oversees the review process.