Visual Informatics最新文献_第3页

MILG: Realistic lip-sync video generation with audio-modulated image inpainting MILG：利用音频调制图像绘制生成逼真的唇音同步视频

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.08.002

Han Bao , Xuhong Zhang , Qinying Wang , Kangming Liang , Zonghui Wang , Shouling Ji , Wenzhi Chen

{"title":"MILG: Realistic lip-sync video generation with audio-modulated image inpainting","authors":"Han Bao , Xuhong Zhang , Qinying Wang , Kangming Liang , Zonghui Wang , Shouling Ji , Wenzhi Chen","doi":"10.1016/j.visinf.2024.08.002","DOIUrl":"10.1016/j.visinf.2024.08.002","url":null,"abstract":"<div><div>Existing lip synchronization (lip-sync) methods generate accurately synchronized mouths and faces in a generated video. However, they still confront the problem of artifacts in regions of non-interest (RONI), <em>e.g.</em>, background and other parts of a face, which decreases the overall visual quality. To solve these problems, we innovatively introduce diverse image inpainting to lip-sync generation. We propose Modulated Inpainting Lip-sync GAN (MILG), an audio-constraint inpainting network to predict synchronous mouths. MILG utilizes prior knowledge of RONI and audio sequences to predict lip shape instead of image generation, which can keep the RONI consistent. Specifically, we integrate modulated spatially probabilistic diversity normalization (MSPD Norm) in our inpainting network, which helps the network generate fine-grained diverse mouth movements guided by the continuous audio features. Furthermore, to lower the training overhead, we modify the contrastive loss in lip-sync to support small-batch-size and few-sample training. Extensive experiments demonstrate that our approach outperforms the existing state-of-the-art of image quality and authenticity while keeping lip-sync.</div></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 3","pages":"Pages 71-81"},"PeriodicalIF":3.8,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142417742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Visual exploration of multi-dimensional data via rule-based sample embedding 通过基于规则的样本嵌入对多维数据进行可视化探索

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.09.005

Tong Zhang, Jie Li, Chao Xu

引用次数: 0

RelicCARD: Enhancing cultural relics exploration through semantics-based augmented reality tangible interaction design RelicCARD：通过基于语义的增强现实有形交互设计加强文物探索

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.06.003

Tao Yu , Shaoxuan Lai , Wenjin Zhang , Jun Cui , Jun Tao

{"title":"RelicCARD: Enhancing cultural relics exploration through semantics-based augmented reality tangible interaction design","authors":"Tao Yu , Shaoxuan Lai , Wenjin Zhang , Jun Cui , Jun Tao","doi":"10.1016/j.visinf.2024.06.003","DOIUrl":"10.1016/j.visinf.2024.06.003","url":null,"abstract":"<div><div>Cultural relics visualization brings digital archives of relics to broader audiences in many applications, such as education, historical research, and virtual museums. However, previous research mainly focused on modeling and rendering the relics. While enhancing accessibility, these techniques still provide limited ability to improve user engagement. In this paper, we introduce RelicCARD, a semantics-based augmented reality (AR) tangible interaction design for exploring cultural relics. Our design uses an easily available tangible interface to encourage the users to interact with a large collection of relics. The tangible interface allows users to explore, select, and arrange relics to form customized scenes. To guide the design of the interface, we formalize a design space by connecting the semantics in relics, the tangible interaction patterns, and the exploration tasks. We realize the design space as a tangible interactive prototype and examine its feasibility and effectiveness using multiple case studies and an expert evaluation. Finally, we discuss the findings in the evaluation and future directions to improve the design and implementation of the interactive design space.</div></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 3","pages":"Pages 32-41"},"PeriodicalIF":3.8,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141839325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

RenderKernel: High-level programming for real-time rendering systems RenderKernel：实时渲染系统的高级编程

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.09.004

Jinyuan Yang, Soumyabrata Dev, Abraham G. Campbell

{"title":"RenderKernel: High-level programming for real-time rendering systems","authors":"Jinyuan Yang, Soumyabrata Dev, Abraham G. Campbell","doi":"10.1016/j.visinf.2024.09.004","DOIUrl":"10.1016/j.visinf.2024.09.004","url":null,"abstract":"<div><div>Real-time rendering applications leverage heterogeneous computing to optimize performance. However, software development across multiple devices presents challenges, including data layout inconsistencies, synchronization issues, resource management complexities, and architectural disparities. Additionally, the creation of such systems requires verbose and unsafe programming models. Recent developments in domain-specific and unified shading languages aim to mitigate these issues. Yet, current programming models primarily address data layout consistency, neglecting other persistent challenges.In this paper, we introduce RenderKernel, a programming model designed to simplify the development of real-time rendering systems. Recognizing the need for a high-level approach, RenderKernel addresses the specific challenges of real-time rendering, enabling development on heterogeneous systems as if they were homogeneous. This model allows for early detection and prevention of errors due to system heterogeneity at compile-time. Furthermore, RenderKernel enables the use of common programming patterns from homogeneous environments, freeing developers from the complexities of underlying heterogeneous systems. Developers can focus on coding unique application features, thereby enhancing productivity and reducing the cognitive load associated with real-time rendering system development.</div></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 3","pages":"Pages 82-95"},"PeriodicalIF":3.8,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142417743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Demers cartogram with rivers 带有河流的 Demers 地图

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.09.003

Qiru Wang, Kai Xu, Robert S. Laramee

引用次数: 0

JobViz: Skill-driven visual exploration of job advertisements JobViz：以技能为导向的招聘广告可视化探索

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.07.001

Ran Wang , Qianhe Chen , Yong Wang , Lewei Xiong , Boyang Shen

{"title":"JobViz: Skill-driven visual exploration of job advertisements","authors":"Ran Wang , Qianhe Chen , Yong Wang , Lewei Xiong , Boyang Shen","doi":"10.1016/j.visinf.2024.07.001","DOIUrl":"10.1016/j.visinf.2024.07.001","url":null,"abstract":"<div><p>Online job advertisements on various job portals or websites have become the most popular way for people to find potential career opportunities nowadays. However, the majority of these job sites are limited to offering fundamental filters such as job titles, keywords, and compensation ranges. This often poses a challenge for job seekers in efficiently identifying relevant job advertisements that align with their unique skill sets amidst a vast sea of listings. Thus, we propose well-coordinated visualizations to provide job seekers with three levels of details of job information: a skill-job overview visualizes skill sets, employment posts as well as relationships between them with a hierarchical visualization design; a post exploration view leverages an augmented radar-chart glyph to represent job posts and further facilitates users’ swift comprehension of the pertinent skills necessitated by respective positions; a post detail view lists the specifics of selected job posts for profound analysis and comparison. By using a real-world recruitment advertisement dataset collected from 51Job, one of the largest job websites in China, we conducted two case studies and user interviews to evaluate <em>JobViz</em>. The results demonstrated the usefulness and effectiveness of our approach.</p></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 3","pages":"Pages 18-28"},"PeriodicalIF":3.8,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2468502X24000391/pdfft?md5=62d1e06a4ba3529c504c7ac24e65e000&pid=1-s2.0-S2468502X24000391-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141843719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Visual evaluation of graph representation learning based on the presentation of community structures 基于群落结构呈现的图形表示学习可视化评估

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-09-01 DOI: 10.1016/j.visinf.2024.08.001

Yong Zhang , Lihong Cai , Yuhua Liu , Yize Li , Songyue Li , Yuming Ma , Yuwei Meng , Zhiguang Zhou

引用次数: 0

VisAhoi: Towards a library to generate and integrate visualization onboarding using high-level visualization grammars VisAhoi：使用高级可视化语法生成和集成可视化入门库

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-06-27 DOI: 10.1016/j.visinf.2024.06.001

Christina Stoiber , Daniela Moitzi , Holger Stitz , Florian Grassinger , Anto Silviya Geo Prakash , Dominic Girardi , Marc Streit , Wolfgang Aigner

{"title":"VisAhoi: Towards a library to generate and integrate visualization onboarding using high-level visualization grammars","authors":"Christina Stoiber , Daniela Moitzi , Holger Stitz , Florian Grassinger , Anto Silviya Geo Prakash , Dominic Girardi , Marc Streit , Wolfgang Aigner","doi":"10.1016/j.visinf.2024.06.001","DOIUrl":"10.1016/j.visinf.2024.06.001","url":null,"abstract":"<div><p>Visualization onboarding supports users in reading, interpreting, and extracting information from visual data representations. General-purpose onboarding tools and libraries are applicable for explaining a wide range of graphical user interfaces but cannot handle specific visualization requirements. This paper describes a first step towards developing an onboarding library called VisAhoi, which is easy to <em>integrate, extend, semi-automate, reuse, and customize</em>. VisAhoi supports the creation of onboarding elements for different visualization types and datasets. We demonstrate how to extract and describe onboarding instructions using three well-known high-level descriptive visualization grammars — Vega-Lite, Plotly.js, and ECharts. We show the applicability of our library by performing two usage scenarios that describe the integration of VisAhoi into a VA tool for the analysis of high-throughput screening (HTS) data and, second, into a Flourish template to provide an authoring tool for data journalists for a treemap visualization. We provide a supplementary website (<span><span>https://datavisyn.github.io/visAhoi/</span><svg><path></path></svg></span>) that demonstrates the applicability of VisAhoi to various visualizations, including a bar chart, a horizon graph, a change matrix/heatmap, a scatterplot, and a treemap visualization.</p></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 3","pages":"Pages 1-17"},"PeriodicalIF":3.8,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2468502X24000214/pdfft?md5=b500608cf3b6d6a02fdc48334024bff3&pid=1-s2.0-S2468502X24000214-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141954338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Generative AI for visualization: State of the art and future directions 用于可视化的生成式人工智能：技术现状与未来方向

IF 3 3区计算机科学

Visual Informatics Pub Date : 2024-06-01 DOI: 10.1016/j.visinf.2024.04.003

Yilin Ye , Jianing Hao , Yihan Hou , Zhan Wang , Shishi Xiao , Yuyu Luo , Wei Zeng

{"title":"Generative AI for visualization: State of the art and future directions","authors":"Yilin Ye , Jianing Hao , Yihan Hou , Zhan Wang , Shishi Xiao , Yuyu Luo , Wei Zeng","doi":"10.1016/j.visinf.2024.04.003","DOIUrl":"https://doi.org/10.1016/j.visinf.2024.04.003","url":null,"abstract":"<div><p>Generative AI (GenAI) has witnessed remarkable progress in recent years and demonstrated impressive performance in various generation tasks in different domains such as computer vision and computational design. Many researchers have attempted to integrate GenAI into visualization framework, leveraging the superior generative capacity for different operations. Concurrently, recent major breakthroughs in GenAI like diffusion models and large language models have also drastically increased the potential of GenAI4VIS. From a technical perspective, this paper looks back on previous visualization studies leveraging GenAI and discusses the challenges and opportunities for future research. Specifically, we cover the applications of different types of GenAI methods including sequence, tabular, spatial and graph generation techniques for different tasks of visualization which we summarize into four major stages: data enhancement, visual mapping generation, stylization and interaction. For each specific visualization sub-task, we illustrate the typical data and concrete GenAI algorithms, aiming to provide in-depth understanding of the state-of-the-art GenAI4VIS techniques and their limitations. Furthermore, based on the survey, we discuss three major aspects of challenges and research opportunities including evaluation, dataset, and the gap between end-to-end GenAI methods and visualizations. By summarizing different generation algorithms, their current applications and limitations, this paper endeavors to provide useful insights for future GenAI4VIS research.</p></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 2","pages":"Pages 43-66"},"PeriodicalIF":3.0,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2468502X24000160/pdfft?md5=c309ceeb991e85de9bb7a69d53c32032&pid=1-s2.0-S2468502X24000160-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141328799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The whole and its parts: Visualizing Gaussian mixture models 整体及其部分高斯混合模型可视化

IF 3.8 3区计算机科学

Visual Informatics Pub Date : 2024-06-01 DOI: 10.1016/j.visinf.2024.04.005

Joachim Giesen , Philipp Lucas , Linda Pfeiffer , Laines Schmalwasser , Kai Lawonn

{"title":"The whole and its parts: Visualizing Gaussian mixture models","authors":"Joachim Giesen , Philipp Lucas , Linda Pfeiffer , Laines Schmalwasser , Kai Lawonn","doi":"10.1016/j.visinf.2024.04.005","DOIUrl":"10.1016/j.visinf.2024.04.005","url":null,"abstract":"<div><p>Gaussian mixture models are classical but still popular machine learning models. An appealing feature of Gaussian mixture models is their tractability, that is, they can be learned efficiently and exactly from data, and also support efficient exact inference queries like soft clustering data points. Only seemingly simple, Gaussian mixture models can be hard to understand. There are at least four aspects to understanding Gaussian mixture models, namely, understanding the whole distribution, its individual parts (mixture components), the relationships between the parts, and the interplay of the whole and its parts. In a structured literature review of applications of Gaussian mixture models, we found the need for supporting all four aspects. To identify candidate visualizations that effectively aid the user needs, we structure the available design space along three different representations of Gaussian mixture models, namely as functions, sets of parameters, and sampling processes. From the design space, we implemented three design concepts that visualize the overall distribution together with its components. Finally, we assessed the practical usefulness of the design concepts with respect to the different user needs in expert interviews and an insight-based user study.</p></div>","PeriodicalId":36903,"journal":{"name":"Visual Informatics","volume":"8 2","pages":"Pages 67-79"},"PeriodicalIF":3.8,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2468502X24000196/pdfft?md5=00c8e5cb6796e180f1417fb1e6e4984c&pid=1-s2.0-S2468502X24000196-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141031229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0