A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design

IF 4.7 1区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

IEEE Transactions on Visualization and Computer Graphics Pub Date : 2022-09-26 DOI:10.48550/arXiv.2209.12588

F. Ibarrola, Tomas Lawton, Kazjon Grace

{"title":"A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design","authors":"F. Ibarrola, Tomas Lawton, Kazjon Grace","doi":"10.48550/arXiv.2209.12588","DOIUrl":null,"url":null,"abstract":"Recent advances in text-conditioned generative models have provided us with neural networks capable of creating images of astonishing quality, be they realistic, abstract, or even creative. These models have in common that (more or less explicitly) they all aim to produce a high-quality one-off output given certain conditions, and in that they are not well suited for a creative collaboration framework. Drawing on theories from cognitive science that model how professional designers and artists think, we argue how this setting differs from the former and introduce CICADA: a Collaborative, Interactive Context-Aware Drawing Agent. CICADA uses a vector-based synthesis-by-optimisation method to take a partial sketch (such as might be provided by a user) and develop it towards a goal by adding and/or sensibly modifying traces. Given that this topic has been scarcely explored, we also introduce a way to evaluate desired characteristics of a model in this context by means of proposing a diversity measure. CICADA is shown to produce sketches of quality comparable to a human user's, enhanced diversity and most importantly to be able to cope with change by continuing the sketch minding the user's contributions in a flexible manner.","PeriodicalId":13376,"journal":{"name":"IEEE Transactions on Visualization and Computer Graphics","volume":" ","pages":""},"PeriodicalIF":4.7000,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Visualization and Computer Graphics","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.48550/arXiv.2209.12588","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 4

Abstract

Recent advances in text-conditioned generative models have provided us with neural networks capable of creating images of astonishing quality, be they realistic, abstract, or even creative. These models have in common that (more or less explicitly) they all aim to produce a high-quality one-off output given certain conditions, and in that they are not well suited for a creative collaboration framework. Drawing on theories from cognitive science that model how professional designers and artists think, we argue how this setting differs from the former and introduce CICADA: a Collaborative, Interactive Context-Aware Drawing Agent. CICADA uses a vector-based synthesis-by-optimisation method to take a partial sketch (such as might be provided by a user) and develop it towards a goal by adding and/or sensibly modifying traces. Given that this topic has been scarcely explored, we also introduce a way to evaluate desired characteristics of a model in this context by means of proposing a diversity measure. CICADA is shown to produce sketches of quality comparable to a human user's, enhanced diversity and most importantly to be able to cope with change by continuing the sketch minding the user's contributions in a flexible manner.

查看原文本刊更多论文

一种协作、交互和上下文感知的协同设计绘图代理

文本条件生成模型的最新进展为我们提供了能够创建质量惊人的图像的神经网络，无论是逼真的、抽象的，还是创造性的。这些模型的共同点是（或多或少明确地），它们都旨在在特定条件下产生高质量的一次性产出，而且它们不太适合创造性的合作框架。根据认知科学中模拟专业设计师和艺术家思考方式的理论，我们讨论了这种设置与前者的区别，并介绍了CICADA：一种协作、交互式上下文感知的绘图代理。CICADA使用基于矢量的优化合成方法来绘制局部草图（例如用户可能提供的草图），并通过添加和/或合理修改轨迹来实现目标。鉴于这一主题很少被探索，我们还介绍了一种方法，通过提出多样性度量来评估模型在这一背景下的期望特征。CICADA被证明能够制作出与人类用户质量相当的草图，增强了多样性，最重要的是能够通过以灵活的方式继续关注用户的贡献来应对变化。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Visualization and Computer Graphics 工程技术-计算机：软件工程

CiteScore

10.40

自引率

19.20%

发文量

946

审稿时长

4.5 months

期刊介绍： TVCG is a scholarly, archival journal published monthly. Its Editorial Board strives to publish papers that present important research results and state-of-the-art seminal papers in computer graphics, visualization, and virtual reality. Specific topics include, but are not limited to: rendering technologies; geometric modeling and processing; shape analysis; graphics hardware; animation and simulation; perception, interaction and user interfaces; haptics; computational photography; high-dynamic range imaging and display; user studies and evaluation; biomedical visualization; volume visualization and graphics; visual analytics for machine learning; topology-based visualization; visual programming and software visualization; visualization in data science; virtual reality, augmented reality and mixed reality; advanced display technology, (e.g., 3D, immersive and multi-modal displays); applications of computer graphics and visualization.