Akinobu Maejima, Hiroyuki Kubo, Seitaro Shinagawa, Takuya Funatomi, T. Yotsukura, Satoshi Nakamura, Y. Mukaigawa
{"title":"Anime Character Colorization using Few-shot Learning","authors":"Akinobu Maejima, Hiroyuki Kubo, Seitaro Shinagawa, Takuya Funatomi, T. Yotsukura, Satoshi Nakamura, Y. Mukaigawa","doi":"10.1145/3478512.3488604","DOIUrl":"https://doi.org/10.1145/3478512.3488604","url":null,"abstract":"In this paper, we propose an automatic Anime-style colorization method using only a small number of colorized reference images manually colorized by artists. To accomplish this, we introduce a few-shot patch-based learning method considering the characteristics of Anime line-drawing. To streamline the learning process, we derive optimal settings with acceptable colorization accuracy and training time for a production pipeline. We demonstrate that the proposed method helps to reduce manual labor for artists.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114172472","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Skeleton2Stroke: Interactive Stroke Correspondence Editing with Pose Features","authors":"Ryoma Miyauchi, Yichen Peng, Tsukasa Fukusato, Haoran Xie","doi":"10.1145/3478512.3488612","DOIUrl":"https://doi.org/10.1145/3478512.3488612","url":null,"abstract":"Inbetweening is an important technique for computer animations where the stroke correspondence of hand-drawn illustrations plays a significant role. Previous works typically require image vectorization and enormous computation cost to achieve this goal. In this paper, we propose an interactive method to construct stroke correspondences in character illustrations. First, we utilize a deep learning-based skeleton estimation to improve the accuracy of closed-area correspondences, which are obtained using greedy algorithm. Second, we construct stroke correspondences based on the estimated closed-area correspondences. The proposed user interface is verified by our experiment to ensure that the users can achieve high accuracy with low correction in stroke correspondence.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128003604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Multi-Stage Advanced Deep Learning Graphics Pipeline","authors":"Mark Wesley Harris, S. Semwal","doi":"10.1145/3478512.3488609","DOIUrl":"https://doi.org/10.1145/3478512.3488609","url":null,"abstract":"In this paper we propose the Advanced Deep Learning Graphics Pipeline (ADLGP). ADLGP is a novel approach that uses existing deep learning architectures to convert scene data into rendered images. Our goal of generating frames from semantic data has produced successful renderings with similar structures and composition as target frames. We demonstrate the success of ADLGP with side-by-side comparisons of frames generated through standard rendering procedures. We assert that a fully implemented ADLGP framework would reduce the time spent in visualizing 3D environments, and help selectively offload the requirements of the current graphics rendering pipeline.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131218688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mehmet Oguz Derin, T. Harada, Yusuke Takeda, Yasuhiro Iba
{"title":"Sparse Volume Rendering using Hardware Ray Tracing and Block Walking","authors":"Mehmet Oguz Derin, T. Harada, Yusuke Takeda, Yasuhiro Iba","doi":"10.1145/3478512.3488608","DOIUrl":"https://doi.org/10.1145/3478512.3488608","url":null,"abstract":"We propose a method to render sparse volumetric data using ray-tracing hardware efficiently. To realize this, we introduce a novel data structure, traversal algorithm, and density encoding that allows for an annotated BVH representation. In order to avoid API calls to ray tracing hardware which reduces the efficiency in the rendering, we propose the block walking for which we store information about adjacent nodes in each BVH node’s corresponding field, taking advantage of the knowledge of the content layout. Doing so enables us to traverse the tree more efficiently without repeatedly accessing the spatial acceleration structure maintained by the driver. We demonstrate that our method achieves higher performance and scalability with little memory overhead, enabling interactive rendering of volumetric data.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124607525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SpiCa: Stereoscopic Effect Design with 3D Pottery Wheel-type Transparent Canvas","authors":"Riwano Ikeda, I. Fujishiro","doi":"10.1145/3478512.3488606","DOIUrl":"https://doi.org/10.1145/3478512.3488606","url":null,"abstract":"Flow effects such as flames, smoke, and liquids play an important role in activating illustrations, but drawing these effects requires artistic expertise as well as a great deal of effort. In this paper, we propose a method for adding stereoscopic flow effects to character illustrations using various shapes of 3D pottery wheel-type transparent canvases. One approach to designing a flow effect to decorate a character relies on simple curved geometry to beautify its flow in an organized composition. We extend this approach to present a drawing system—SpiCa (spinning canvas), which enables users to use transparent surface of revolution canvases to design 3D flow effects. User evaluations showed that users were able to create such effects more easily and effectively and reduce their workload with SpiCa in comparison with an existing 2D illustration tool.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117052306","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Inverse Free-form Deformation for interactive UV map editing","authors":"Seung-tak Noh, T. Igarashi","doi":"10.1145/3478512.3488614","DOIUrl":"https://doi.org/10.1145/3478512.3488614","url":null,"abstract":"Free-form deformation (FFD) is useful for manual 2D texture mapping in a 2D domain. The user first places a coarse regular grid in the texture space, and then adjusts the positions of the grid points in the image space. In this paper, we consider the inverse way of this problem, namely, inverse FFD. In this problem setting, we assume that an initial image-to-texture dense mapping is already obtained by some automatic method, such as a data-driven inference. However, this initial dense mapping may not be satisfactory, so the user may want to modify it. Nonetheless, it is difficult to manually edit the dense mapping due to its huge degrees of freedom. We thus convert the dense mapping to a coarse FFD mapping to facilitate manual editing of the mapping. Inverse FFD is formulated as a least-squares optimization, so one can solve it very efficiently.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130189708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Experimental Analysis of Multiple Scattering BRDF Models","authors":"Feng Xie, J. Bieron, P. Peers, P. Hanrahan","doi":"10.1145/3478512.3488601","DOIUrl":"https://doi.org/10.1145/3478512.3488601","url":null,"abstract":"","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114307023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lucio Moser, Jason Selfe, Darren Hendler, Douglas Roble
{"title":"Dynamic Neural Face Morphing for Visual Effects","authors":"Lucio Moser, Jason Selfe, Darren Hendler, Douglas Roble","doi":"10.1145/3478512.3488596","DOIUrl":"https://doi.org/10.1145/3478512.3488596","url":null,"abstract":"In this work we present a machine learning approach for face morphing in videos, between two or more identities. We devise an autoencoder architecture with distinct decoders for each identity, but with an underlying learnable linear basis for their weights. Each decoder has a learnable parameter that defines the interpolating weights (or ID weights) for the basis which can successfully decode its identity. During inference, the ID weights can be interpolated to produce a range of morphing identities. Our method produces temporally consistent results and allows blending different aspects of the identities by exposing the blending weights for each layer of the decoder network. We deploy our trained models to image compositors as 2D nodes with independent controls for the blending weights. Our approach has been successfully used in production, for the aging of David Beckham in the Malaria Must Die campaign.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133247299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Haowei Han, Meng Sun, Siyu Zhang, Dongying Liu, Tiantian Liu
{"title":"GPU Cloth Simulation Pipeline in Lightchaser Animation Studio","authors":"Haowei Han, Meng Sun, Siyu Zhang, Dongying Liu, Tiantian Liu","doi":"10.1145/3478512.3488616","DOIUrl":"https://doi.org/10.1145/3478512.3488616","url":null,"abstract":"We present the simulation pipeline of character effects in Lightchaser Animation Studio and how we utilize GPU resources to accelerate clothing simulations. Many characters from ancient Chinese tales in our films are in complex costumes.(Figure 1) Such costumes contain five to six layers of fabrics, where several hundred thousand triangles are used to show the delicate folds of different cloth materials. At the same time, there are more than 600 similar shots bringing more than 1000 cloth simulation tasks in a film project. Therefore, the efficiency and accuracy of our cloth simulator is the key to the CFX production pipeline.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130372242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comic Image Inpainting via Distance Transform","authors":"N. Ono, K. Aizawa, Yusuke Matsui","doi":"10.1145/3478512.3488607","DOIUrl":"https://doi.org/10.1145/3478512.3488607","url":null,"abstract":"Inpainting techniques for natural images have progressed significantly. However, if these methods are applied to comic images, the results are not satisfactory because of very noticeable artifacts, especially around line drawings. Line drawings are challenging to inpaint because of their high-frequency components. In this paper, we propose a novel method for inpainting comic images in the distance transform domain. In this method, we first convert a line drawing into a distance image and then inpaint the distance image. By transforming line drawings into distance images, we can eventually reduce the high-frequency components, which leads to improve inpainting performance. We compared the results of our proposed method with those of the conventional methods. The results showed that the proposed method achieved 0.1% lower l1 loss, 0.5dB higher PSNR, and 0.5% higher SSIM than those of the conventional inpainting methods.","PeriodicalId":156290,"journal":{"name":"SIGGRAPH Asia 2021 Technical Communications","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130192269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}