Yijie Tang, Jiazhao Zhang, Zhimiao Yu, He Wang, Kaiyang Xu
{"title":"MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction","authors":"Yijie Tang, Jiazhao Zhang, Zhimiao Yu, He Wang, Kaiyang Xu","doi":"10.1145/3618363","DOIUrl":"https://doi.org/10.1145/3618363","url":null,"abstract":"We introduce MIPS-Fusion, a robust and scalable online RGB-D reconstruction method based on a novel neural implicit representation - multi-implicit-submap. Different from existing neural RGB-D reconstruction methods lacking either flexibility with a single neural map or scalability due to extra storage of feature grids, we propose a pure neural representation tackling both difficulties with a divide-and-conquer design. In our method, neural submaps are incrementally allocated alongside the scanning trajectory and efficiently learned with local neural bundle adjustments. The submaps can be refined individually in a back-end optimization and optimized jointly to realize submap-level loop closure. Meanwhile, we propose a hybrid tracking approach combining randomized and gradient-based pose optimizations. For the first time, randomized optimization is made possible in neural tracking with several key designs to the learning process, enabling efficient and robust tracking even under fast camera motions. The extensive evaluation demonstrates that our method attains higher reconstruction quality than the state of the arts for large-scale scenes and under fast camera motions.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"36 1","pages":"1 - 16"},"PeriodicalIF":0.0,"publicationDate":"2023-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139350269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Neural Categorical Priors for Physics-Based Character Control","authors":"Qing Zhu, He Zhang, Mengting Lan, Lei Han","doi":"10.1145/3618397","DOIUrl":"https://doi.org/10.1145/3618397","url":null,"abstract":"Recent advances in learning reusable motion priors have demonstrated their effectiveness in generating naturalistic behaviors. In this paper, we propose a new learning framework in this paradigm for controlling physics-based characters with improved motion quality and diversity over existing methods. The proposed method uses reinforcement learning (RL) to initially track and imitate life-like movements from unstructured motion clips using the discrete information bottleneck, as adopted in the Vector Quantized Variational AutoEncoder (VQ-VAE). This structure compresses the most relevant information from the motion clips into a compact yet informative latent space, i.e., a discrete space over vector quantized codes. By sampling codes in the space from a trained categorical prior distribution, high-quality life-like behaviors can be generated, similar to the usage of VQ-VAE in computer vision. Although this prior distribution can be trained with the supervision of the encoder's output, it follows the original motion clip distribution in the dataset and could lead to imbalanced behaviors in our setting. To address the issue, we further propose a technique named prior shifting to adjust the prior distribution using curiosity-driven RL. The outcome distribution is demonstrated to offer sufficient behavioral diversity and significantly facilitates upper-level policy learning for downstream tasks. We conduct comprehensive experiments using humanoid characters on two challenging downstream tasks, sword-shield striking and two-player boxing game. Our results demonstrate that the proposed framework is capable of controlling the character to perform considerably high-quality movements in terms of behavioral strategies, diversity, and realism. Videos, codes, and data are available at https://tencent-roboticsx.github.io/NCP/.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"14 1","pages":"1 - 16"},"PeriodicalIF":0.0,"publicationDate":"2023-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139350760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Praneeth Chakravarthula, Ji-ping Sun, Xiao Li, Chenyang Lei, Gene Chou, Mario Bijelic, Johannes Froesch, A. Majumdar, Felix Heide
{"title":"Thin On-Sensor Nanophotonic Array Cameras","authors":"Praneeth Chakravarthula, Ji-ping Sun, Xiao Li, Chenyang Lei, Gene Chou, Mario Bijelic, Johannes Froesch, A. Majumdar, Felix Heide","doi":"10.1145/3618398","DOIUrl":"https://doi.org/10.1145/3618398","url":null,"abstract":"Today's commodity camera systems rely on compound optics to map light originating from the scene to positions on the sensor where it gets recorded as an image. To record images without optical aberrations, i.e., deviations from Gauss' linear model of optics, typical lens systems introduce increasingly complex stacks of optical elements which are responsible for the height of existing commodity cameras. In this work, we investigate flat nanophotonic computational cameras as an alternative that employs an array of skewed lenslets and a learned reconstruction approach. The optical array is embedded on a metasurface that, at 700 nm height, is flat and sits on the sensor cover glass at 2.5 mm focal distance from the sensor. To tackle the highly chromatic response of a metasurface and design the array over the entire sensor, we propose a differentiable optimization method that continuously samples over the visible spectrum and factorizes the optical modulation for different incident fields into individual lenses. We reconstruct a megapixel image from our flat imager with a learned probabilistic reconstruction method that employs a generative diffusion model to sample an implicit prior. To tackle scene-dependent aberrations in broadband, we propose a method for acquiring paired captured training data in varying illumination conditions. We assess the proposed flat camera design in simulation and with an experimental prototype, validating that the method is capable of recovering images from diverse scenes in broadband with a single nanophotonic layer.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"63 1","pages":"1 - 18"},"PeriodicalIF":0.0,"publicationDate":"2023-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139351655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jonathan Panuelos, Ryan Goldade, E. Grinspun, D. Levin, Christopher Batty
{"title":"PolyStokes: A Polynomial Model Reduction Method for Viscous Fluid Simulation","authors":"Jonathan Panuelos, Ryan Goldade, E. Grinspun, D. Levin, Christopher Batty","doi":"10.1145/3592146","DOIUrl":"https://doi.org/10.1145/3592146","url":null,"abstract":"Standard liquid simulators apply operator splitting to independently solve for pressure and viscous stresses, a decoupling that induces incorrect free surface boundary conditions. Such methods are unable to simulate fluid phenomena reliant on the balance of pressure and viscous stresses, such as the liquid rope coil instability exhibited by honey. By contrast, unsteady Stokes solvers retain coupling between pressure and viscosity, thus resolving these phenomena, albeit using a much larger and thus more computationally expensive linear system compared to the decoupled approach. To accelerate solving the unsteady Stokes problem, we propose a reduced fluid model wherein interior regions are represented with incompressible polynomial vector fields. Sets of standard grid cells are consolidated into super-cells, each of which are modelled using a quadratic field of 26 degrees of freedom. We demonstrate that the reduced field must necessarily be at least quadratic, with the affine model being unable to correctly capture viscous forces. We reproduce the liquid rope coiling instability, as well as other simulated examples, to show that our reduced model is able to reproduce the same fluid phenomena at a smaller computational cost. Futhermore, we performed a crowdsourced user survey to verify that our method produces imperceptible differences compared to the full unsteady Stokes method.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"31 1","pages":"1 - 13"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81514765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daoming Liu, Davide Pellis, Yu-Chou Chiang, F. Rist, J. Wallner, H. Pottmann
{"title":"Deployable strip structures","authors":"Daoming Liu, Davide Pellis, Yu-Chou Chiang, F. Rist, J. Wallner, H. Pottmann","doi":"10.1145/3592393","DOIUrl":"https://doi.org/10.1145/3592393","url":null,"abstract":"We introduce the new concept of C-mesh to capture kinetic structures that can be deployed from a collapsed state. Quadrilateral C-meshes enjoy rich geometry and surprising relations with differential geometry: A structure that collapses onto a flat and straight strip corresponds to a Chebyshev net of curves on a surface of constant Gaussian curvature, while structures collapsing onto a circular strip follow surfaces which enjoy the linear-Weingarten property. Interestingly, allowing more general collapses actually leads to a smaller class of shapes. Hexagonal C-meshes have more degrees of freedom, but a local analysis suggests that there is no such direct relation to smooth surfaces. Besides theory, this paper provides tools for exploring the shape space of C-meshes and for their design. We also present an application for freeform architectural skins, namely paneling with spherical panels of constant radius, which is an important fabrication-related constraint.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"44 1","pages":"1 - 16"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84936949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Eric Tabellion, Nikhil Karnad, Noa Glaser, Ben Weiss, David E. Jacobs, Y. Pritch
{"title":"Computational Long Exposure Mobile Photography","authors":"Eric Tabellion, Nikhil Karnad, Noa Glaser, Ben Weiss, David E. Jacobs, Y. Pritch","doi":"10.1145/3592124","DOIUrl":"https://doi.org/10.1145/3592124","url":null,"abstract":"Long exposure photography produces stunning imagery, representing moving elements in a scene with motion-blur. It is generally employed in two modalities, producing either a foreground or a background blur effect. Foreground blur images are traditionally captured on a tripod-mounted camera and portray blurred moving foreground elements, such as silky water or light trails, over a perfectly sharp background landscape. Background blur images, also called panning photography, are captured while the camera is tracking a moving subject, to produce an image of a sharp subject over a background blurred by relative motion. Both techniques are notoriously challenging and require additional equipment and advanced skills. In this paper, we describe a computational burst photography system that operates in a hand-held smartphone camera app, and achieves these effects fully automatically, at the tap of the shutter button. Our approach first detects and segments the salient subject. We track the scene motion over multiple frames and align the images in order to preserve desired sharpness and to produce aesthetically pleasing motion streaks. We capture an under-exposed burst and select the subset of input frames that will produce blur trails of controlled length, regardless of scene or camera motion velocity. We predict inter-frame motion and synthesize motion-blur to fill the temporal gaps between the input frames. Finally, we composite the blurred image with the sharp regular exposure to protect the sharpness of faces or areas of the scene that are barely moving, and produce a final high resolution and high dynamic range (HDR) photograph. Our system democratizes a capability previously reserved to professionals, and makes this creative style accessible to most casual photographers.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"6 1","pages":"1 - 15"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82038566","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Oliver Gross, Yousuf Soliman, Marcel Padilla, Felix Knöppel, U. Pinkall, P. Schröder
{"title":"Motion from Shape Change","authors":"Oliver Gross, Yousuf Soliman, Marcel Padilla, Felix Knöppel, U. Pinkall, P. Schröder","doi":"10.1145/3592417","DOIUrl":"https://doi.org/10.1145/3592417","url":null,"abstract":"We consider motion effected by shape change. Such motions are ubiquitous in nature and the human made environment, ranging from single cells to platform divers and jellyfish. The shapes may be immersed in various media ranging from the very viscous to air and nearly inviscid fluids. In the absence of external forces these settings are characterized by constant momentum. We exploit this in an algorithm which takes a sequence of changing shapes, say, as modeled by an animator, as input and produces corresponding motion in world coordinates. Our method is based on the geometry of shape change and an appropriate variational principle. The corresponding Euler-Lagrange equations are first order ODEs in the unknown rotations and translations and the resulting time stepping algorithm applies to all these settings without modification as we demonstrate with a broad set of examples.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"18 1","pages":"1 - 11"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80794666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Scratch-based Reflection Art via Differentiable Rendering","authors":"Pengfei Shen, Ruizeng Li, Beibei Wang, Ligang Liu","doi":"10.1145/3592142","DOIUrl":"https://doi.org/10.1145/3592142","url":null,"abstract":"The 3D visual optical arts create fascinating special effects by carefully designing interactions between objects and light sources. One of the essential types is 3D reflection art, which aims to create reflectors that can display different images when viewed from different directions. Existing works produce impressive visual effects. Unfortunately, previous works discretize the reflector surface with regular grids/facets, leading to a large parameter space and a high optimization time cost. In this paper, we introduce a new type of 3D reflection art - scratch-based reflection art, which allows for a more compact parameter space, easier fabrication, and computationally efficient optimization. To design a 3D reflection art with scratches, we formulate it as a multi-view optimization problem and introduce differentiable rendering to enable efficient gradient-based optimizers. For that, we propose an analytical scratch rendering approach, together with a high-performance rendering pipeline, allowing efficient differentiable rendering. As a consequence, we could display multiple images on a single metallic board with only several minutes for optimization. We demonstrate our work by showing virtual objects and manufacturing our designed reflectors with a carving machine.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"82 1","pages":"1 - 12"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85535338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Example-Based Procedural Modeling Using Graph Grammars","authors":"Paul C. Merrell","doi":"10.1145/3592119","DOIUrl":"https://doi.org/10.1145/3592119","url":null,"abstract":"We present a method for automatically generating polygonal shapes from an example using a graph grammar. Most procedural modeling techniques use grammars with manually created rules, but our method can create them automatically from an example. Our graph grammars generate graphs that are locally similar to a given example. We disassemble the input into small pieces called primitives and then reassemble the primitives into new graphs. We organize all possible locally similar graphs into a hierarchy and find matching graphs within the hierarchy. These matches are used to create a graph grammar that can construct every locally similar graph. Our method generates graphs using the grammar and then converts them into a planar graph drawing to produce the final shape.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"49 1","pages":"1 - 16"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81359843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"P2M: A Fast Solver for Querying Distance from Point to Mesh Surface","authors":"Chen Zong, Jiacheng Xu, Jiantao Song, Shuangmin Chen, Shiqing Xin, Wenping Wang, Changhe Tu","doi":"10.1145/3592439","DOIUrl":"https://doi.org/10.1145/3592439","url":null,"abstract":"Most of the existing point-to-mesh distance query solvers, such as Proximity Query Package (PQP), Embree and Fast Closest Point Query (FCPW), are based on bounding volume hierarchy (BVH). The hierarchical organizational structure enables one to eliminate the vast majority of triangles that do not help find the closest point. In this paper, we develop a totally different algorithmic paradigm, named P2M, to speed up point-to-mesh distance queries. Our original intention is to precompute a KD tree (KDT) of mesh vertices to approximately encode the geometry of a mesh surface containing vertices, edges and faces. However, it is very likely that the closest primitive to the query point is an edge e (resp., a face f), but the KDT reports a mesh vertex υ instead. We call υ an interceptor of e (resp., f). The main contribution of this paper is to invent a simple yet effective interception inspection rule and an efficient flooding interception inspection algorithm for quickly finding out all the interception pairs. Once the KDT and the interception table are precomputed, the query stage proceeds by first searching the KDT and then looking up the interception table to retrieve the closest geometric primitive. Statistics show that our query algorithm runs many times faster than the state-of-the-art solvers.","PeriodicalId":7077,"journal":{"name":"ACM Transactions on Graphics (TOG)","volume":"878 ","pages":"1 - 13"},"PeriodicalIF":0.0,"publicationDate":"2023-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91520776","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
