Real-Time Knit Deformation and Rendering

Abstract

The knit structure consists of interlocked yarns, with each yarn comprising multiple plies comprising tens to hundreds of twisted fibers. This intricate geometry and the large number of geometric primitives present substantial challenges for achieving high-fidelity simulation and rendering in real-time applications. In this work, we introduce the first real-time framework that takes an animated stitch mesh as input and enhances it with yarn-level simulation and fiber-level rendering. Our approach relies on a knot-based representation to model interlocked yarn contacts. The knot positions are interpolated from the underlying mesh, and associated yarn control points are optimized using a physically inspired energy formulation, which is solved through a GPU-based Gauss-Newton scheme for real-time performance. The optimized control points are sent to the GPU rasterization pipeline and rendered as yarns with fiber-level details. In real-time rendering, we introduce several decomposition strategies to enable realistic lighting effects on complex knit structures, even under environmental lighting, while maintaining computational and memory efficiency. Our simulation faithfully reproduces yarn-level structures under deformations, e.g., stretching and shearing, capturing interlocked yarn behaviors. The rendering pipeline achieves near-ground-truth visual quality while being 120,000× faster than path tracing reference with fiber-level geometries. The whole system provides real-time performance and has been evaluated through various application scenarios, including knit simulation for small patches and full garments and yarn-level relaxation in the design pipeline.