ArchiDiff: Interactive design of 3D architectural forms generated from a single image

Abstract

3D Reconstruction Using Images has made strides in small-scale, uncomplicated scenes but struggles with complex, large-scale architectural forms. Targeting early-stage architectural design, we introduce ArchiDiff, a platform for 3D architectural form generation and editing from images to point clouds. First, we curated a dataset specifically tailored for architectural design, ArchiCloudNet. Second, we proposed a 3D generation method using a conditional denoising diffusion model, with an arbitrary object segmentation model to enhance recognition capabilities in complex input. Finally, we incorporate an interactive feature enabling instantaneous 2D image editing through simple drag-and-drops with simultaneous updates to 3D forms, giving designers improved control. We evaluated ArchiDiff’s generation accuracy against cutting-edge baselines on ArchiCloudNet and two other datasets, RealCity3D and BuildingNet. We also validated it with real sketches from early-stage architectural design. The experiments indicated that our model could generate accurate architectural point clouds, providing rapid-response modification and effective processing of complex backgrounds. Demostration: http://39.101.72.109:3000/archidiff.