Few-shot learning with large foundation models for automated segmentation and accessibility analysis in architectural floor plans

Abstract

This paper presents a novel approach for extracting accessibility features from 2D raster floor plans by integrating few-shot learning techniques with the Segment Anything Model (SAM) and GPT-4. The proposed method addresses the limitations of existing deep learning-based floor plan analysis, which often require extensive annotated datasets and struggle with the variability of raster floor plans. Furthermore, there is a lack of research on extracting accessibility features from 2D raster floor plans, which remain one of the most common formats for storing architectural plans post-design and construction. Our approach, GPT-integrated Multi-object Few-shot SAM (GMFS), leverages similarity maps and cluster-based point sampling to generate accurate visual prompts for SAM, enabling the segmentation of rooms and doors using only five reference samples. The segmented masks are then classified using GPT-4, enhancing the semantic richness of the floor plan analysis. We validated GMFS using the CubiCasa and Rent3D datasets, demonstrating impressive performance in segmentation and classification. A detailed case study further showcased the practical application of our approach in calculating accessible means of egress and wheelchair clear space, which are critical features for accessibility compliance. The results highlight the effectiveness and adaptability of our approach in real-world scenarios, underscoring its potential to improve building accessibility and safety analysis in the architecture, engineering, and construction (AEC) industry.