Stone masonry design automation via reinforcement learning

Abstract The use of local natural and recycled feedstock is promising for sustainable construction. However, unlike versatile engineered bricks, natural and recycled feedstock involves design challenges due to their stochastic, sequential, and heterogeneous nature. For example, the practical use of stone masonry is limited, as it still relies on human experts with holistic domain knowledge to determine the sequential organization of natural stones with different sizes/shapes. Reinforcement learning (RL) is expected to address such design challenges, as it allows artificial intelligence (AI) agents to autonomously learn design policy, that is, identifying the best design decision at each time step. As a proof-of-concept RL framework for design automation involving heterogeneous feedstock, a stone masonry design framework is presented. The proposed framework is founded upon a virtual design environment, MasonTris, inspired by the analogy between stone masonry and Tetris. MasonTris provides a Tetris-like virtual environment combined with a finite element analysis (FEA), where AI agents learn effective design policies without human intervention. Also, a new data collection policy, almost-greedy policy, is designed to address the sparsity of feasible designs for faster/stable learning. As computation bottleneck occurs when parallel agents evaluate designs with different complexities, a modification of the RL framework is proposed that FEA is held until training data are retrieved for training. The feasibility and adaptability of the proposed framework are demonstrated by continuously improving stone masonry design policy in simplified design problems. The framework can be generalizable to different natural and recycled feedstock by incorporating more realistic assumptions, opening opportunities in design automation for sustainability.