Design Form and Function Prediction From a Single Image

Estimating the form and functional performance of a design in the early stages can be crucial for a designer for effective ideation Humans have an innate ability to guess the size, shape, and type of a design from a single view. The brain fills in the unknowns in a fraction of a second. However, humans may struggle with estimating the performance of designs in the early stages of the design process without making prototypes or doing back-of-the-envelope calculations. In contrast, machines need information about the full 3D model of a design to understand its structure. Machines can estimate the performance using pre-defined rules, expensive numerical simulations, or machine learning models. In this paper, we show how information about the form and functional performance of a design can be estimated from a single image using machine learning methods. Specifically, we leverage the image-to-image translation method to predict multiple projections of an image-based design. We then train deep neural network models on the predicted projections to provide estimates of design performance. We demonstrate the effectiveness of our method by predicting the aerodynamic performance from images of aircraft models. To estimate ground truth aero-dynamic performance, we run CFD simulations for 4045 3D aircraft models from the ShapeNet dataset and use their lift-to-drag ratio as the performance metric. Our results show that single images do carry information for both form and functional performance. From a single image, we are able to produce six additional images of a design in different orientations, with an average Structural Similarity Index score of 0.872. We also find image-translation methods provide a promising direction in estimating the performance of design. Using multiple images of a design (gathered through image-translation) to predict design performance yields a recall value of 47%, which is 14% higher than a base guess, and 3% higher than using a single image. Our work identifies the potential and provides a framework for using a single image to predict the form and functional performance of a design during the early-stage design process. Our code and additional information about our work are available at http://decode.mit.edu/projects/formfunction/.