Non-invasive assessment of the visual environment using conditional generative adversarial networks

Luminance monitoring within the field of view (FOV) is required for assessing visual comfort and overall visual preferences, but it is practically challenging and intrusive. As a result, real-time, human-centered daylighting operation remains a challenge. This paper presents a novel deep-learning based framework method to demonstrate that meaningful features in the occupant’s visual field can be extracted without invasive measurements. It is the first proof of concept to show that it is feasible to monitor luminance distributions as perceived by people, using a non-intrusive camera integrated with deep learning neural networks. A Conditional Generative Adversarial Network (CGAN), pix2pix is used to transfer information from non-intrusive images to FOV images. Two datasets were collected in an open-plan office with compact, low-cost High Dynamic Range Image (HDRI) cameras installed at two alternate locations (a wall or a monitor), to separately train two pix2pix models with the same target FOV images. The results show that the generated FOV images closely resemble the measured FOV images in terms of pixelwise luminance errors, mean luminance, and structural similarity. The main errors are due to bright scenes, visible through windows, confined to a very limited number of pixels. Overall, this work establishes a basis for future studies to assess the effect of visual environment on human perception using non-intrusive measurements. It also provides the theoretical foundation for a connected paper [27], which demonstrates that non-intrusive measurements and deep learning techniques can be used to discover daylight preferences and enable AI-assisted daylighting operation.