Hyperspectral Image Reconstruction of Heritage Artwork Using RGB Images and Deep Neural Networks

Abstract

The application of our research is in the art world where the scarcity of available analytical data from a particular artist or physical access for its acquisition is restricted. This poses a fundamental problem for the purpose of conservation, restoration or authentication of historical artworks. We address part of this problem by providing a practical method to generate hyperspectral data from readily available RGB imagery of artwork by means of a two-step process using deep neural networks. The particularities of our approach include the generation of learnable colour mixtures and reflectances from a reduced collection of prior data for the mapping and reconstruction of hyperspectral features on new images. Further analysis and correction of the prediction are achieved by a second network that reduces the error by producing results akin to those obtained by a hyperspectral camera. Our method has been used to study a collection of paintings by Amadeo de Souza-Cardoso where successful results were obtained.