Nonrigid Registration of Hyperspectral and Color Images with Vastly Different Spatial and Spectral Resolutions for Spectral Unmixing and Pansharpening

In this paper, we propose a framework to register images with very large scale differences by utilizing the point spread function (PSF), and apply it to register hyperspectral and hi-resolution color images. The algorithm minimizes a least-squares (LSQ) objective function with an incorporated spectral response function (SRF), a nonrigid freeform deformation applied on the hyperspectral image and a rigid transformation on the color image. The optimization problem is solved by updating the two transformations and the two physical functions in an alternating fashion. We executed the framework on a simulated Pavia University dataset and a real Salton Sea dataset, by comparing the proposed algorithm with its rigid variation, and two mutual information-based algorithms. The results indicate that the LSQ freeform version has the best performance for the nonrigid simulation and real datasets, with less than 0.15 pixel error given 1 pixel nonrigid distortion in the hyperspectral domain.