Boosting cross-modality image registration

Cross-modality image registration is a difficult problem because the same structures have different intensity patterns in the two modalities, making straightforward methods based on SSD or cross-correlation not applicable. This paper presents a learning based approach to cross-modality image registration. First, it describes a method to map the image registration problem into a problem of binary classification. Then, it presents a method to select a number of image registration algorithms from a larger pool and combine them by AdaBoost into a boosted algorithm that is more accurate than any of the algorithms in the pool. Finally, it presents a method named virtual boosting that allows to directly obtain the result of the boosted algorithm without performing any parameter search. In our cross-modality image registration application, the algorithm pool consists of many feature-based registration algorithms with different configurations. An experimental validation on the registration of thousands of aerial video frames with satellite images from Google Maps showed that the boosted algorithm has a 20–30% smaller error than the best registration algorithm from the pool (based on SIFT features). More generally, the method presented can be applied to combine a number of algorithms aimed at solving the same problem into a boosted algorithm that is more accurate than any of them.