Image fusion for human observers: How should we choose the method?

Abstract

Image fusion is used to improve target detection and identification. In human-observer applications it is useful to rank fusion methods according to how well they assist the observer in a decision task. Two images (medium- and long-wave infrared), acquired for each of a number of outdoor scenes, were fused by each of nine methods. For each scene, a set of observers assessed each of the 36 pairwise combinations of fused images, choosing from each pair the one that was deemed best for target identification. We used that set of preferences to rank the fusion methods for their effectiveness in the identification task. A classical technique for ranking these “discriminal processes” is Thurstone's Law of Comparative Judgment and its implementation as the Thurstone-Mosteller (TM) Method of Paired Comparisons, which is reviewed briefly here. To make meaningful statements about preferences, one should have a measure of uncertainty for each rank. The TM method, however, cannot readily provide such a measure. An alternative, the Bradley-Terry (BT) method, does permit calculation of confidence intervals for ranks. To our knowledge, BT has not previously been applied in the evaluation of fusion methods. We present results from a multi-observer, multi-view trial, evaluated using TM and BT. The methods yield similar rankings of the fusion methods. But the additional information provided by BT - that is, whether there are significant differences between the ranks - can have a substantial impact on the implementation of fusion in real systems. There could be meaningful tradeoffs among fusion methods - e.g., performance vs. computation time - that may not be exploited in the absence of those insights.