Human-centric approaches to image understanding and retrieval

Abstract

The amount of digital medical image data is increasing rapidly in terms of both quantity and heterogeneity. There exists a great need to format medical image archives so as to facilitate diagnostics and preventive medicine. To achieve this, in the past few decades great efforts have been made to investigate methods of applying content-based image retrieval (CBIR) techniques to retrieve images. However, several critical challenges remain. Recently, CBIR research has become intertwined with the fundamental problem of image understanding and it is recognized that computing solutions that bridge the “semantic gap” must capture higher-level domain knowledge of medical end users. We are investigating the incorporation of state-of-the-art visual categorization techniques into conventional CBIR approaches. Visual attention deployment strategies of medical experts serve as an objective measure to help us understand the perceptual and conceptual processes involved in identifying key visual features and selecting diagnostic regions of the images. Understanding these processes will inform and direct feature selection approaches on medical images, such as the dermatological images used in our study. We also explore systematic and effective information integration methods of image data and semantic descriptions with the long-term goals of building efficient human-centered multi-modal interactive CBIR systems.