Semi-supervised image classification for automatic construction of a health image library

Abstract

Images represent an important aspect of the entire medical knowledge. Health illustrations and photos are essential for patient education and self-care. Currently there is few effort in automatically building comprehensive image bases for general health information consumers. We present an approach to automatically construct a consumer-oriented image library containing images of human organs, diseases, drugs and other medical entities. Such an image library can then be annotated with standard medical ontologies such as the FMA [26] and RxNorm [20], creating a rich information resource for illustration and educational purposes. As a first step in constructing such an image library, we present a semi-supervised bootstrapping classification method to retrieve relevant images from the web for given medical terms. Our method uses one positive image (for each term) as a seed and iteratively searches for images that are visually similar to the seed. We have conducted experiments in classifying web images for seven organs, four diseases and four drug tablets. We achieved average precisions of 62.4% and 66.8% for organ and drug images, which are comparable to those of a supervised classification method based on standard support vector machine (SVM) (precisions of 59.5% and 56.7%), while requiring minimum human intervention.