CLIP Augmentation for Image Search

Abstract

: We devised a probabilistic method for adding face recognition to the neural network model CLIP. The method was tested by creating a prototype and matching 1000 images to their descriptions. The method improved the text to image Recall @ 1 metric from 14.0% matches for CLIP alone to 21.8% for CLIP + method, for a sample size of 1000 images and descriptions.