Assessing the performance of an automated video ground truthing application

Present methods of quantifying the performance of ATR algorithms involves the use of large video datasets that must be truthed by hand, frame-by-frame, requiring vast amounts of time. To reduce this cost, we have developed an application that significantly reduces the cost by only requiring the operator to grade a relatively sparse number of data "keyframes". A correlation-based template matching algorithm computes the best position, orientation and scale when interpolating between keyframes. We demonstrate the performance of the automated truthing application, and compare the results to those of a series of human operator test subjects. The START-generated truth is shown to be very close to the mean truth data given by the human operators. Additionally the savings in labor producing the results is also demonstrated.