Accessmath: Indexing and retrieving video segments containing math expressions based on visual similarity

Abstract

Access Math project is a work in progress oriented toward helping visually impaired students in and out of the class-room. The system works with videos from math lectures. For each lecture, videos of the whiteboard content from two different sources are provided. An application for extraction and retrieval of that content is presented. After the content has been indexed, the user can select a portion of the whiteboard content found in a video frame and use it as a query to find segments of video with similar content. Graphs of neighboring connected components are used to describe both the query and the candidate regions, and the results of a query are ranked using the recall of matched graph edges between the graph of the query and the graph of each candidate. This is a recognition-free method and belongs to the field of sketch-based image retrieval.