Content based Lecture Video Retrieval using Textual Queries: to be Smart University

Abstract

The amount of lecture videos is rapidly growing due to the popularity of massive online open courses in academic institutions. Thus, the efficient method for lecture video retrieval in various languages is needed. In this paper, we propose an approach for automated lecture video indexing and retrieval. First, the lecture video is segmented into keyframes in a manner that the duplication of these frames is minimal. The textual information embedded in each keyframe is then extracted. We consider this issue as a matter of text detection and recognition. The text detection is solved by our segmentation network in which we propose a binarization approach for optimizing text locations in an image. For text recognition, we take advantage of VietOCR to recognize both English and Vietnamese text. Lastly, we integrate a vector-based semantic search in ElasticSearch to enhance the ability of lecture video search. The experimental results show that our approach gives high performance in detecting and recognizing the text content in both English and Vietnamese as well as enhancing the speed and accuracy of lecture video retrieval.