Video key concept extraction using Convolution Neural Network

Abstract

Objectives: This work aims to develop an automated video summarising methodology and timestamping that uses natural language processing (NLP) tools to extract significant video information.Methods: The methodology comprises extracting the audio from the video, splitting it into chunks by the size of the pauses, and transcribing the audio using Google's speech recognition. The transcribed text is tokenised to create a summary, sentence and word frequencies are calculated, and the most relevant sentences are selected. The summary quality is assessed using ROUGE criteria, and the most important keywords are extracted from the transcript using RAKE.Findings: Our proposed method successfully extracts key points from video lectures and creates text summaries. Timestamping these key points provides valuable context and facilitates navigation within the lecture. Our method combines video-to-text conversion and text summarisation with timestamping key concepts, offering a novel approach to video lecture analysis. Existing video analysis methods focus on keyword extraction or summarisation, while our method offers a more comprehensive approach. Our timestamped key points provide a unique feature compared to other methods. Our method enhances existing video reports by (i) providing concise summaries of key concepts and (ii) enabling quick access to specific information through timestamps. (iii) Facilitating information retrieval through a searchable index. Further research directions: (i) Improve the accuracy of the multi-stage processing pipeline. (ii) Develop techniques to handle diverse accents and pronunciations. (iii) Explore applications of the proposed method to other video genres and types.Application/Improvements: This approach is practical in giving accurate video summaries, saving viewers time and effort when comprehending the main concepts presented in a video.