Sorting of Scrambled Video Frames Using Temporal Order Verification

The Sorting Problem is exceedingly popular in our contemporary world for its contribution in many of our daily activities. In this paper, we tackle the problem of sorting through providing a method that sorts a variable number of scrambled video frames. The sorting algorithm is accompanied by two Machine Learning models. A supervised learning approach using video frames with semantic labels is presented through these models. The learning method was formulated as a sequential verification task, that is, determine whether a sequence of frames from a video is in the correct temporal order. Using two different classes of artificial neural networks, Three Dimensional Convolutional Neural Network (3D-CNN), and Recurrent Neural Network (RNN), temporally shuffled frames (frames in non- chronological order) are taken as inputs, then, the neural network is trained to first determine whether that sequence is sorted or not Afterwards, these models are used as validators in the sorting algorithm, to sort shuffled frames in the input sequence, to end up with a sorted frame sequence. The experimental results show good accuracy as different tests ended up with accuracy above 82%. This accuracy is related to the performance of an individual neural network, which means that it could be higher if both models are combined.