Analyzing CNN Model Performance Sensitivity to the Ordering of Non-Natural Data

Convolutional Neural Networks (CNN) have had significant success in identifying and classifying image datasets. CNNs have also been used effectively in classifying non-visual datasets such as malware and gene expression. In all of these applications, CNNs require data to be organized in a certain order. In the case of images, this order is naturally presented. However, in the case of non-visual data, this order is sometimes not naturally defined and hence requires an artificially defined order. The sensitivity of a CNN model’s performance to various artificial orders of non-natural datasets is not well-understood. In this paper, we investigate this problem by experimenting with various orders of a dataset derived from malware behavior in a cloud auto-scaling environment. We show that the ordering can have a major impact on the performance of the CNN and offer some insights on how to derive one or more orderings that could provide better performance.