Spark-based log data analysis for reconstruction of cybercrime events in cloud environment

In recent times, the number of cybercrimes against cloud systems and services is rapidly growing. Although, there are numerous protection systems such as firewalls and intrusion detection and prevention system, and anti-viruses that are developed to protect cloud infrastructures and services from severe attacks, but still the risk of criminal activities exists. This lead to attract the attention of researchers and scientists around the world to digital forensic which is a science to aid law enforcement officers and digital investigator to identify, collect and analyze digital footprints or evidence which are collected from a crime scene. One of the significant sources of as a digital evidence in the cloud is log data because they frequently connect events in certain time. The process of log data forensics mitigates the investigation process by identifying the malicious behavior and reveal the hidden malicious activities. Cloud log analysis can help to reconstruct cybercrime events which occurred in the cloud. Traditional log data analysis procedures and tools can be adapted to cloud through using new fast on memory computing platforms such as Apache Spark. Spark is a general-purpose cluster-computing engine, which is very fast and reliable. This paper presents analysis approach for batch and stream log data using Apache Spark. The results show that Spark can be used as a fast platform for handling the diverse large size of log data and extract useful information that can assist digital investigators in the analysis immense amount of generated cloud log data in a given frame of time. Furthermore, the results can make provision to reconstruct and generate a timeline related to historical past sequence events occurred during a cloud crime as well as identify the malicious user's IP address, date and time, with a number of accesses.