Performance Comparison between S3, HDFS and RDS storage technologies for real-time big-data applications

Abstract

Low latency and fast delivery became crucial challenges in real-time applications of Big Data and remained a hot research topic. This research sheds light on the performance differences between three types of storage technologies, which are: the storage rational database system (MySQL), object storage (S3), which is powered by Amazon Cloud Services, and the Hadoop distributed file system (HDFS). Four system architectures with three different storage technologies were tested on AWS cloud environment with many complex queries to determine the most appropriate technology for real-time applications (i.e., web application, air traffic control systems, networked multimedia systems, command control systems) in terms of speed (computation time), network overload, and cost (price).