ELK Stack – Improving the Computing Clusters at DFCTI Through Log Analysis

The full stack logging service provided by Elastic™ has become a powerful tool within the highperformance computing community due to its ease of use, lightweight impact on the machines, performance speeds, and scalability. In the current work, we attempt to deploy such a stack on a server inside our department, which will be used for ingesting, parsing, and analyzing logs coming from multiple clusters. By analyzing the overall performance of each machine that is under continuous monitoring, we can provide immediate support in case of any issues that might occur, and more importantly, we can improve the computing power of our clusters through optimizations in terms of system management, networking, and other specific features.