Using network traffic to passively detect under utilized resources in high performance cluster grid computing environments

In this paper we propose a technique for detecting under utilized resources (less than 70% memory utilization) due to memory bound processes by passively monitoring network traffic produced by the resource. To our knowledge, this is the first approach of its kind. One application of this technique is dynamic resource discovery (detection of resources with under utilized memory) in a High Performance Desktop or Cluster Grid computing environment confined to a low latency Local Area Network (LAN). Our method removes the need to communicate directly with resources to determine if their memory is under utilized, thus reducing traffic on the network. This is very important in a High Performance computing environment since data or computational intensive applications may be present. The proposed method creates a delay sensitive profile generated by the analysis of monitored network traffic due to High Performance UDP based services such as file transfer applications (FOBS, Tsunami, UDT, SABUL, etc.), message passing platforms (MPICH-G2/Score, etc.), and many more. An energy value is derived from the delay sensitive profile, which represents the state (over utilized memory or under utilized memory) of the resource of interest. Then a simple threshold is applied to the energy value to identify the state of the resource. Several scenarios have been investigated to determine the feasibility of the proposed technique. Results suggest that the proposed technique can use network traffic to extract delays associated with a resources' memory utilization and accurately determine the state of the resource.