Exploiting communication latency hiding for parallel network computing: model and analysis

Very large problems with high resource requirements of both computation and communication could be tackled with large numbers of workstations. However for LAN-based networks, contention becomes a limiting factor whereas latency appears to limit communication for WAN-based networks, nominally the Internet. We describe a model to analyze the gain of communication latency hiding by overlapping computation and communication. This model illustrates the limitations and opportunities of communication latency hiding for improving speedup of parallel computations that can be structured appropriately. Experiments show that latency hiding techniques increase the feasibility of parallel computing in high-latency networks of workstations across the Internet as well as in multiprocessor systems.