High-performance technical computing with erlang

Abstract

High-performance Technical Computing (HPTC) is a branch of HPC (High-performance Computing) that deals with scientific applications, such as physics simulations. Due to its numerical nature, it has been traditionally based on low-level or mathematically-oriented languages (C, C++, Fortran), extended with libraries that implement remote execution and inter-process communication (like MPI and PVM). But those libraries just provide what Erlang does out-of-the-box: networking, process distribution, concurrency, interprocess communication and fault tolerance. So, is it possible to use Erlang as a foundation for developing HPTC applications? This paper shows our experiences in using Erlang for distributed number-crunching systems. We introduce two extensions: a simple and efficient foreign function interface (FFI), and an Erlang binding for numerical libraries. We use them as a basis for developing a simple mathematically-oriented programming language (in the style of Matlab™) compiled into Core Erlang. These tools are later used for creating a HPTC framework (based on message-passing) and an IDE for distributed applications. The results of this research and development show that Erlang/OTP can be used as a platform for developing large and scalable numerical applications.