A Tale of Two C’s: Convergence and Composability

Abstract

Cyberinfrastructure is everywhere in diverse forms in service of applications in science, business and society. From IoT to extreme-scale computing data and computing have never been so distributed with the potential for real-time integration into these applications. The common theme to these applications, mostly composed of (big) data-integrated workloads, is their need to run in specialized environments for reasons such as on-demand or 24x7 nature of the tasks they are performing, and difficulties regarding their portability, latency, privacy, and performance optimization. Moreover, in many data-driven scientific applications, there is a need for heterogeneous integration of tasks requiring specialized computing capabilities with traditional high-throughput computing or high-performance computing tasks. Although some key middleware technologies enabled demonstration of standalone heterogeneous applications, such integration requires expertise convergence from a large group of people in very specialized settings. There are still many challenges for streamlined, scalable, repeatable, responsible, and explainable integration of data-integrated applications. Key opportunities for further innovations include intelligent systems and automated workflow management software that can compose and steer dynamic applications that can adapt to changing conditions in a data-driven fashion while integrating many tools to explore, analyze and utilize data. This talk will discuss some examples for data-integrated applications, describe emerging systems that enabled these applications, and overview our recent research to enable composable applications including a convergence application development methodology, intelligent middleware, and workflow composition.