Evolution of a Stream Transformation DSL

Abstract

In this paper, we present our experiences during the evolution of a stream transformation Domain Specific Language (DSL). We initially implemented and evolved the DSL in the scope of our work on stream-based computer network monitoring. However, the application domain is not limited to computer network monitoring. While our initial DSL focused on transforming raw byte-array stream data into Java data structures such as maps, we added support for more input and output formats over time. The initial motivation for the DSL was to improve maintainability, extendibility, and performance. During the development, we were confronted with changing and evolving use-cases and requirements, which impacted the evolution of the DSL. We, e.g., added dynamic capabilities, which can be used, e. g., for self-adaptive mechanisms. In our opinion, the separation of concerns introduced by a DSL-based approach helped to improve our development efficiency. We implemented our DSL using the Clojure programming language, which, in our opinion, offers excellent features for implementing a dynamic DSL. We published our implementations as Open Source Software.