NEOS: Non-intrusive Edge Observability stack based on Zero Trust security model for Ubiquitous Computing

Abstract

The Edge computing paradigm has emerged as the new industrial norm for creating distributed applications. These distributed applications need to target high reliability and scalability to meet the goals and requirements of the users. Achieving this definitely requires a real time observability stack to closely observe, track, debug and improve the application. In this paper we introduce the Non-Intrusive Edge Observability Stack(NEOS) that simplifies the process of collecting, analyzing, and visualizing telemetry data. It reduces the amount of code instrumentation needed to collect telemetry data up to 80% and offers extensive configuration capabilities within the subcomponents of the process. It offers a set of user-friendly abstractions and easy-to-use APIs, which minimizes the effort needed for manual instrumentation of application code. NEOS leverages popular open-source tools such as OpenTelemetry, Grafana, Prometheus, Jaeger, and Loki, for the collecting and visualizing of telemetry data. Furthermore, NEOS implements security based on the zero-trust model, which means that we assume that no user or system can be trusted by default. The security of every connection establised in NEOS employs mutual Transport Layer Security (mTLS) to prevent unauthorized access and safeguard sensitive data. Experiments were conducted to assess the efficiency of the stack by comparing the time and effort needed to instrument code with and without the stack. The outcomes showed a considerable reduction in instrumentation code. NEOS can be used by product managers, engineering and operation team for system and application health monitoring, real-time business insights, and debugging system.