Microservices for multimedia: video encoding

Netflix has been one of the pioneers that has driven the industry adoption of a new paradigm of system architecture referred to as "microservices". Microservices, or more accurately, microservice architecture refers to an architecture where applications are modeled as a collection of services which are: highly maintainable and independently testable, loosely coupled, independently deployable and organized around business capabilities. Typically, each microservice is owned by a small team of developers that is responsible for its development, testing and deployment, i.e., its end-to-end lifecycle. Traditional microservices such as those used outside of multimedia processing at Netflix typically consist of an API with stateless business logic which is autoscaled based on request load. These APIs provide strong contracts and separate the application data and binary dependencies from systems. As useful as traditional microservices are, several peculiarities of multimedia applications render such stateless services non ideal for media processing. Specifically, media processing (which includes video/audio processing, media encoding, timed-text processing, computer vision analysis etc.) relies on data that is embedded in files where the files themselves are contracts as opposed to fully visible data models that are common in non-media applications. At Netflix, media processing is resource intensive and bursty in nature. It is also highly parallelizable and re-triable, and so, even though work is generally a continuous stream with deadlines and priorities, the system can balance resources by evicting jobs as needed which can be retried at a later time. In this talk, we will summarize Cosmos, a project that we've developed in order to enable workflow-driven media processing using a microservice architecture. Cosmos is a computing platform that combines the best aspects of microservices with asynchronous workflows and serverless functions. It is designed specifically for resource intensive algorithms which are coordinated via complex hierarchical workflows. Cosmos supports both high throughput and low-latency workloads. The Cosmos platform offers: observability through built in logging, tracing, monitoring, alerting and error classification; modularity (both compile-time and run-time) through an opinionated framework for structuring a service; productivity through tooling such as code generators, containers, and command line interfaces; and delivery through a managed continuous-delivery pipelines. The Cosmos platform allows media developers to build and run domain-specific, scale-agnostic components which are built atop three scale-aware subsystems that handle distributing the work. Each component can thus be independently developed, tested and deployed with clear abstraction from the underlying platform thereby providing a logical separation between the application and platform so that the details of distributed computing are hidden from media developers Cosmos enables our media developers to take a service from commit to deployment in a matter of hours. To ensure the success of the large-scale overall system with independent fast-moving microservice development, innovative testing strategies are applied with various testing tools and quick rollback capability in production. In the talk, using the Netflix Video Encoder Service as an example, we will describe the Cosmos architecture and our migration to microservices-based media processing. The talk will also cover our learnings around managing a large-scale migration and the mindset required in order to plan and execute a multi-year goal.