Development of a Scalable and Distributed Streaming Platform

The increasing demand for video streaming services highlights a gap in the availability of free and open-source tools that support scalable and distributed streaming with video conversion capabilities. Existing platforms like YouTube and Vimeo offer scalability and conversion but are neither open source nor self-hostable. Conversely, open-source solutions such as Jellyfin and PhotoPrism lack the ability to convert videos upon upload and do not scale efficiently.

This thesis aims to develop a backend server that facilitates scalable and distributed video streaming with integrated conversion capabilities. The objective is to create a free and open-source solution that addresses the limitations of current offerings in the market.

The development process began with an evaluation of streaming technologies, focusing on factors such as client support, openness of the standard, and streaming requirements. Dynamic Adaptive Streaming over HTTP (DASH) was selected due to its open standard, extensive client support, and compatibility with royalty-free formats such as WebM-DASH. To achieve scalability, the server was designed to be stateless. Video files are automatically converted and published post-upload utilizing a message queue system for signaling, ensuring scalable and atomic conversion tasks.

Using DASH as the underlying streaming technology allowed the use of modern video formats, reducing storage requirements. The stateless design enabled horizontal scaling, while the message queue ensured efficient handling of video conversion tasks. Each conversion task is processed exactly once and in the order received, ensuring reliability and temporal decoupling.

The developed streaming platform successfully meets the needs for a scalable, distributed, and open-source video streaming service with integrated conversion capabilities. It fills the existing gap by providing a self-hostable solution that leverages modern streaming standards and scalable architecture, making it a valuable tool for communities and organizations seeking an alternative to proprietary platforms.