A description of the simple orchestrator that I implemented in C/C++ to download livestreams.
Published: May 2026
In this post I go through some of the implementation details of the orchestrator that I implemented as a workaround. For full context on the problems that I try to solve see the previous article Save the Frames.
I have implemented a simple orchestrator that starts and maintains multiple processes of yt-dlp that redundantly download the stream. These processes are started using --abort-on-unavailable and, if a yt-dlp process exits, the orchestrator will waitpid(2) on it and start a new worker. As it automatically restart workers the user is no longer required to babysit the download. Through careful selection of flags it can even start download a new video (with different id) should the streamer stop the stream and restart it on their end.
This is obviously not a great solution. The underlying assumption is that if one process encounters a problem downloading a fragment the other processes might be spared. The solution requires a lot of extra storage and a lot of extra network bandwidth. It might not even work that well if the stream encounters a problem though at least it will restart the download should it encounter a problem.
The main loop of the orchestrator maintains two sets of (up to) N workers; the main workers and the outgoing workers. The main workers are the workers that are currently downloading the stream. After the main workers have been running for T seconds they are marked as outgoing and a new set of main workers are spawned to take over. This is, in part, to keep the sizes of the video files manageable. After the outgoing workers have been running for D seconds (overlapping with the newly spawned main workers) they are signalled to terminate. First with SIGINT to allow them to cleanly exit and, after an additional delay, a SIGKILL is sent to make sure that they are terminated.
For each iteration of the main loop the main workers are checked to see that they are still running. If any workers have exited unexpectedly a new worker is spawned to take its place. If too many workers unexpectedly exits within the duration T the orchestrator will exit and terminate all workers. The streamer probably ended the stream.
The orchestrator creates a new directory for each worker. It then forks and the worker cd:s into the directory and executes yt-dlp using exec(3).
The workers need to know what to download and instead of giving the URL to the stream it is better to give the URL to the streamer. Using some selected options to yt-dlp it is possible to tell it to only download the current active livestream. This also solves the problem if the streamer restarts the stream. The most important flags are --lazy-playlist which prevents yt-dlp to first generate (a potentially long) playlist and --break-match-filter is_live which tells yt-dlp to exit if the next video in the playlist is not live.
It is important that the user is able to cleanly shutdown the orchestrator and workers. When the orchestrator receives a SIGINT signal it will start the teardown process and signal its workers to terminate. It is important for the orchestrator though that the SIGINT is captured synchronously. It would be bad if the main loop could be interrupted at any point during its execution leading to an inconsistent state. E.g. a worker could have been spawned but a signal interrupted the orchestrator before the number of workers was increased. Therefore the orchestrator must block SIGINT using sigprocmask(2) and, in order to not block the main loop, poll for signals using sigtimedwait(2) to guarantee the correct state of the program.
It is important for the forked workers to unblock SIGINT otherwise it will be blocked for yt-dlp after exec(3). Otherwise the orchestrator could not tell its workers to terminate.
Download the orchestrator using git clone https://mvidell.se/livestream.git and compile it (on Linux) using make.
Usage: livestream [OPTIONS] URL
Options
-h, --help Print this help and exit.
--test Run the test_worker instead of yt-dlp.
Make sure it is in $PATH.
Any other options are passed to yt-dlp.
Examples
livestream https://www.youtube.com/@USER/streams
livestream -f <code> -r <rate> https://www.youtube.com/@USER/streams
Default Options
yt-dlp --lazy-playlist --max-downloads 1
--skip-playlist-after-errors 1
--break-match-filter is_live
--abort-on-unavailable-fragments
--fixup never