User:MarcoZ/Streaming the Rigcam

The Problem

The Rigcam, aka Skype, has certainly done a lot for the cup. It's brought us from the days of Jack and Ken waiting a good minute to see a goal before doing their commentary on it as that was the latency of livestream or twitch to a real feeling of a commentary booth and cutting menu time to a much more reasonable amount since the managers have immediate feedback, both visual and from the streamer's voice.

However, whatever the medium should the Rigcam move on in the future, be it Skype, Discord Video calls or something else, there is one fundamental problem with the current implementation of the rigcam: The rigcam is nothing but a video stream over which we have no control over. Skype can mysteriously ruin callers' voice quality, fail to recover from dropped packets and start to deliver unwatchable video and conference calls like the ones we use seem to put a strain on it more often than not.

On the other hand, services like Twitch and Smashcast are able to keep the quality decently stable at the cost of latency, something we also have no control over as their strict guidelines for what kind of video input they accept do not help on the latency front, and we also don't know what goes on behind the scenes once the video reaches their servers.

However, with the aide of a server and a few pieces of software, we can take back control of every step of the streaming pipeline and go from a one-size-fits-all solution to something tailor made for our means.

The Solution

Three pieces of software make up the entire solution:

• nginx, a web server that can be configured to receive and output RTMP streams with this module. It can handle multiple streams, pushing options (say, for example, that we want to stream both on Smashcast and on Twitch), recording for archival, and much more.
• ffplay, the client part of the ffmpeg project. I've tested a few video players: /g/'s memetic mpv player, mplayer, even VLC with some of the buffering options turned off, but this seemed to be the best option for latency, while also maintaining very watchable video quality. It's also conveniently static compiled, meaning that the rigcam end users would get would just be an .exe file that will work on a huge variety of Windows installations, as it includes everything it needs to run, and a .bat file (or a tailor-made UI if we do come up with one) which would launch ffplay with the proper parameters.
• OBS Studio. After further testing, it seems OBS classic adds further latency (around one second) and we don't want that~

There's two main ways to implement this to our streams, one of which has already been used at an invitational level during the Beta Bowl. I'll detail it first.

Passthrough Method

This method also requires ffmpeg to be installed on the server that's running nginx.

The streamer streams as usual, keeping all settings and software to his preference, however instead of streaming directly to Smashcast, he will stream to the server running nginx, which will then relay to Smashcast. The channel will automatically go live at this point upon input. I then used ffmpeg on the server itself to strip the audio off of the incoming stream and make it live on another URL.

What you end up with is two nearly identical streams, with the benefit that the rigcam has no audio, saving some miniscule amount of bandwidth, but most importantly it seems to help with latency, as ffplay won't be as eager to skip ahead without buffering if there's an audio stream to sync to.

Pros

• Absolutely no impact on the hardware of the streamer. The server handles the entire relay process, without straining the streamer's CPU, GPU or upload bandwidth in any way. The only requirement on the streamer's side of things is that he must not use any delay, and keep the buffer size to a minimum, but it is a lower priority.
• The rigcam's quality is always identical to the maximum video quality of the stream. If any frames drop, it's either the originating streamer dropping frames as he would have on Smashcast or the client having connection issues.
• Ease of use on the streamer's side. If you know how to stream, you can stream this way. Nothing changes on your side but changing the URL you stream to.
• Archival, multistreaming, you name it. Passing the main stream through nginx means that we can use many of its options to stream to multiple services, record the stream and so on and so forth. Haven't really looked into this yet but if you're listening, Klaxa, this is (You).

Cons

• Server maintenance. I'm sure most of this could be automated and secured, but I have not looked into that yet and since the main stream would be passing through a third server, it would become vitally important to make sure nobody messes with either the stream or the server. I also didn't automate the stream-splitting process, meaning that for every day of the Beta Bowl I had to start ffmpeg for the rigcam to work.
• The rigcam's quality is always identical to the maximum video quality of the stream. People with terrible connections will not be able to watch a 720p60 stream and will get either no or terrible video. More testing is required on this front, as no problems manifested itself during the Beta Bowl but the pool of testers there was severely limited.
• Latency. This method means we're still using a Smashcast-compatible video stream for the Rigcam, limiting our options on how to encode the stream. AFAIR on this method it can vary from 3 to 5 seconds of latency, but I would need to re-test it as I've mainly worked on twinstreaming after the Bowl.
• Versatility. The same video stream as the stream itself means that everything that needs to be shown to the managers needs to go to Smashcast too.

Twin-stream Method

Also known as the slightly insane method, this would require the streamer to set up two streams, his usual Smashcast stream and another one going to the server running nginx, to use as a rigcam.

I use OBS classic for the second stream and some x264 parameters that give me the minimum possible latency, see "technical shit" at the bottom of the page. This is unwieldly but doable, especially for people with more than one monitor (I have a 4480x1440 desktop, as of now).

However, as of right now I can't get PES 2018 to display on both OBS instances if I use two Game Capture plugins, meaning that for the rigcam I have to use Display Capture and keep PES unobstructed somewhere. I can work with this but I have way more resolution than what is normal, I'd assume. It would probably help to have a TV just for displaying PES like Sou (or PAVAL? I don't remember exactly) does.

Pros

• Versatility. Want to rig half an hour before the cup day starts? No problem. Need to show something to the call that you can't show to the public for some reason? Sure. Have a brazilian manager on dialup trying to watch the rigcam? You can probably stream down to something even he can watch, while leaving the main stream untouched. Won't be pretty but it'll work. Want to kick off someone who's using the rigcam for any kind of nefarious purpose? You don't need a f4r at your disposal to change the stream key.
• Latency. It's fast. Real fast. Less than an ingame minute behind, and from my last input latency tests it was between 0.5 seconds and 2 seconds, and it stays that way without a hitch.
• Server maintenance. Not as vital when you're just using it to implement a fast rigcam and don't need to store the channel's stream key on a third party server, and you can always go back to Skype as a backup. Also wastes a lot less bandwidth, if that's a concern.

Cons

• OH SHIT NIGGER WHAT ARE YOU DOING Streaming is a balancing act as it is, and adding another stream on top of the main stream is probably more than some people can handle.
• Hardware requirements. New Rigcams or not, Pro Evolution Soccer 2018 will be the biggest streamer killer in the cup's history. The fact that it can chew through a 4.2 GHz desktop Haswell i5 on its own means you need a good rig to even consider streaming one stream of 18, let alone two. You can work around this by using the GPU to encode the main stream (NVENC, AMD VCE and Intel Quick Sync) but some people won't have any of those and this method is just impossible to do while entirely relying on the CPU for streaming, unless you have a monster rig.

Demo

Short Smashcast VOD demonstrating the latency of method 2.

Client

At the moment, the client consist of a DOS batch file which clears the screen, asks you for the stream key (which will be provided to you by the streamer) and then launches the video player, ffplay, with a couple of parameters detailed at the bottom of the page. The stream then starts automatically. There's no controls, the player is just a window and a Command Prompt-looking window which tells you various info about the stream.

So what's all this for?

What we should end up with is a simpler, more streamlined and faster video delivery method for managers and commentators, potentially less straining on everyone's hardware and less headache inducing than Skype's notorious unreliability, and one that also leaves us indipendent from any service. Audio call doesn't work on Skype for some reason? Discord down? Heck we could use Teamspeak for commentators for all I care.

Technical Shit

FFplay parameters: ffplay -fflags nobuffer -flags low_delay -framedrop -strict experimental -hide_banner OBS options for method 2: 1000 kbps bitrate at 854x480, 30 FPS. Delay (seconds) 0, CPU preset superfast, encoding profile high, keyframe interval 5, check "Custom x264 Encoder Settings" and add: tune=zerolatency -crf=25:vbv-maxrate=1000:vbv-bufsize=350:intra-refresh=1:slice-max-size=250:keyint=30:ref=1