Video encoders that display frame with most movement

Question

Are there any video encoders that can monitor multiple video frames and display whichever frame is the most active? I would use this during a live streaming event and it would be great to automate the process.

Answer 1

This feature is called "motion detection" in the video surveillance jargon and can be found on most IP cameras and NVRs. There is a large variety of solutions out there depending on the level of accuracy that you need, the ability to divide the monitored perimeter into zones, whether the analytics run off a live or recorded feed, the OS that you use, and more.

If you look for something free and simple that just feeds you with XMLs then you might check Shotdetect. If you look for more of a "solution" then check motion, note that it is unix-specific.

It is also possible to use ffmpeg to extract thumbnails from your feed and run SNR comparisons on the fly, if you extract only I-Frames then the overhead is minimal. You are bound though to a detection rate that depends on your keyframe rate.

Try to provide some more detail regarding your operating system and overall use case for a more specific answer.

Answer 2

I assume you have multiple inputs and want some kind of automatic editing that will switch which feed is live based on what has activity at the time. Unfortunately, pure changes in pixel values aren't really a super ideal way to tell what is interesting.

There are systems which handle automatic selection of the speaker such as Microsoft's RoundTable system which is able to recognize who is speaking based on position of the audio source and this kind of triggering is pretty common for source selection in things like WebEx and other meeting platforms.

You might be able to find something along the lines of what you are looking for in security software as well as security systems would want to display the most active frame. In general, it probably isn't something you will find in most streaming systems though. It would require a relatively high level of development to do well and the nicer streaming systems are going to be more about allowing an editor to directly control things as any high end stream is going to have a dedicated editor manning the console.