6

The effect I'm referring to:

Essentially, the snow/confetti in the shot means lots of motion, hence a lot of the available bitrate is taken up by it, thereby reducing the quality of the other elements in the shot.

In particular, I see this problem with videos that have animated LCD displays in the background:

Those videos are uploaded by the original TV broadcaster (with access to the highest quality original video), and more or less top out looking like 480p, even when viewing at 720p or 1080p. The same quality issues appear in videos on the broadcaster's own site, so it's not just YouTube's compression going overboard:

http://www.mbcplus.com/Player/MusicBox/RetrieveClipInfo.aspx?clipId=736

If a human were deciding what elements in the shot to dedicate the bitrate to, they would obviously allocate it to the foreground objects (the people) at the expense of lowering the quality of the background (the LCD).

Is there any technique (encoding/compression/etc.) that can improve the quality of the foreground elements when such a background is present in the shot?

And I'm sure some people would also be interesting in any techniques that can be done at the time of filming that would produce better results, when they cannot change the content of the shot and have to deal with snow/confetti/animated LCD in the background.

1
  • Good question. I wonder if using noise reduction would work? Might be worth having a try with different noise reduction algorithms, because the bandwidth hogging background elements are a bit like noise. Of course if you had the time you could mask / roto out the background and add a bit of blur to it, but in those examples it would be a lot of work.
    – stib
    Oct 28, 2016 at 1:49

4 Answers 4

2

Yes this feature exists, but it is not widely used yet.

Texas Instrument developped such solution, called SmartCodec, which looks to be implemented in a Samsung video surveillance system: Megapixel Camera This is a HW based solution, that is for market of video surveillance.

I think it would be implemented in future video codecs HEVC, adopted by many boradcaster, and it's called Region-of-Interest Based Rate Control Scheme HEVC is already released, and in used today, but this feature really looks new. And it will take months~years for it to be implemented in the encoder on the production house, I guess. Apparently, it is something really new, since a paper about that has been accepted for presentation at the ICIP 2016, which took place in september.

1
  • To my knowledge, this is active research in the context of video calls in sign language.
    – TRiG
    Feb 24, 2018 at 21:29
1

Is there any technique (encoding/compression/etc.) that can improve the quality of the foreground elements when such a background is present in the shot?

I don't know of any encoding algorithm that can differentiate between objects in a video and allocate different amounts of bits for those objects based on how important they are. That's not really how compression algorithms work. Maybe with machine learning we will get there eventually, but I'm pretty sure no such thing exists.

Those videos are uploaded by the original TV broadcaster (with access to the highest quality original video), and more or less top out looking like 480p, even when viewing at 720p or 1080p. The same quality issues appear in videos on the broadcaster's own site, so it's not just YouTube's compression going overboard:

The bad quality is caused by the low bitrate in combination with video content that can't be compressed very good (as explained in the video by Tom Scott). Another way to put this is the video has high entropy, i.e. more information than a regular video (without confetti, snow or something similar). To save a video with high entropy in the same quality as a video with low quality, you need a higher bitrate. So if you have control over the final video, just raise the bitrate and you will be fine. Another possibility is to use a variable bitrate. This way, the encoder can allocate a higher bitrate to the parts of the video with high entropy and a lower one for the parts with low entropy, thus raising the overall quality significantly.

If you upload to Youtube, there's nothing you can do. No matter how high the bitrate and quality of your original video, Yotube will compress it upon upload, making it look bad again. You could export your video in 4k, this might help since Youtube uses a higher bitrate for 4k video, ergo better quality. But if your original footage isn't shot in 4k, I wouldn't recommend that.

And I'm sure some people would also be interesting in any techniques that can be done at the time of filming that would produce better results, when they cannot change the content of the shot and have to deal with snow/confetti/animated LCD in the background.

What can you control?
If you want to avoid this mess, get the confetti out of the shot. Turn the LCD screen off. If you can't do that, change the composition and/or pespective so that less of that stuff is in your shot. Zoom in closer on the actors/dancers so they fill more of the screen, ergo less background noise.

But honestly, this is really a Youtube-specific problem because they use a terribly low bitrate for their encoding. I wouldn't compromise my video composition or content to avoid this problem. Just use a different hoster that encodes uploaded videos with higher bitrates such as Vimeo.

2
  • 1
    Well, of course if we all uploaded and downloaded in 4K this would be much less of an issue. But reality being what it is, we're a long way away from that day, so I don't see this as a YouTube-specific problem. E.g. even the videos on the Korean TV broadcaster's site have the same quality problems, and their site exclusively targets the country with the fastest internet connections in the world. Anywhere video compression is needed will face this problem.
    – subuleech
    Oct 28, 2016 at 3:37
  • @subuleech Actually, a higher resolution does nothing to improve the quality of your video if you don't raise the bitrate as well. In fact, if you use the same bitrate, the 4k video will probably look worse than a 1080p one. Their website is clearly ancient and/or poorly coded (they still use flash ...), so it's no surprise to me that their video content has poor quality as well. They probably have some encoder running that compresses the video too much, or they use bad settings for their exports as well. Raising the bitrate just a bit can do much for your overall quality.
    – MoritzLost
    Oct 28, 2016 at 20:03
0

And I'm sure some people would also be interesting in any techniques that can be done at the time of filming that would produce better results, when they cannot change the content of the shot and have to deal with snow/confetti/animated LCD in the background.

Obtaining a narrow depth of field (which relates to the selection of camera lenses) should help, blurring out some of the 'noise' both in front of and behind the subject. This may incur more effort on the camera operators to keep the subject in focus.

-1

Maybe you can give video editor (Adobe Premiere Pro, After Effects, etc) or video enhancer (AVCLabs Video Enhancer AI) a try.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.