Video format with dynamic resolution

Question

Are there any video formats that change resolution per frame and with different resolutions at different parts of the screen?

All videos I know today they have fixed resolution.

For video blogs I don't need the highest resolution when I am only talking with frame on me. Selectively I would like to have the higher resoltion on photos I am showing and short clips I attach.

Why I want it?

• 4+K videos would globally kill any internet connection,
• Storage of 4+K unoptimized videos take huge disk space.

When I present a photo I don't need also to copy the photo to each frame of video where it is visible.

The closes thing is presentation like *.ppt . But it is far too complex to play, create(with sound all the time) and stream. Maybe I only need some smart video compression tool?

Answer 1

As others have mentioned, you seem to be confusing resolution and frame rate. There are variable resolution systems for things like VR for foviated rendering (where the area being looked at is rendered in higher resolution than the surrounding areas) and there are also some large format displays where the resolution of the overall visual display may not be consistent, but you aren't talking about either of these situations.

For a conventional display viewed by multiple people, you aren't going to be able to meaningfully change the resolution throughout the video without it being pretty jarring. What you are talking about instead is the idea of varying the frame rate (how often the image updates) or group of pictures compression (where surrounding frames are described in terms of their differences from another frame rather than describing the full image for every frame.

In fact, virtually all modern video codecs use the latter of these two. Groups of pictures have been used in pretty much every video format since the MPEG-2 and the DVD (with some usage before as well). This compression technique is why we can get such incredible compression with so little quality loss.

To put it in perspective, an individual raw frame of 4k video from my Ursa Mini digital cinema camera is 10MB. At 24 frames a second, that's 240MB/sec (bytes, not bits), yet Netflix can give a reasonable quality 4k video feed at only 15Mb/sec (bits, not bytes), that's 1/128th of the data involved in the RAW video with relatively minimal visible difference (and that's before calculating the color information which would actually nearly triple the overall data rate making it less than 1/300th of the uncompressed video).

Simply put, our video compression options available already are staggering using just the latter of these options and will tend to work even better on low motion video where very little changes between frames. 2 pass encoding will give the best result as it will determine the amount of change between frames in the first pass and then optimize the data storage on the second.

Variable frame rate is an option you can pursue to further lower the bandwidth requirements if large portions truly do not need to change at all, but such encoders are more complicated, less commonly available and don't play back quite as reliably across devices. Overall, I'd try seeing what you can get with careful tweaking of modern h.264 or h.265 algorithms with fairly long GOP settings (long key frame intervals).

Answer 2

You can't get "dynamic resolution" because essentially that would mean that some of the time you used a 4K display (for example) to show a 2160P image and some of the time you displayed a 1080P image by pixel doubling - so either way it's 4K but what you could gain (IF there were such a Codec) is an increase in the compression, and that's where you should look.

The h265+ Codec developed by HikVision analyses the image frame by frame and decides what is important and what can be compressed to a greater extent, relying upon prior frames to have the data to 'fix' where the extreme compression was used - it can be decoded with a standard h265 (non plus) decoder.

Here is a quote from their Website: "The average decreased rate between H.264 and H.265 is 47,8%. When applying Hikvision H.265+ the decrease rate increases to an average of 83.7 percent.".

Here's the explanation of this: http://www.hikvision.com/europe/Press-Release-details_71_i14097.html

PDF: http://www.hikvision.com/upload/20161209211521855.pdf

As far as I know it's only available as Hardware in HikVision's cameras, if someone knows where there's an Open Source version of h265+ I'd sure like to know - thanks.

Answer 3

You might look into svg. It's a system for vector graphics, but it allows you to include raster graphics, too, like .png and .jpg files. You could have an SVG file displayed in the same place as your video, and at the appropriate time switch the SVG to being visible over the video. When you want to show the video again, make the SVG file transparent, etc. You could simply make the video black at the points where the SVF file is visible. It should compress very well in those sections.

SVG files can be created in a text editor as they are just XML files. They are supported by every major browser in use today.

Answer 4

I think that this is not needed, both in terms of computational and bandwidth needs. H.264 and H.265 as well the vp8 and vp9 codec use a GOP line for the frames.

Once you have a certain frame and big blocks of the frame are the same or there's just a small movement this is deduced and rendered in decoding phase with a small cpu and very low bandwidth usage.

TL; DR: codecs do it already that for you. You concerns could apply only if you are using uncompressed or some crazy work codec, but that's not the case for the delivery on YT.