2

JPEG has a progressive enhancement mode where a full frame can be downloaded quickly at low quality, and then the quality enhanced as the file is continually loaded.

Progressive JPEG Example

(Example image from http://promokit.eu/prestahop-blog/image-optimization-to-speed-up-your-site/)

I'm wondering if there is a widely compatible video codec that enables the same functionality.

If the codec were structured this way, it should be technically possible for a player to download a first-pass low-quality stream, begin playback immediately, and then download second-pass and third-pass quality enhancing streams if bandwidth is available to do so, combining them on playback for form progressively higher quality frames.

On the server, there would be segment files for each quality stream, similar to how HLS works today.

HLS chunking diagram

(Diagram from http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-is-HLS-(HTTP-Live-Streaming)-78221.aspx)

The key difference here is that with HLS, entire separate copies of the stream are stored in each stream's segments. If I want a mid-quality stream, I read from the mid-quality files only. If I want low-quality, I read from the low-quality files only.

What I'm proposing with progressive enhancement is that all players would always read the low-quality (first pass) stream files, and additionally the mid-quality (second pass) stream files if they wanted that mid-quality stream. For high quality, they would read all three. This should greatly reduce the storage overhead of the stream since the total disk usage to support low, mid, and high quality streams should be only slightly more than would be required for just the high quality stream. This means that the upload bandwidth required on the source end is much less as well, since the stream is effectively uploaded once (plus some overhead) for all three qualities. I imagine there would be overall a bit less CPU used in the encoding process since I'm only really making one stream (albeit in an unusual structure) rather than having to run three independent codecs simultaneously where much of the codec work is likely duplicated.

I have read about some proprietary codecs in random tech news articles, but never found anything concrete. I also know that a few companies did audio codecs this way when audio streaming was booming in the mid/late 90s, but that's apples and oranges.

Questions

  • Is there a codec available that allows for progressive enhancement and segementation?
  • Is there a way to do this with a widely implemented codec like VP8 or VP9?

I suppose there is some hacky stuff I could do with MJPEG, since MJPEG is basically JPEG progressive enhancement under the hood, but I'd rather find a better codec if it exists.

  • 2
    Not quite the same thing but look up Scalable Video Coding. + – Gyan Apr 18 '16 at 19:34
  • Wouldn't the problem be that the player would have to download each stream entirely before it could play? Though I guess you could download GOPs so that you download GOP1-low, GOP1-mid, GOP1-hi, then GOP2-low, GOP2-mid etc., only downloading what you can in the time available. – stib Apr 27 '16 at 14:10
  • @stib Segmentation would be used, similar to how it is used in HLS, so that a few seconds of video would be encoded in each chunk. You're right, each chunk of each quality stream would have to be downloaded to decode the desired quality for that chunk of time. And, the client would automatically adjust between downloading 1, 2, or 3 chunks per segment as it has bandwidth to do so. For HLS, the client decides which bitrate chunk to download, somewhat dynamically. This would be the same, except that there's 3 requests for the high quality stream instead of one. – Brad Apr 27 '16 at 16:39
  • 1
    Quick! get a patent! – stib Apr 27 '16 at 21:54
  • @stib I think that would require me to actually make such a codec. Math is hard. :-) Although, I could probably actually implement it in MJPEG since MJPEG already does the heavy lifting... – Brad Apr 27 '16 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.