Intra-frame H.264 / H.265 compared to DNxHR or Prores as intermediate codecs for editing

Question

IMPORTANT: ISSUE SOLVED (STILL A COUPLE OF QUESTIONS, THOUGH), I POSTED SOME INFO BELOW THE ORIGINAL POST AS AN UPDATE IF SOMEONE FINDS THIS USEFUL:

Quick question: what are the advantages of using the most common non long-GOP codecs for editing (DNxHD/DNxHR and Prores) vs intra-frame H.264 with a high bitrate? In theory the compression capability, even if is intra-frame only, is better with H.264, and just by being intra-frame the playback performance while editing should be equivalent to the other two codecs. Besides, H.264 suppports up to 12 bits and 4:4:4 (so it's flexible). I've read what I could regarding these codecs, but I've yet to find a reason for why intra-frame H.264 is not used more extensively for editing.

For the record, I'm asking this because I'm starting an HDR project from a H.264 high 10 UHD 4:2:0 video, and I have two issues: if I try to edit with proxies (with DNxHR or Prores), I get serious sync issues between the source file and proxies, so I can't edit properly. And if I transcode the source file to a file that doesn't have those sync issues (like DNxHR, with DNxHR for proxies too), I lose the HDR data and the video looks like a SDR one (and this happens with any codec, not just DNxHR. I haven't been able to preserve the HDR info of the source file with any codec, and tried with FFmpeg and Adobe Media Encoder), but that is an issue for another post. The thing is that I'm stuck with using the original footage as the source file, but can't edit that way without proxies (obviously the playback is extremely slow), so I was wondering if transcoding the source file to an intra-frame H.264 video and work with that (and without proxies) would have an impact on the final quality. I haven't found info comparing intra-frame H.264 to other intermediate codecs, quality and performance wise.

Thanks in advance.

UPDATE (02/03/20):

I made some tests to see how intra-frame H.264 behaves with Adobe Premiere Pro 2020:

1) I transcoded the original footage (H.264, MKV container, HDR, 10 bits, UHD, 4:2:0, VBR) with FFmpeg to an intra-frame "version" of the file, without changing any other setting (just added -intra to my original FFmpeg command line). I used CRF 18 and veryslow as preset (I have a very good CPU, so the whole file was transcoded overnight). I then imported the file to Adobe Premiere Pro 2020. First, I have to say that I haven't started to edit yet, but at least I could tell that it was compatible, and behaved like an intra-frame video while testing playback (I could go forward and backward very fast). I couldn't see any difference in quality when compared to the original footage either. In other words intra-frame H.264, so far, looks like a good alternative to other intermediate codecs like Prores or DNxHD/DNxHR. In fact, because of H.264 intra-frame compression, file size is smaller than with the others mentioned, and I don't know if it's because of my system (Threadripper 2990WX, 64GB ram, very good motherboard), but performance wise I couldn't tell a difference when comparing to DNxHR at least.

2) I know that extra transcoding steps are never a good thing quality wise, but since I had to transcode to an intra-frame video anyway, I made another test and transcoded the original footage to an HEVC video using FFmpeg (with libx265), keeping all the original settings. CRF used was 18, with veryslow as preset too. I used the main10-intra profile of x265. Then I did the same with another video, which was SDR. It took a little longer as you can expect, but I wanted to do this for a couple of reasons: first, because I wanted to know how Adobe Premiere Pro 2020 handles an H.265 HDR UHD intra-frame video. Second, because I've read (and don't quote me on this) that after transcoding any 8 bit video to a 10 bit one, many perceive an increase in quality, because of the broader colorspace that lets the encoder pick from many more colors while transcoding, which reduces banding. Well, I didn't perceive any difference quality wise (compared to the intra-frame H.264 file, and to the original footage, on both the HDR and SDR files), but the file sizes were obviously smaller and at least on my PC they performed very well on Premiere Pro (playback was as fast as with the intra-frame H.264 videos). Obviously the HDR video playback doesn't show the correct colors, but that's a restriction of Premiere because of the way it handles color spaces (no REC2020 yet).

3) Because I had color issues while transcoding to DNxHR before and couldn't solve that, I started to think that it may had to do with the chroma subsampling (none of the DNxHR flavors support 4:2:0, which is the subsampling of the original video). That was another reason for trying with intra-frame H.264 (or H.265), to see if transcoding to 4:2:0, 4:2:2 or 4:4:4 made a similar difference in colors compared to DNxHR. Turns out that when transcoding to 4:2:0 (with H.264 or H.265 as codecs) the colors look exactly the same as the original footage, and both 4:2:2 and 4:4:4 look a lot like the DNxHR video (colors washed out). Can't see a difference between 4:2:2 and 4:4:4, but when compared to 4:2:0, the difference is huge. I never wanted to upsample the video in the first place, it was just because DNxHR doesn't support 4:2:0, but I never expected such a difference. And if it was because of upsampling, I don't quite get why 4:2:2 and 4:4:4 look exactly the same. Maybe it's some kind of a FFmpeg bug that messes with colorspace when upsampling, idk.

Anyway, now I have working H.264 and H.265 intra-frame videos, without the color issues (checked the files visually, with Mediainfo, and with the Lumetri scopes tab of Premiere. They indeed preserved all the metadata needed for HDR), without sync issues (I also made a couple of proxies with the exact same settings, but only with less resolution. They sync perfecly with the source file), with smaller file size than with DNxHR and Prores, and that perform very well on Premiere Pro 2020 while previewing (maybe they don't with an inferior CPU, I don't know). So one could say, for the meantime (I have to start editing, maybe I'll encounter some issues along the way. And I haven't tested to export from Premiere using these files yet), that my issue is solved.

But my question remains after these tests: why aren't intra-frame H.264 or intra-frame H265 more extended as alternatives to DNxHR or Prores (the most commonly used intermediate codecs)? I can't see nothing but advantages: smaller file size, good playback performance, very good quality (and if you have enough space, you could even make an intra-frame H.264 lossless file if you want), they preserve the HDR data, and both codecs are very well known and extended. They even have their own intra-frame profiles (for example, H.265 has main10-intra, main444-10-intra, etc.). Transcoding times, in my experience, at least using FFmpeg on a PC, is not that different when compared to DNxHR or Prores. Is there any reason for this not being the ideal way to go while editing, besides the fact that these intra-frame "versions" of H.264 and H.265 are not as frequently used as the other codecs in bigger productions?

Thanks, any insight on this would be appreciated. And I don't mind sharing the FFmpeg commands I used if someone finds this useful.

Answer 1

ProRes/CineForm/DNxHR caught on because of the platforms which decided to use them. Initially, H.264 didn't have 10-bit implementations and H.265 didn't exist.

Those formats are made to hold up to being re-encoded while being very fast to encode/decode. They're not painstakingly tweaked for the best possible quality at a given bitrate.

Several cameras can save files in all-intra H.264. I know Panasonic and Sony support it. Sony calls it XAVC S-I. That's just their brand name for H.264 all-intra with 10-bit, 12-bit, 422, and 444 support. And it supports PCM audio which is not actually compliant with the MP4 container spec. These formats are catching on in cameras because the cameras have very specific bitrate requirements to write to an SD card. But then if you look at an external recorder like the Ninja V which can record to an SSD, that's going to record in ProRes/DNxHR because it can handle the higher bitrate — and because nobody's going to run out to buy something that says it supports XAVC S-I. People want ProRes/DNxHR because it makes their life simpler. I can pop the SSD out of my Ninja and plug it into my Mac and drop it onto a timeline in Resolve and I know that works and it's not going to try to make a proxy of it.

So if you want the best quality all-intra file you can certainly get that. You can make it lossless. You can do a "veryslow" FFMPEG render. But Premiere/Resolve aren't going to add the option to do an all-night proxy render.

Here's a very good discussion of the quality of all-intra H.265 vs. H.264 vs. ProRes: https://www.eoshd.com/comments/topic/46562-prores-vs-h264-vs-h265-and-ipb-vs-all-i-how-good-are-they-actually/

The reason H.265 isn't much better than H.264 is because most of the gains in compression in newer codecs are from advanced temporal compression, and all-intra by definition has no temporal compression.

Maybe in the future we'll have more standard all-intra formats, but that's probably going to be something post-HEVC. There is arguably a benefit to keeping archival footage in an all-intra format that's easy to work with. An all-intra format has less quirks and might be better for uploading to YouTube in some cases depending on how it gets along with however they're transcoding it. But these are edge cases. The all-intra version is by definition lower quality for a given bitrate.

The real nightmare — which is what made you ask the question in the first place — is dealing with that HDR metadata. It's all a bunch of voodoo which you might be able to sort out with ffprobe, mkvmerge, and mkvpropedit.

You need to figure out what your problem actually is: What metadata is on the file? Is it tags on the video stream or tags in a separate side data stream? You should be able to get FFMPEG to preserve that. But what works in one container format might not work at all for another container format.

It looks like Premiere doesn't like your source file and screws up the sync, and FFMPEG doesn't like your source file and screws up the metadata. Where exactly is this source coming from?