What's the difference between "visually lossless" and real lossless and what does this mean for future encodes?

Question

In FFmpeg's H.264 Video Encoding Guide we are told:

Tip: If you're looking for an output that is roughly "visually lossless" but not technically lossless, use a -crf value of around 17 or 18 (you'll have to experiment to see which value is acceptable for you). It will likely be indistinguishable from the source and not result in a huge, possibly incompatible file like true lossless mode.

This raises some questions. The first and obvious one is "how did they decide that?" The important question is the one in the title. What's the literal difference and if I need to encode from this output later (e.g. I want to compress and archive old data), will a second encode on a crf 18 encoded file give me problems that a crf 0 encoded file wouldn't?

Answer 1

Technically lossless, or real lossless, means that when the output is decoded by a conformant decoder, the result is mathematically identical to the input.

Visually lossless is a non-rigorous term that indicates subjective transparency i.e. output which to the lay human eye looks indistinguishable from the original. The output may look different if a viewer scrutinizes it in fine detail, but not in normative viewing conditions. As to how 'visual-losslessness' is determinined: there are video quality metrics, like PSNR and SSIM, that provide a quantitative measure of the difference of an image with respect to a reference. Subjective opinion tests have been carried out to establish threshold values for these metrics that can be used to define 'visual losslessness'

From Performance Analysis of Visually Lossless Image Compression,

Note that there are several visual quality metrics (indices) such as MSSIM [6], PSNR-HVS-M [7], WSNR [8] and some others that are able to characterize well enough quality of lossy compressed images [9]. Recently, it has been shown for grayscale images [5] that it is enough to provide MSSIM values larger than 0.985 or PSNR-HVS-M values larger than 40dB in order to guarantee practically invisibility of distortions in lossy compressed images.

---

As for re-encoding resilience, depends on what the goal of the re-encoding is. For viewing-only or casual re-editing purposes, CRF 18 in x264 should be fine. For professional re-editing, I would drop the CRF to, say, 12 or 14. Information, once lost, can't be perfectly recovered. But you can set your own standards by encoding at different CRFs and comparing manually. See https://video.stackexchange.com/a/17639

Answer 2

In addition to "visually lossless", there's the concept of "visually almost lossless":

• "Visually almost lossless" is usually established by static side-by-side viewing. This is probably what the CRF is about, since it "looks lossless". I believe Gyan's mention of "scrutiniz[ing] in fine detail" is about this one too.
• "Visually lossless" has a stricter requirement. It is established by a "flicker" test or a "panning" test. The former is the accepted standard under ISO/IEC 29170-2 written by the JPEG. The latter is a proposal for addition used in an evaluation experiment for Display Stream Compression in DisplayPort. It is apparently more sensitive.

Although both measures are inevitably subjective, humans do appear quite consistent with their judgements as the one large-scale experiment shows.

Answer 3

How did they decide that?

To answer literally, I am the original author of that article and added the claim years ago that CRF ~18 is visually lossless (but not technically lossless like CRF 0). I got that information directly from the main x264 developer (Glaser aka Dark_Shikari) in the #x264dev IRC room, and it looked lossless to me.

Re-encoding: 0 vs 18

If you need an intermediate and/or temporary file for re-encoding, use -crf 0 or some other lossless compressed format, such as UT video or FFv1, that works within your workflow. Might as well avoid any unnecessary compression artifacts and provide the best quality you can to your clients.