What exactly do FFmpeg options `-itsoffset` and `-itsscale` do?

Question

If I understand correctly, the answer at Lossless A/V delay with ffmpeg seems to suggest that option -itsoffset leads to additional metadata, called "editlist", being added to the video file. It's then the video player's responsibility to honor this metadata and shift the stream accordingly.

Is it possible to display this metadata to find out if a video file has been edited this way?

What about -itsscale? Does it work the same way?

Answer 1

Is it possible to display this metadata to find out if a video file has been edited this way?

In the MPEG-4 parlance, it is not metadata — it is an edit list table inside an edit list box.

Note: My previous links point to the QuickTime file format (MPEG-4 is derived from it, and it is almost the same), and what is called an atom in this format, is called a box in the MPEG-4.

You may see the edit list table(s) in any hexadecimal viewer — search for the elst string, then

• skip 4 bytes (version and flags of this edit list atom, probably 00000000).
• In the next 4 bytes is the number of entries in subsequent edit list table — probably it will be 00000001 or 000000002 — see red boxes in my 2 pictures of 2 similar .mp4 files.

After it follow edit list table itself - 12 bytes for every entry.

In every of these 12-bytes long entries:

• you may skip the first 4 bytes (the duration of this edit list segment),
• in the next 4 bytes (green boxes in my pictures) is the starting time of this edit list segment (in media timescale units) — the value FFFFFFFF means that this is an empty list (so you may completely ignore this entry)
• in the last 4 bytes (blue boxes in my pictures) is the media rate of this edit list segment in the so-called fixed point number format:
  • in the first 2 bytes is the integer part,
  • in the last 2 bytes is the fractional part
    (in the hexadecimal number base, i.e. the first hexadecimal digit represents the number of sixteenths, the next number of 1/256, etc.).

Notes:

1. elst is the box type value of the edit list atom.
2. I used the internal viewer of Total Commander (in the hexadecimal mode), but you may use any other.
3. I doubt that the current versions of FFmpeg use edit lists for processing -itsoffset and -itsscale input options.