I have a source video (source.mkv, which is a 25fps 1080p x264 encoded video), and I have reencoded it using SVT-AV1, like this:

ffmpeg -i source.mkv -nostdin -f rawvideo -pix_fmt yuv420p - | SvtAv1EncApp -i stdin -w 1920 -h 1080 --fps 25 -b output.ivf
ffmpeg -i output.ivf -c:v copy -c:a copy output.mkv

The video output.mkv can be played successfully in any video player. When I compare it with the source video file, it looks like the video quality is barely the same. So I proceed to confirm this using vmaf. The procedure I'm following is:

# cut
ffmpeg -i source.mkv -t 00:01:00.000 -c:v ffv1 -an ref.mkv
ffmpeg -i output.mkv -t 00:01:00.000 -c:v ffv1 -an dis.mkv

# convert to yuv
ffmpeg -i ref.mkv -pix_fmt yuv420p -vsync 0 ref.yuv
ffmpeg -i dis.mkv -pix_fmt yuv420p -vsync 0 dis.yuv

# run vmaf
vmafossexec yuv420p 1920 1080 ref.yuv dis.yuv vmaf_v0.6.1.pkl --log vmaf_output.xml

I'm cutting the videos to just measure a portion of them. As you can see, I'm reencoding the video files with ffv1. This is just to cut as precise as possible (I've also tried copying the stream, without reencoding, but the result is the same). I've also tried ffmpeg2vmaf instead of manually converting files to YUV before calling vmaf, but the result is the same. It is important to say that I have followed this same methodology to compare x264 with x265 and I had not any problems with the vmaf score. However, when I look at the vmaf output, I get a nonsense value (I am expecting more than 90):

[drnoob@linux svt-av1]$ grep -o 'aggregateVMAF="[^"]*"' vmaf_output.xml

ffmpeg, vmaf and SVT-AV1 have been compiled manually by me using gcc 8.3.0. Moreover, specific versions are:

  • ffmpeg: N-97127-g5a0575e
  • vmaf: commit 31565cf4
  • SVT-AV1: commit 882e97f

My research so far

I compare both videos side by side (the original x264 one and the output using av1) and it seems like the quality is perfect. So, the first thing I thought was this issue is caused because of both videos being not synced. It is interesting to note that, although the aggregate VMAF score is so low, the first frame gets a score of 90. If I run ffmpeg on both, I can see there are differences:

[drnoob@linux svt-av1]$ ffmpeg -i ref.mkv
Duration: 00:01:00.00, start: 0.040000, bitrate: 149417 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv420p, 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
      ENCODER         : Lavc58.77.101 ffv1
      DURATION        : 00:01:00.000000000
[drnoob@linux svt-av1]$ ffmpeg -i dis.mkv
  Duration: 00:01:00.00, start: 0.000000, bitrate: 108269 kb/s
    Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv420p, 1920x1080, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
      ENCODER         : Lavc58.77.101 ffv1
      DURATION        : 00:01:00.000000000

As you can see, the start time is not the same. However, the difference should not be the cause of such a big vmaf loss, so there must be another reason. I am also suspecting that my procedure to convert the SVT-AV1 output to a file might not be the best (from ivf to mkv), and may be causing some problems. However, decoding the file with a video player shows no issues.

Any idea? I don't think this is a vmaf bug, there must be a problem with my approach.

Thank you!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.