Using ffmpeg alphamerge to mask two images before calculating similarity index

Question

I'm working on a script that removes duplicate frames from images stored on disk. I need to ignore certain irrelevant parts of the images.

The following ffmpeg command line should work, but it returns the same SSIM statistics as without mask.

ffmpeg -i 1.webp -i 2.webp -i mask.png -filter_complex \
"[0][2]alphamerge[img1]; \
 [1][2]alphamerge[img2]; \
 [img1]format=yuva420p[masked1]; \
 [img2]format=yuva420p[masked2]; \
 [masked1][masked2]ssim=stats_file=-" -f null -

Example output:

[Parsed_ssim_4 @ 0x7fb76b70ff00] SSIM Y:0.829392 (7.680003) U:0.987882 (19.165656) V:0.993201 (21.675403) All:0.883108 (9.322165)
[out#0/null @ 0x7fb76c604700] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.602x    

Result is the same with ffmpeg -i 1.webp -i 2.webp -lavfi ssim=stats_file=- -f null -.

[Parsed_ssim_0 @ 0x7f9c0cf06d80] SSIM Y:0.829392 (7.680003) U:0.987882 (19.165656) V:0.993201 (21.675403) All:0.883108 (9.322165)
[out#0/null @ 0x7f9c0da08e80] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown

This is weird because the masked output files look as expected if I output them with:

ffmpeg -i 1.webp -i 2.webp -i mask.png -filter_complex \
"[0][2]alphamerge[img1]; \
 [1][2]alphamerge[img2]; \
 [img1]format=yuva420p[masked1]; \
 [img2]format=yuva420p[masked2]" \
-map "[masked1]" masked_1.webp \
-map "[masked2]" masked_2.webp

Does anyone know what the right syntax would be for getting the SSIM values from a single command using a mask?

Side note: If I use -map "[masked1]" masked_1.jpg it outputs the original picture, not the masked one. Could there be a problem in the ffmpeg pipeline?

Thank you very much!

Answer 1

What alphamerge does is attach a grayscale image as an alpha component to the base image. Video quality metrics don't factor in the alpha component.

What you need to use is a filter like maskedmerge. e.g.,

ffmpeg -i 1.webp -i 2.webp -i mask.png -f lavfi -i "color=black:s=WxH" -filter_complex \
"[3][0][2]maskedamerge[img1]; \
 [3][1][2]maskedmerge[img2]; \
 [img1]format=yuv420p[masked1]; \
 [img2]format=yuv420p[masked2]; \
 [masked1][masked2]ssim=stats_file=-" -f null -

where W and H should be replaced with dimensions of the base images.