Color leakage effects with ffmpeg blend=all_expr

Question

I generate two test videos (the first a test pattern, the second solid red) with

ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 pattern.mp4
ffmpeg -f lavfi -i [email protected]:duration=5:size=1280x720:r=10 red.mp4

If I blend these together with the solid red as background, I see the expected output: left half test pattern, right half solid red.

ffmpeg -i pattern.mp4 -i red.mp4 -filter_complex "[0:v][1:v]blend=all_expr=if(lt(X\,600)\,A\,B)" output.mp4 

However, when blending in the opposite order, the output is not as expected, with the solid red "bleeding" over the entire image.

ffmpeg -i red.mp4 -i pattern.mp4 -filter_complex "[0:v][1:v]blend=all_expr=if(lt(X\,600)\,A\,B)" output.mp4

Why is the "A" video leaking into the "B" zone?

Answer 1

The default pixel format for testsrc filter is rgb24, which gets converted to yuv444p for encoding to H264. The default pixel format for color is yuv420p, which is kept as-is.

The blend filter requires both inputs to have the same pixel format, and will, where possible, convert the pixel format of the 2nd input to match that of the first. When red.mp4 is the first input, the frames from pattern.mp4 are converted to yuv420p. In yuv420p. the chroma planes are half the width (and height) of the luma plane, so in this case, 640x360.

Your blend expression, which is applied to all planes is, if(lt(X\,600)\,A\,B). For the chroma planes of a 1280x720 frame in yuv420p format, X = 600 is at ~6% from the right edge, hence the result you're seeing. You can modify the expr to if(lt(X\,SW*600)\,A\,B) to get the expected result. SW is the scale factor for the processed plane and represents the ratio of the current plane to the luma plane.