FFMPEG : Add video (partial video effect) under the image with ffmpeg

Question

I have a video, video.mp4, and an image, artwork.png

I would like, to use FFmpeg:

• To display the video.mp4 effect inside the image artwork.png
• If possible, resize output.mp4 as per the demo demo_output.mp4

Is this possible?

Resources: https://www.dropbox.com/transfer/AAAAAGgUAt5aTw0W5DYH75y-CvJTKQUqxXatnMPNjHBxpz4DDiOH9IM

Have tried : ffmpeg -i video.mp4 -i artwork.jpg -filter_complex '[1:v][0:v]scale2ref[ckout][vid];[vid][ckout]blend=all_mode='lighten'[out]' -map '[out]' output.mp4

Answer 1

You would use scale2ref to match the fx clip with the artwork and then the blended result with the demo reference. For correct results, blend will have to work with RGB inputs, so those inputs have to be converted first.

ffmpeg -framerate 30 -i artwork.jpg -i video.mp4 -i demo_output.mp4 -filter_complex "[1][0]scale2ref[fx][bg];[bg]format=gbrp[bg];[fx]format=gbrp[fx];[fx][bg]blend=all_mode=lighten[mix];[mix][2]scale2ref[v][ref];[v]setsar=1[v];[ref]nullsink" -pix_fmt yuv420p -map [v] out.mp4