Video Production Stack Exchange is a question and answer site for engineers, producers, editors, and enthusiasts spanning the fields of video, and media creation. It only takes a minute to sign up.
Sign up to join this community
I'm looking for an ffmpeg-like tool to blur/remove/change a static background from a "talking head" video, on the command line with on-device processing.
Essentially,
ffmpeg-like-tool [--blur-background | --remove-background | --change-background] --background image.jpg --in input.mp4 --out output.mp4
Does such a tool exist yet?
There is a reason why monotone background are used, they allow one to determine where the object is, and where is the background. As far as i know no took exists, which can always determine what is the background, and what is the object that is actually being filmed.
If the background is really static (not moving in the frame, rather than simply always being behind the person), then you could attempt to extract frames and determine the pixels that keep their value above some arbitrary percentage of frames. You then can create a mask based on if the pixels of the specific frame are the same as the value of the 'stable pixels', and blur in that mask.
The the background is moving in the frame due to camera movement or changes in some manner, this becomes very difficult, and i may even guess impossible in some situations. Also even if it will work, it will need to be double checked to see that you won't get false positives. You will definitely get false negatives near the person's head.
In the even you have resources for development and really need it to be done, you could go neural network route. Be prepared to provide neural net lots and lots of training data. For that you would need to film akin to the videos that you will be processing, but doing that in front of the green screen. This will allow you to determine where the head truly is in each frame, because once you put in the background it will be the job of the neural net to figure out where the head is, and your job to tell it whether it is correct or not during the training, and to be able to do that in testing to see how well it has learnt to identify the human head.
Good luck.
