Crop a video with a moving crop area of fixed size

Question

I have 3840 × 2160 videos where I want to cut out a moving person on stage (e.g. music video). The crop area is fixed (1080 x 1920). The problem is, the person is moving around. Not much, but in a 3min. video the crop area would move a couple of times.

Example: Frame 0-889, area(500,34,1080,1920) - Frame 950-1735, area (800,10,1080,1920) - Frame 1820-2530, area(600,40,1080,1920)

I gave some example numbers above. The solution should interpolate the crop area between Frame 889 and Frame 950; Frame 1735 and Frame 1820. Ideally I would like to define those areas in a visual editor which allows me to move a fixed size crop rectangle for the Frames and coordinates given above.

Optionally I could also figure out the crop areas without visual help and then use that data to crop the video in something like ffmpeg (although I don't know how to do this properly including the interpolation step).

I am on Linux and tried different video editors but their crop filters are not useful to solve my task. I have Windows available but no commercial editing software installed at the moment.

Thanks for the help!

Answer 1

I recommend AVISynth (a script based video editor for VirtualDub) and the DeShaker plugin.

You can learn a minimum amount about AVISynth and simply load DeShaker, have it stabilize on the person's body and it will follow them smoothly as they move.

You will have a finished product that is stabilized as a bonus.

You can also learn more about AVISynth, establish precise crops that are animated to follow the person, and get a perfect result (based on the quality of the input). You can set your moves with frame and pixel accuracy by typing numbers into the script and reloading the frame in VirtualDub.

It might be worth 5 mins. exploring those links to see if you want to try another video editor.

If one of the current video editors you've installed has image stabilization you can try it, but DeShaker is one of the better free ones (as of a year ago) that I've used.

Answer 2

The best and easiest (and only) way that I have found to do this in Linux is to use OBS with the window size set to the desired crop output dimension. Add your video file as an input source, arrange the video output relative to the OBS window so that the desired rectangle that you wish to display initially is lined up with the OBS window, and start recording in OBS at the same time as you start playing the video. While the video plays, use your mouse to drag the video around in real time so that the part which you wish to view continues to line up with the OBS window. If you have a slow moving subject and you only need a couple of slow pans within a 3-minute window, this works really well. It can be a little more annoying if your crop area shifts around a lot.

If you only need to crop in one dimension (e.g. 1920x1080 to 1080x1080) then you can temporarily lock your mouse pointer to move in one direction only, at least on X (not sure about Wayland). For example:

xinput set-prop 'Microsoft Corporation Microsoft ® Laser Mouse 6000' 'Coordinate Transformation Matrix' 1 0 1 0 0 0 0 0 1

is what I use with my hardware (the idea for this comes from https://askubuntu.com/questions/1153697/screen-detection-and-mouse-reverse-x-y-axis). To get back normal mouse movement:

xinput set-prop 'Microsoft Corporation Microsoft ® Laser Mouse 6000' 'Coordinate Transformation Matrix' 1 0 0 0 1 0 0 0 1

Of course the details will differ if you have a different mouse. This is useful while recording your crop because it prevents you from dragging the video in any direction other than the direction in which you need to pan.