I have a lot of video files but also a lot of duplicate with different encoding differences (eg 720p and 4K versions). I wasn't able to find an open source solutions. I tried a Windows paid program who did half the job.
You could check out https://theophanemayaud.github.io/video-simili-duplicate-cleaner/ (macOS binaries available) It's open source, I created it based on vidupe and QT : https://github.com/kristiankoskimaki/vidupe (windows binaries available but no "auto deletion based on specific criterion")
The idea is to extract a few (eg 2, the first and the last) frames, and compare them. If the duration is close, and the frames look similar : you've got a duplicate.
It uses opencv for frame comparison, and ffmpeg for frame extraction (ffmpeg is also for video metadata which is very useful when trying to tell which video to keep !). Since it's open source, you can improve it to fit your needs !