I have a lot of video files but also a lot of duplicate with different encoding differences (eg 720p and 4K versions). I wasn't able to find an open source solutions. I tried a Windows paid program who did half the job.
2
-
1That looks really useful. You might want to edit your question to take out the answer and then answer your own question.– stibNov 27, 2018 at 3:03
-
I managed to answer it by creating an open source program around ffmpeg and findimagedupes. Feel free to try it : github.com/PierreCrette/VideoDedup/blob/master/README.md– PierreNov 27, 2018 at 19:02
Add a comment
|
2 Answers
In fact, there are mostly 3 software for finding duplicate videos.
- Video Compare https://www.video-comparer.com
- Duplicate Video Search https://duplicatevideosearch.com
- Duplicate Media Finder (my favorite) https://duplicate-media-finder.kdo-rg.com
All of them use fingerprints.
You could check out https://theophanemayaud.github.io/video-simili-duplicate-cleaner/ (macOS and Windows binaries available) It's open source, created with QT and relies on ffmpeg.
The idea is to extract a few (eg 2, the first and the last) frames, and compare them. If the duration is close, and the frames look similar : you've got a duplicate.
It uses opencv for frame comparison, and ffmpeg for frame extraction (ffmpeg is also for video metadata which is very useful when trying to tell which video to keep !). Since it's open source, you can improve it to fit your needs !
