1

I have a five second clip that is stripped from a much longer video. I need to identify exactly where in the longer video this clip begins/ends. There could be multiple occurrences of the clip, I would like to identify them all, not just the first one. How would I go about doing this?

I have used the mpdecimate filter in ffmpeg to identify repeated frames before, but I can't quite think of anything to use for this specific task of searching for a video within a video.

2 Answers 2

2

I can think you should try this project, https://github.com/nielstenboom/recurring-content-detector. I would recommend trying the Color Histogram or Color Texture Motion algorithms. The Convoluted Neural Network one is quite compute intensive and the results haven’t been that good for me. Since your looking for such a small clip, you might need to reduce the frame skip parameter to get more accurate timings.

2
  • Exactly what I am looking for, but not setup to do any Python development. Thank you! I will take a stab at making a wrapper Commented Dec 19, 2020 at 18:41
  • I believe you can run it as a docker container, if you don’t want to setup a full python environment with all dependencies. Commented Dec 20, 2020 at 19:43
0

Perhaps you can export both videos (long and short) to frames and then use duplicate image finder software to identify the matching frames?
I don't have concrete suggestions, but overall the process would be:

  1. Export the short video to frames to one folder, the filename of every frame should be the timecode of that frame for easy identification and sorting. This answer on StackOverflow explains partially how to do this with ffmpeg.
  2. Export the long video to frames in a similar way
  3. Get a duplicate image finder, a web search will turn up many. Ensure that the software uses the content to identify duplicates, not the metadata. This Photo.SE question has some suggestions.
  4. Take the first frame of the short video and set it as your reference image in your duplicate finder and run the software in the folder that contains the frames from your long video.
  5. If all goes well the software will return a set of frames that visually match the reference frame. From the filename of those matches you can identify the timecode.
2
  • Thank you for the reply, I am looking for something much more efficient. I was thinking some way to hash a sequence of frames and scan for it with a filter. The video equivalent of the classic Shazam app, if you will. I know it will be an intensive operation, but I can not consider exporting every frame. Commented Nov 11, 2020 at 17:09
  • I would also prefer a shazam like app, but I was not able to find one. Hopefully someone else can suggest it. But exporting every frame maybe sounds a more labour intensive than it is: ffmpeg can do this fairly quickly, you just need some spare harddisk space (relevant question on SO). Perhaps you can indicate in your question why exporting every frame is not an option? Is there some diskspace requirement? Commented Nov 11, 2020 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.