0

I observed a strange behavior while using the select filter. In short, I'm using ffmpeg to detect frozen sections in a video, calculate the exact middle point of these sections, then using filters to extract image snapshots at these timestamps.

The frames are selected with the "select" filter by comparing the timestamps with the desired position: select=eq(t\,TIMESTAMP1)[+eq(...)], which is supplemented with the -vsync 0 option to drop other frames.

As you can see in the last two lines of the console output, the filter returned only 3 frames out of 6:

/opt/ffmpeg/ffmpeg-4.4-amd64-static/ffmpeg -y -hide_banner -i "/tmp/in.mp4" -map 0:v:0 -an -vf 'select=eq(t\,30)+eq(t\,76.84)+eq(t\,104.32)+eq(t\,150.32)+eq(t\,290.6)+eq(t\,524.32)' -q:v 1 -vsync 0 -f image2 /tmp/%06d.jpg

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/in.mp4':
  Metadata:
    major_brand     : M4V
    minor_version   : 1
    compatible_brands: M4V mp42isom
    creation_time   : 2014-10-02T11:59:55.000000Z
  Duration: 00:08:47.92, start: 0.000000, bitrate: 373 kb/s
  Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv), 1024x768 [SAR 1:1 DAR 4:3], 371 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2014-10-02T11:59:55.000000Z
      handler_name    : Mainconcept MP4 Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x5556655cc480] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/tmp/%06d.jpg':
  Metadata:
    major_brand     : M4V
    minor_version   : 1
    compatible_brands: M4V mp42isom
    encoder         : Lavf58.76.100
  Stream #0:0(eng): Video: mjpeg, yuvj420p(pc, progressive), 1024x768 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn (default)
    Metadata:
      creation_time   : 2014-10-02T11:59:55.000000Z
      handler_name    : Mainconcept MP4 Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=    3 fps=0.8 q=1.0 Lsize=N/A time=00:08:44.36 bitrate=N/A speed= 145x
video:209kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

My intention was to extract these images in a single step, but if the filter solution proves to be unreliable, I'll stick to the -ss [timestamp] option. The question is, what causes FFmpeg to behave this way?

EDIT: There seems to be a correlation with the timestamps. When I drop the fractional parts of the timestamps, the command produces four (4!!!) frames instead of three.

3
  • 1
    The 'exact middle point' has to correspond to a frame timestamp; can't be between frames.
    – Gyan
    Apr 5, 2023 at 10:00
  • @Gyan Thanks for the hint. That would imply that frames are handled as infinitely short entities and arbitrary t values are generally unreliable with the select filter. Apr 5, 2023 at 12:13
  • No, t denotes the presentation timestamp i.e. the time at which the frame is expected to be painted. There is a frame duration field but effectively it is implicit and based upon when the next frame is painted.
    – Gyan
    Apr 5, 2023 at 13:47

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.