I observed a strange behavior while using the
select filter. In short, I'm using ffmpeg to detect frozen sections in a video, calculate the exact middle point of these sections, then using filters to extract image snapshots at these timestamps.
The frames are selected with the "select" filter by comparing the timestamps with the desired position:
select=eq(t\,TIMESTAMP1)[+eq(...)], which is supplemented with the
-vsync 0 option to drop other frames.
As you can see in the last two lines of the console output, the filter returned only 3 frames out of 6:
/opt/ffmpeg/ffmpeg-4.4-amd64-static/ffmpeg -y -hide_banner -i "/tmp/in.mp4" -map 0:v:0 -an -vf 'select=eq(t\,30)+eq(t\,76.84)+eq(t\,104.32)+eq(t\,150.32)+eq(t\,290.6)+eq(t\,524.32)' -q:v 1 -vsync 0 -f image2 /tmp/%06d.jpg Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/in.mp4': Metadata: major_brand : M4V minor_version : 1 compatible_brands: M4V mp42isom creation_time : 2014-10-02T11:59:55.000000Z Duration: 00:08:47.92, start: 0.000000, bitrate: 373 kb/s Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv), 1024x768 [SAR 1:1 DAR 4:3], 371 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default) Metadata: creation_time : 2014-10-02T11:59:55.000000Z handler_name : Mainconcept MP4 Video Media Handler vendor_id :  encoder : AVC Coding Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native)) Press [q] to stop, [?] for help [swscaler @ 0x5556655cc480] deprecated pixel format used, make sure you did set range correctly Output #0, image2, to '/tmp/%06d.jpg': Metadata: major_brand : M4V minor_version : 1 compatible_brands: M4V mp42isom encoder : Lavf58.76.100 Stream #0:0(eng): Video: mjpeg, yuvj420p(pc, progressive), 1024x768 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn (default) Metadata: creation_time : 2014-10-02T11:59:55.000000Z handler_name : Mainconcept MP4 Video Media Handler vendor_id :  encoder : Lavc58.134.100 mjpeg Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A frame= 3 fps=0.8 q=1.0 Lsize=N/A time=00:08:44.36 bitrate=N/A speed= 145x video:209kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
My intention was to extract these images in a single step, but if the filter solution proves to be unreliable, I'll stick to the
-ss [timestamp] option.
The question is, what causes FFmpeg to behave this way?
EDIT: There seems to be a correlation with the timestamps. When I drop the fractional parts of the timestamps, the command produces four (
4!!!) frames instead of three.
tvalues are generally unreliable with the
tdenotes the presentation timestamp i.e. the time at which the frame is expected to be painted. There is a frame duration field but effectively it is implicit and based upon when the next frame is painted.