I observed a strange behavior while using the select
filter. In short, I'm using ffmpeg to detect frozen sections in a video, calculate the exact middle point of these sections, then using filters to extract image snapshots at these timestamps.
The frames are selected with the "select" filter by comparing the timestamps with the desired position: select=eq(t\,TIMESTAMP1)[+eq(...)]
, which is supplemented with the -vsync 0
option to drop other frames.
As you can see in the last two lines of the console output, the filter returned only 3 frames out of 6:
/opt/ffmpeg/ffmpeg-4.4-amd64-static/ffmpeg -y -hide_banner -i "/tmp/in.mp4" -map 0:v:0 -an -vf 'select=eq(t\,30)+eq(t\,76.84)+eq(t\,104.32)+eq(t\,150.32)+eq(t\,290.6)+eq(t\,524.32)' -q:v 1 -vsync 0 -f image2 /tmp/%06d.jpg
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/in.mp4':
Metadata:
major_brand : M4V
minor_version : 1
compatible_brands: M4V mp42isom
creation_time : 2014-10-02T11:59:55.000000Z
Duration: 00:08:47.92, start: 0.000000, bitrate: 373 kb/s
Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv), 1024x768 [SAR 1:1 DAR 4:3], 371 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
Metadata:
creation_time : 2014-10-02T11:59:55.000000Z
handler_name : Mainconcept MP4 Video Media Handler
vendor_id : [0][0][0][0]
encoder : AVC Coding
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x5556655cc480] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/tmp/%06d.jpg':
Metadata:
major_brand : M4V
minor_version : 1
compatible_brands: M4V mp42isom
encoder : Lavf58.76.100
Stream #0:0(eng): Video: mjpeg, yuvj420p(pc, progressive), 1024x768 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn (default)
Metadata:
creation_time : 2014-10-02T11:59:55.000000Z
handler_name : Mainconcept MP4 Video Media Handler
vendor_id : [0][0][0][0]
encoder : Lavc58.134.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame= 3 fps=0.8 q=1.0 Lsize=N/A time=00:08:44.36 bitrate=N/A speed= 145x
video:209kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
My intention was to extract these images in a single step, but if the filter solution proves to be unreliable, I'll stick to the -ss [timestamp]
option.
The question is, what causes FFmpeg to behave this way?
EDIT: There seems to be a correlation with the timestamps. When I drop the fractional parts of the timestamps, the command produces four (4!!!
) frames instead of three.
t
values are generally unreliable with theselect
filter.t
denotes the presentation timestamp i.e. the time at which the frame is expected to be painted. There is a frame duration field but effectively it is implicit and based upon when the next frame is painted.