FFmpeg raspberry pi OMX hardware encoder with inline_headers enabled = Increasing latency over the time

Question

I managed to enable inline header on the OMX H.264 hardware encoder. inline header(SPS/PPS) are mandatory to play an already launched stream.

https://github.com/Serveurperso/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef

Yes but there is a problem : the latency grow over the time. There is a strange time calculation problem inside the muxer? I get a lot of traces "application provided invalid, non monotonically increasing dts to muxer in stream X/0" and the ffmpeg process always freeze at 3335/0 (with no trace).

Without inline header : no latency (about 0.2s on raspberry pi 4 / 720x576 25FPS) No "... increasing dts to muxer" error. disadvantage: I can't resume the already launched stream.

With software x264 encoder : no latency (about 0.2s on raspberry pi 4 / 720x576 25FPS) No "... increasing dts to muxer" error. disadvantage: It consume a lot of CPU power...

How to fix the timestamp(?) calculation inside ffmpeg muxer code to take into account the addition of SPS / PPS NALUs ?

Answer 1

Gyan is right :

I just need to add +global header flag and only the "dump_extra" bit stream filter

My patch become useless, ffmpeg can add SPS/PPS itself inside the stream.

My old cmdline is - missing inline SPS/PPS for infinite streaming - :

ffmpeg -f v4l2 -i /dev/video0 -vf hqdn3d=0:0:50:50 -c:v h264_omx -profile:v baseline -b:v 1500000 -f rawvideo tcp://127.0.0.1:8003

I just need to add "-flags:v +global_header -bsf:v dump_extra" the cmdline become :

ffmpeg -f v4l2 -i /dev/video0 -vf hqdn3d=0:0:50:50 -c:v h264_omx -profile:v baseline -b:v 1500000 -flags:v +global_header -bsf:v dump_extra -f rawvideo tcp://127.0.0.1:8003

(live result here : https://www.vigibot.com/ click on the "Astro" public robot)