ffmpeg thread_queue_size warning: What does this mean?

Question

*Edit: changing the render to -preset veryfast will no longer display the warning message. I also noticed that the video itself for the versions where the warning displayed had the last few minutes cutoff on both the audio and the video. I'm quite lost as to why this is.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Thread message queue blocking; consider raising the thread_queue_size option 
  (current value: 8)

I'm trying to find information on this warning message and what it means. The information I found online is indicating that not increasing my thread_queue_size can create artifacts and sync issues..? However the postings are many years old, and I'm unsure if they are true or still apply etc. They also apply towards screencasting specifically, which I am not knowingly doing.

The command I use that prompts this message is as follows:

ffmpeg -framerate 500/21 -start_number 1518 -i %d.png -i audio.ogg -c:a copy -vf "scale=3840:-1" -c:v libx265 -preset fast -crf 19 -pix_fmt yuv422p10le output.ts

So basically I'm turning thousands upon thousands of 1080p png into a 10 bit 4k h.265 .ts video. Whether I use a preset or not, or do audio separately, the warning still displays.

While I'm doing my command ffmpeg uses 90%-100% of my cpu and 80%-90% of my ram. I want to know if my render is suffering in some way by ignoring the warning message. The stuff I read online was concerning. For the sake of argument I'll link the info I found, just note the language is nsfw.

I will add that my content is just over 30 minutes long. Also I noticed that after rendering it out and lining it up with the original content in my Video editor, that there is a slight audio echo. Meaning the audio does not sync 100% correctly as it should. It's irrelevant for my video in the grand scheme, but I'd like to know if its related in any way as the linked info suggests. I'd also like to know what I can/should do to address the warning, if anything.

s speed = 0.117x
using preset fast

I came across information in my search that said speed needs to be 1.0 or faster...? I don't know if that was related to a specific situation in the post read, or if there's something I never knew about happening here. Almost all my renders occur under 1.0x speed unless I'm rewrapping/copying streams.

Answer 1

The "Thread message queue blocking" message is an indicator that your computer cannot keep up with encoding in real-time. It's recommending the "thread_queue_size" parameter because it thinks you want to process data in real-time (based on your command). By setting "thread_queue_size" to a higher value, you are increasing the buffer size for packets which are waiting to be encoded. If you exceed the packet buffer size (i.e., buffer gets full), ffmpeg will start to drop packets to keep up in real-time; increasing the buffer will hopefully allow your computer to process the packets before it has to drop them... But that won't work if you eventually run out of memory or exceed the new buffer size anyway since you are processing in less than real-time (less than 1.0x speed).

With your command, the reason ffmpeg is trying to process in realtime is because you are using -framerate 500/21 as an input instead of your desired output fps. So basically, ffmpeg is trying to encode 500/21 worth of images per second, which your computer can't keep up with when using the fast preset. Using -preset veryfast lowers the output quality, and your computer can apparently keep up with that, but I doubt that's what you want...

Since you really don't need to process in real-time (you're using a file-based source and not something live), you should use -r for the output instead of using framerate on the input.

ffmpeg -start_number 1518 -i %d.png -i audio.ogg -c:a copy -vf "scale=3840:-1" -c:v libx265 -preset fast -crf 19 -pix_fmt yuv422p10le -r 500/21  output.ts

Answer 2

I first came across this message today when trying to pipe curl output (inside a while loop) as input to ffmpeg and then remux to mkv. The process was much more time-consuming than downloading all the files locally first and then concating and remuxing them with ffmpeg afterwards. I believe this message indicates how many bytes of the input file ffmpeg is trying to read at a time. In my case the message was

[mpegts @ 0x55c7f9fe7a00] Thread message queue blocking; consider raising the thread_queue_size option (current value: 1)

which would explain the delay in the process. After putting -thread_queue_size 4096 before the input, the message disappeared and the process was as fast as it should.