FFMPEG is hanging on a single frame

Question

We're having a Node application where users can upload their own videos files. All videos are converted to mp4 files to make them available in all browsers, since some users may upload other video formats like webm or 3gp.

This is our FFMPEG command within our application:

this.inputFile = tmp.fileSync({dir:"/tmp/videos"});
this.outputFile = tmp.fileSync({dir:"/tmp/videos"});

new ffmpeg({timeout:600}) // 10 minutes timeout
        .input(this.inputFile.name)
        .input(__dirname + "/../logo.png")
        .complexFilter(['[0:v]scale=w=trunc(oh*a/2)*2:h=720[scaled]','[scaled][1:v]overlay=main_w-overlay_w:main_h-overlay_h'])
        .withVideoCodec('libx264')
        .withAudioCodec('aac')
        .toFormat('mp4')
        .audioBitrate('128k')
        .videoBitrate("1000k")
        .fps(24)
        .outputOptions(['-profile:v high', '-level 4.2', '-movflags +faststart','-preset ultrafast'])

        .on('error', (err,stdout,stderr) => {
            this.outputFile.removeCallback();
            this.inputFile.removeCallback();
        })

        .on('end', () => {

            //Cleaning input
            this.inputFile.removeCallback();

            var readStream = fs.createReadStream(this.outputFile.name);

            readStream.on('open', function () {
                readStream.pipe(writeStream);
            });

            readStream.on('close', () => {
                this.outputFile.removeCallback();
            });
        })

        .output(this.outputFile.name)
        .run();

Most of the time, the conversion process is working fine. But some conversions ran into a timeout. It seems like FFMPEG is hanging on a single frame and never finishes:

1|project- | Error: process ran into a timeout (600s)
1|project- |     at Timeout._onTimeout (/var/www/project-files/source/node_modules/fluent-ffmpeg/lib/processor.js:465:23)
1|project- |     at ontimeout (timers.js:365:14)
1|project- |     at tryOnTimeout (timers.js:237:5)
1|project- |     at Timer.listOnTimeout (timers.js:207:5)
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0311x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0311x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0311x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.031x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.031x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.031x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0309x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0309x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0309x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0309x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0308x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0308x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0308x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0307x    
1|project- | frame=  405 fps=0.7 q=32.0 size=    2272kB time=00:00:17.10 bitrate=1088.1kbits/s dup=233 drop=0 speed=0.0307x   

I've also noticed that this error happens on corrupted webm files, which seem to not have a correct timestamp. I've also tried to set shortest=1 to the overlay filter:

overlay=main_w-overlay_w:main_h-overlay_h:shortest=1

This solves the loop, but the converted mp4 file only shows a single frame and not more (it's "frozen").

Any idea what could cause this issue and how I could fix it?

// Edit: I'm now able to reproduce it. The problem occurs if the user is recording a video on his smartphone and changes from landscape<->portrait mode during recording.

This is the output of FFMPEG on my dev machine:

Past duration 0.679985 too large
Past duration 0.799995 too large
Past duration 0.719994 too large
Past duration 0.639992 too large
Past duration 0.759987 too large
Input stream #0:0 frame changed from size:640x480 fmt:yuv420p to size:480x640 fmt:yuv420p
frame=   33 fps=0.0 q=34.0 size=     173kB time=00:00:01.67 bitrate= 845.0kbits/s dup=2 drop=7 speed=3.33x    
frame=   33 fps= 33 q=34.0 size=     173kB time=00:00:01.67 bitrate= 845.0kbits/s dup=2 drop=7 speed=1.67x    
frame=   33 fps= 22 q=34.0 size=     173kB time=00:00:01.67 bitrate= 845.0kbits/s dup=2 drop=7 speed=1.11x    
frame=   33 fps= 16 q=34.0 size=     173kB time=00:00:01.67 bitrate= 845.0kbits/s dup=2 drop=7 speed=0.831x    
frame=   33 fps= 13 q=34.0 size=     173kB time=00:00:01.67 bitrate= 845.0kbits/s dup=2 drop=7 speed=0.665x   

Is there a way to use the overlay filter with it?

Answer 1

Adding -loop 1 to the FFMPEG command solves the problem:

new ffmpeg({timeout:600}) 
[...]
.inputOptions(["-loop 1"])
.run();

Answer 2

This problem comes from frame size changing. Scale your input videos to the same size first, then do the overlay stuff.