Recording and playing video with variable frame rate

Question

We are using a streaming camera to record and to play back surveillance video.

This camera sends a RTSP stream which will be recorded by our application as a H264 video.

In our application, the H264 videos will be stored in a MP4 container and will be published on a HTTP-server (Apache2). The stored videos can now be watched with any HTML5 web browser. So far, so good!

Our problem is, that the camera sends his frames with a variable frame rate. This is not a problem as long we watch the stream live. But when we watch the recorded H264/MP4 videos the time begins to drift. This means because of the variable frame rate, sometimes the video will be played too fast and the time is not correct!

So, does someone have experience with H264 videos with variable frame rate (VFR)? How can we play these VFR videos without time drift? Is it possible to convert VFR video to video with constant frame rate?

(All tested players like VLC seems to have the same problem...)

Here the output of mediainfo:

Complete name                            : VIDEO_REPLICA_12.h264
Format                                   : AVC
Format/Info                              : Advanced Video Codec
File size                                : 249 MiB
Overall bit rate mode                    : Variable

Video
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.2
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 1 frame
Format settings, GOP                     : M=1, N=15
Bit rate mode                            : Variable
Maximum bit rate                         : 20.0 Mbps
Width                                    : 1 280 pixels
Height                                   : 800 pixels
Display aspect ratio                     : 1.600
Frame rate mode                          : Variable
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive

Answer 1

Are you sure the files your recorded have indeed a variable framerate? It sounds like you recorded with a constant framerate (even though you get a variable frame rate video feed) which introduced this "artifact" and the players then of course play it back with a constant frame rate even though they shouldn't.

To properly convert variable frame rate footage into constant frame rate footage you can reside to Handbrake. It's an easy to use tool for h264 encoding that lets you choose to convert your video to a constant frame rate.

Answer 2

I recently ran into this issue...my transcoded-into-H264 videos started coming out with VARIABLE frame-rates, whereas in the past, they had come out as fixed framerate.

I use VLC-cmdline to do my final transcodes (because VLC supports features that HandBrake and others don't...such as marquee-text and font-color control, etc.

When I recently googled this issue, I noticed that wikipedia, and a few other comments on variable frame-rate mention that 'container-type' can make a difference on whether fixed or variable frames get generated. (That puzzled me...I never would have guessed that.)

But, sure enough ! In my VLC cmd-line, I changed my 'mux' from MP4 to MKV and voila. 'Mediainfo' checker shows clearly that I was getting variable when using MP4-container, and fixed frames with MKV.

[Great...I'd been planning to switch to MKV output files anyway, and this provided me with a great reason to do so! Be aware -- version 1.x and early VLC supported only MP4-containers, but v2.x now DOES also support MKV.]

Hope this helps others.

Answer 3

Transcode Your Video First You need to transcode your video first using a "real" encoder like Apple Compressor or Adobe Media Encoder.

Quicktime MP4 29.97 or 30 frames per second

Make sure you use a standard resolution like 640x360,720x405,1280x720,1920x1080 or 640x360 for standard def 4x3.

You could also try using HandBrake, it may very well do the trick for you and be a lot easier.