OGV Format plays properly on my computer but transcoding drops (duplicate?) frames

Question

I've made a set of screencasts using recordmydesktop on ubuntu 12.10. The output is an ogv file. When I watch the ogv file using the default movie player (totem) it looks fine - the audio and video are in sync. When it is transcoded (by me or youtube) the audio and video are out of sync. It looks like I skip through a slide or two while narrating.

Update

I suspect the problem is more properly characterized as dropping duplicate frames during transcoding. Converting video where the mouse is moving seems to normally work fine. But when I'm just talking during a slide those duplicate frames are dropped.

I saw this but it isn't quite my situation (trying to go from ogv -> anything) https://superuser.com/questions/436187/ffmpeg-convert-video-w-dropped-frames-out-of-sync

AVI files seem to translate correctly! I assume this will be a big hint to someone. I'd still like to track down the underlying issue. I'm testing the conversion of my previous videos to AVI but this takes a while since I have to check each transition.

Examples

This is the original OGV file from gtk-recordmydesktop: http://dl.dropbox.com/u/64693533/sync_test/sync_test1.ogv

The video starts with a slide for 10s, then advances to 3 more slides 5s each. Each time I advance slides I tap the microphone too (10s, 15s, 20s, 25s).

Here are some conversions that were done (each displays it's own video timing issues):

http://dl.dropbox.com/u/64693533/sync_test/sync_test1.mp4

• this one shows the first slide in the first frame but quickly advances past it
• this was done using the stock ffmpeg

http://dl.dropbox.com/u/64693533/sync_test/sync_test1.ffmpeg-static.mp4

• this one is pretty close - for some reason at 13s it decides to advance though
• this was done using the static build of ffmpeg from a few days ago

Here it is on youtube - you can see that at around 13s it advances early (from slide 1 -> slide 2):

Here is proof the OGV file works correctly:

ffmpeg translation

Using ffmpeg or avconv I seem to get similar results as youtube (transitions appear to happen early but not necessarily at the same time).

Here is the command I use (with a recent static build of ffmpeg) and output:

$ ~/ffmpeg/ffmpeg -i JSP.ogv JSP.mp4
ffmpeg version N-50025-gb8bb661 Copyright (c) 2000-2013 the FFmpeg developers
  built on Feb 17 2013 05:23:03 with gcc 4.6 (Debian 4.6.3-1)
  configuration: --prefix=/root/ffmpeg-static/64bit --extra-cflags='-I/root/ffmpeg-static/64bit/include -static' --extra-ldflags='-L/root/ffmpeg-static/64bit/lib -static' --extra-libs='-lxml2 -lexpat -lfreetype' --enable-static --disable-shared --disable-ffserver --disable-doc --enable-bzlib --enable-zlib --enable-postproc --enable-runtime-cpudetect --enable-libx264 --enable-gpl --enable-libtheora --enable-libvorbis --enable-libmp3lame --enable-gray --enable-libass --enable-libfreetype --enable-libopenjpeg --enable-libspeex --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-version3 --enable-libvpx
  libavutil      52. 17.101 / 52. 17.101
  libavcodec     54. 91.103 / 54. 91.103
  libavformat    54. 63.100 / 54. 63.100
  libavdevice    54.  3.103 / 54.  3.103
  libavfilter     3. 38.100 /  3. 38.100
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[ogg @ 0x34d4640] Multiple fisbone for the same stream is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[ogg @ 0x34d4640] Header parsing failed for stream 0
[ogg @ 0x34d4640] Broken file, keyframe not correctly marked.
Input #0, ogg, from 'JSP.ogv':
  Duration: 00:12:49.67, start: 0.000000, bitrate: 224 kb/s
    Stream #0:0: Data: none
    Stream #0:1: Video: theora, yuv420p, 1600x880 [SAR 1:1 DAR 20:11], 15 fps, 15 tbr, 15 tbn, 15 tbc
    Metadata:
      RECORDMYDESKTOP : 0.3.8.1
    Stream #0:2: Audio: vorbis, 22050 Hz, mono, fltp, 89 kb/s
[libx264 @ 0x369c5e0] using SAR=1/1
[libx264 @ 0x369c5e0] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2 AVX
[libx264 @ 0x369c5e0] profile High, level 4.0
[libx264 @ 0x369c5e0] 264 - core 129 r2230 1cffe9f - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'JSP.mp4':
  Metadata:
    encoder         : Lavf54.63.100
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1600x880 [SAR 1:1 DAR 20:11], q=-1--1, 15360 tbn, 15 tbc
    Metadata:
      RECORDMYDESKTOP : 0.3.8.1
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 22050 Hz, mono, s16, 128 kb/s
Stream mapping:
  Stream #0:1 -> #0:0 (theora -> libx264)
  Stream #0:2 -> #0:1 (vorbis -> libvo_aacenc)
Press [q] to stop, [?] for help
[ogg @ 0x34d4640] Broken file, non-keyframe not correctly marked.
    Last message repeated 2 times
Broken file, non-keyframe not correctly marked.=00:00:08.37 bitrate=  28.7kbits/s dup=66 drop=0    
Broken file, keyframe not correctly marked.time=00:00:51.01 bitrate= 125.3kbits/s dup=675 drop=0    
Broken file, keyframe not correctly marked.time=00:00:55.05 bitrate= 140.2kbits/s dup=782 drop=0    
Broken file, keyframe not correctly marked.time=00:00:59.60 bitrate= 140.5kbits/s dup=836 drop=0    
[ogg @ 0x34d4640] Broken file, keyframe not correctly marked.
Broken file, keyframe not correctly marked.time=00:01:08.00 bitrate= 143.0kbits/s dup=900 drop=0    
Broken file, keyframe not correctly marked.time=00:01:11.86 bitrate= 141.6kbits/s dup=910 drop=0    

...repeated many times...

Broken file, keyframe not correctly marked.time=00:12:47.62 bitrate= 153.0kbits/s dup=9087 drop=0    
frame=11521 fps= 87 q=-1.0 Lsize=   14849kB time=00:12:49.48 bitrate= 158.1kbits/s dup=9087 drop=0    
video:2401kB audio:12024kB subtitle:0 global headers:0kB muxing overhead 2.938094%
[libx264 @ 0x369c5e0] frame I:49    Avg QP:16.05  size: 29658
[libx264 @ 0x369c5e0] frame P:2912  Avg QP: 9.88  size:   114
[libx264 @ 0x369c5e0] frame B:8560  Avg QP:12.76  size:    78
[libx264 @ 0x369c5e0] consecutive B-frames:  0.9%  0.1%  0.2% 98.9%
[libx264 @ 0x369c5e0] mb I  I16..4: 90.8%  0.4%  8.8%
[libx264 @ 0x369c5e0] mb P  I16..4:  0.0%  0.0%  0.0%  P16..4:  0.0%  0.0%  0.0%  0.0%  0.0%    skip:99.9%
[libx264 @ 0x369c5e0] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.3%  0.0%  0.0%  direct: 0.0%  skip:99.7%  L0:65.3% L1:34.6% BI: 0.1%
[libx264 @ 0x369c5e0] 8x8 transform intra:0.5% inter:15.8%
[libx264 @ 0x369c5e0] coded y,uvDC,uvAC intra: 6.4% 0.1% 0.1% inter: 0.0% 0.0% 0.0%
[libx264 @ 0x369c5e0] i16 v,h,dc,p: 94%  4%  2%  0%
[libx264 @ 0x369c5e0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 22% 44%  1%  2%  2%  3%  1%  6%
[libx264 @ 0x369c5e0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 17% 19%  4%  5%  5%  5%  5%  5%
[libx264 @ 0x369c5e0] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 0x369c5e0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x369c5e0] ref P L0: 82.5%  1.4% 11.9%  4.3%
[libx264 @ 0x369c5e0] ref B L0: 47.2% 52.4%  0.4%
[libx264 @ 0x369c5e0] ref B L1: 99.2%  0.8%
[libx264 @ 0x369c5e0] kb/s:25.60

The video still advances early but at different times. It sounds like gtk-recordmydesktop is generating a "broken file". What's annoying is that the OGV does work so it seems like I should be able to make this work with some set of options.

I have found that I can render the video in kdenlive and it seems to be working there. I'd still like to know what is going on. kdenlive does a much better job, but it still advances early sometimes.

Answer 1

Why convert to OGV when your final upload is going to be to youtube, I may be wrong but you can convert to x264 video codec with AAC Audio even on linux and upload that to youtube considering that is what they prefer to be uploaded anyway. Have you tried making an h264 and uploading to youtube instead of the OGV file and seeing if that was the issue. Because I would bet that if that solves it, you then know it was an issue with the OGV being uploaded to youtube, and if it does not solve it, it could be a frame-rate issue with youtube's interpretation or something similar.

There have been a lot of issues with OGV files uploaded to youtube in the past. I cant imagine that it is 100% fixed even at this point.

http://support.google.com/youtube/bin/answer.py?hl=en&answer=1722171

EDIT: also just noticed that your original footage is at 15fps...this could very well be the source of the issue

EDIT 2: I seemed to slightly have misread the question a little bit...being that you are starting with a video file that is OGV, and I saw that you are going to MP4...this changes things a little bit...but I am going to guess it has something to do with the 15fps and 22050 Hz audio...I know the sample rate has nothing to do with syncing the audio, but from experience when using non standard frame-rates and audio samplerates, I have tended to see drifting...getting these to sync may be rather difficult while not being able to edit these after the initial recording with a cheap video editor...

While software has gotten better about drifting audio it is still a common problem when using uncommon framerates and samplerates as the keyframed sync points are not standard and could be rounding key frames etc...

You see where it says "Broken file, keyframe not correctly marked." that is what it is referring to...

my advice for you would be to get it as close as possible, take it into a video editor and slip and cut up the audio to get it to match the way you want it. Unfortunately sometimes this is how it is fixed...

Software based transcoders do not always work as they should...hense why a protools setup and/or an avid setup would come with hardware to further ensure sync capabilities and constant frame-rates etc...

Another thing you could try is to convert the footage to a standard framerate, and try to re-mary the audio...as I am pretty sure it is the video drifting...probably ever so slightly slowing down and then speeding up towards the end or vice versa.

EDIT: I was able to get the video to sync to the original using this ffmpeg command...it may have need the rate clause which is what I suspected

ffmpeg -i sync_test1.ogv -strict experimental -pix_fmt yuv420p -r 15 -vcodec h264 -acodec aac sync_test1.mp4

Answer 2

I struggled with a similar issue on Ubuntu 12.04.3 LTS. I fixed the problem using the static ffmpeg build which is available from http://johnvansickle.com/ffmpeg/

Answer 3

It seems this bug is still in recordmydesktop, or at least I got the exact same symptoms here with the version from Debian testing (0.3.8.1+svn602-1.1).

I just managed to transcode this video almost correctly, in the following way:

1. Use VLC.
2. Ask it to transcode to the WebM "profile".
3. Ask it to display the video as it is transcoded.

The result still has the correct synchronization (yay!), but it still suffers from a problem: the resulting video announces itself as lasting more than 4h whereas my video only lasts 25min. Peertube hence lists it as a 4h video, tho once you start playing it the correct 25min duration is displayed, so the problem is not too serious.

Point 3 is important, but point 2 is also: maybe other profiles will also work, but at least the H264 profile gave me an unusable video (either with bad sync like ffmpeg or worse).

Answer 4

Try just changing the container to avi rather than transcoding, which seems to work better for youtube:

ffmpeg -i JSP.ogv -vcodec copy -acodec copy JSP.avi