Resynchronizing audio in a variable framerate video with missing timing data

Question

I was recording a moderately long (20-30 minute) video when suddenly my camera died, resulting in a corrupt file. I was able to repair the file by following the steps in How to open and repair an m4v or mp4 video file? on SuperUser, but now the audio in the repaired file is horribly out of sync with the video. (Presumably because the data keeping the video in sync was lost.)

The video seems to play at a variable speed, with some portions appearing sped up, and other portions appearing to play at a normal speed. The audio does not appear to have this issue, playing at a seemingly normal rate for the duration of the video.

I tried stretching the video to align two audio events with the video near the start and end of the recording, but because the playback speed of the video appears to be variable, the audio in the middle portion of the recording is still very much out of sync.

Is there a good way to fix this? Maybe a program or something that I can use to go through the recording and manually identify multiple "sync points" between the audio and video to ensure they stay in sync throughout the recording? I'm guessing the video speed will need to change dynamically throughout the recording to compensate for this sync problem.

I suppose I could try doing this completely manually with a video editor, but I imagine doing it that way (at least in Adobe Premiere Elements, which is what I'm using) would require me to make a cut every time the video speed is adjusted, and that sounds like it'd be extremely time consuming. Is there a better way?

Answer 1

No matter how you slice it, this sounds like it's going to be painful if not impossible, mostly because you say the rate changes dynamically. My best suggestion would be to use a software like Adobe Premiere Pro and its "rate stretch" tool to manually adjust individual segments. I'm not aware of any software that can do that sort of thing automatically, mostly because you don't have another (non-corrupted) audio track to compare against. If you did (like, if you had also been recording on an external recorder), you could use Adobe Audition to sync them.

Syncing two audio tracks of varying rates isn't too terribly hard, but syncing a video and audio track of different rates could prove very challenging.