1

We recorded a video of a talk and its audio on separate devices (camcorder and phone). Both are relatively new an the camera indeed does nice full HD video. My problem is that they both seem out of sync when seen from within the editing progams.

I post the Mediainfo data of the video and audio below. Basically, when I import both the video and audio into Lightworks (or some other software), I can align the waveforms but they drift from each other. I have wondered whether this is due to different bitrates, but adjusting the speed does not seem to help. Is this our problem, editing program's limitations, the medium, file formats...?

Any hint on improving our workflow would be appreciated. Our recording equipment is a Canon Legria R68 and an Android (Moto G 3rd gen) mobile phone for the video and audio, respectively.

Video info (one chunck out of two)
General
ID                                       : 0 (0x0)
Format                                   : BDAV
Format/Info                              : Blu-ray Video
File size                                : 3.88 GiB
Duration                                 : 37 min 59 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 14.6 Mb/s
Maximum Overall bit rate                 : 18.0 Mb/s

Video
ID                                       : 4113 (0x1011)
Menu ID                                  : 1 (0x1)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 2 frames
Format settings, GOP                     : M=3, N=12
Codec ID                                 : 27
Duration                                 : 37 min 59 s
Bit rate mode                            : Variable
Bit rate                                 : 13.8 Mb/s
Maximum bit rate                         : 16.0 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan type, store method                  : Separated fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.266
Stream size                              : 3.66 GiB (94%)

Audio
ID                                       : 4352 (0x1100)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Mode extension                           : CM (complete main)
Format settings, Endianness              : Big
Codec ID                                 : 129
Duration                                 : 37 min 59 s
Bit rate mode                            : Constant
Bit rate                                 : 256 kb/s
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 spf)
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : -80 ms
Stream size                              : 69.6 MiB (2%)

----
Audio file info (it is longer because it covers the whole session)

Format                                   : MPEG-4
Format profile                           : 3GPP Media Release 4
Codec ID                                 : 3gp4 (isom/3gp4)
File size                                : 61.9 MiB
Duration                                 : 1 h 8 min
Overall bit rate mode                    : Constant
Overall bit rate                         : 126 kb/s
Encoded date                             : UTC 2016-11-10 18:27:57
Tagged date                              : UTC 2016-11-10 18:27:57

Audio
ID                                       : 1
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 1 h 8 min
Bit rate mode                            : Constant
Bit rate                                 : 128 kb/s
Channel(s)                               : 1 channel
Channel positions                        : Front: C
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 spf)
Compression mode                         : Lossy
Stream size                              : 61.2 MiB (99%)
Title                                    : SoundHandle
Language                                 : English
Encoded date                             : UTC 2016-11-10 18:27:57
Tagged date                              : UTC 2016-11-10 18:27:57
Improve this question
1
  • Your audio streams are at different sample rates. Now, a good NLE should resample all input audio to the project sampling rate but I don't know how Lightworks handles it. Also, were there any interruptions in recording, either in the video or audio?
    – Gyan
    Nov 24, 2016 at 11:38

4 Answers 4

Reset to default
3

This is a rather common problem. The issue is NOT sample-rate. And the issue is NOT frame-rate, either. Most modern editing software (or file-manipulation software) is capable of easily resolving sample-rate or frame-rate issues. For example, clearly there is no "frame-rate" or even "sample-rate" setting on a typical smart phone.

The issue is that the internal clock references in mass-market consumer gadgets like your camcorder and your phone are not precise enough to stay in lip-sync for more than a few minutes. For most of the history of shooting film and video, professionals actually connected the camera clock and the audio recorder clock together with a hard wire. (This was called "genlock".) And in more modern times, pro gear uses internal (or external) reference clocks that cost more than your whole camcorder and phone combined. And even then they are "jam-synced" together several times a day.

Depending on which editing software you are using, you will very likely find it much more acceptable to use the audio track as the "master reference" and "pull-up" the video to match the audio. Many editing applications do pretty horrible things to the audio track if you try to lengthen or shorten the clip slightly.

But doing the same thing to the video track will result only in a dropped or duplicated frame every few seconds, and almost nobody viewing your video will notice that. But they will certainly complain about lousy audio if you try to "tweak" the length.

But try it for yourself on your own software and see how it works for yourself. Only YOU can do that experiment with YOUR video and audio and editing system.

You may get lucky. Start by syncing the beginning of the video/audio together. Then go to the end and drag the length of the video clip so that the audio is in sync at the end. If you are lucky, the audio will be in sync through the entire length of the recording. But if there was more drift during the clip, you may have to divide your video into sections and "pull-up" the video to match the audio at several points.

We typically use "B-roll" (PowerPoint slides, presentation video, audience shots, etc.) to cover edit and/or sync points. It is always good to get some "audience reaction" shots to cover edit points. Even if you don't shoot the "B-roll" during the main presentation.

Improve this answer
3
  • Thanks, those are nice ideas. Indeed we are switching to having two video tracks: screencast of presentation and recording of speaker + audience. That should help mask all those technical problems... Pity that time/frequency standards are so crappy in phones (I myself work in metrology-related physics and am very surprised of that, hehe)
    – Juanjo
    Dec 6, 2016 at 10:20
  • I don't think it's because the internal clocks drift. A phone just would not be able to get GPS location or connect to the cell network if its clock was that bad. I think it's because they often record with variable frame rate.
    – stib
    Dec 6, 2016 at 11:36
  • There is a vast difference between the kind of clock drift that causes lip-sync problems vs. the kind that causes the "time of day" display to be incorrect on your screen. Even if the phone were actually phase-locked to the GPS reference (which no phone is), the camcorder is not referenced to GPS and so there will be DIFFERENTIAL DRIFT. GPS does NOT require local-clock stability to work. GPS works by comparing the signal coming from different satellites, it does not depend on any local clock reference. Neither the camcorder nor phone is running at "variable frame rate".
    – Richard Crowley
    Dec 6, 2016 at 12:31
0

Sample Rates are not the issue - you will find that one of the devices is running at a non-integer frame rate. This is your problem. When doing multi-camera shoots you have to ensure that both devices are running at the same frame rate or you will not be able to synch these up in post. Some NLE's will be able to conform your non-standard footage to the project frame rate, but looking at the frame-rates in the OP. these are very non-standard frame-rates. Try and run at 25FPS if at all possible across all devices. Then you have a good chance of being able to edit the footage.

Improve this answer
0

Usually when recording externally or with two cameras a clapper comes in handy. You match the spikes and then mute the other audio tracks so you just have one. I would expand the audio tracks and find similarities.

Improve this answer
0

If you're liquid enought to shell out 300.- then try https://www.redgiant.com/products/pluraleyes/

Since you synced the audio manually in front and it's deviating after several minutes, you need to squeeze one movie to fit.

  1. Have a clap at start of record and one at end of record
  2. align the first spike in audio
  3. count the samples / seconds/frames between last spike in first and last spike in second audio
  4. calculate difference in percentage
  5. squeeze clip by that percentage
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.