1

I want to produce a virtual choir recording of an amateur choir. I want to give 80-120 members of the choir an accompaniment/click track of some kind. They will record themselves, on their mobile phones, singing along to the accompaniment track. The accompaniment track will play on a second device (say, a laptop), via headphones or ear buds. The send their recordings to me, and I merge them.

How can I synchronise the resulting 80-120 amateur-captured recordings, in an automated way that scales to 120 recordings, with a low budget? What sync aid do I put in the accompaniment track? What do the members need to do as they record themselves? What do I need to do in post-production to accomplish the sync and merge the audio?

I am quite comfortable scripting a tool like ffmpeg or Audacity to align and merge the recordings. I can even write a Python program to read through the samples of an audio recording to detect the audio peak of a clapper, or DTMF tones. What I want to avoid is any task that I have to do manually for each of the contributed recordings. I fear that I won't be able to get through manual tasks 120 times.

I suspect that a challenge for this synchronisation task is that the contributed recordings will be done with a variety of devices, at a variety of resolutions and sample rates. The choir members need to record themselves. They are not sound engineers. They will make some mistakes in production. I am looking for a sync method which is robust in the face of this variation.

I want to focus on audio merge at first. I expect that the contributed recordings will have video as well. Composing all those videos into a grid, maybe with attractive panning or video effects, is nice to have but not essential.

Let's assume for the sake of argument that the contributed videos will be 3-10 minutes long. It would be nice to stretch to 45 minutes, eventually, but not at first.

I don't know how precise the sync needs to be. As a guess, I imagine a variability of at most 0.1 seconds might be enough. 1 second probably is not good enough. 0.001 seconds is probably more precise than we need.

1

At the start of the accompaniment track, about 15 seconds before the music starts, record a clap, or other loud but brief sound. Have your choir members record this clap sound from the accompaniment track onto their contributed recording, but not record the rest of the accompaniment.

You can do this by having the singers put on their headphones, but unplug the jack from the playback computer. They start recording on their phone, then start playing back the accompaniment track. The sound of the clap will be played out loud, and captured on their recording, because the headphones are unplugged. Immediate after the clap, the singer plugs in their headphones. This stops any more of the accompaniment from being audible on the recording. (Important detail: experiment with how much time you need to leave between the clap and the start of the accompaniment, so that singers can plug in their headphones and get ready to sing.)

When the singers finish the piece, they stop recording, and send you their recordings. You need some system for naming the audio files, and tracking the identity and vocal part of each singer. (The details of this tracking are not germane to the sync question.)

Make a copy of the first few seconds of each contributed recording in .WAV format. Write a Python program to detect when the clap occurs relative to the start of a recording. Do this by reading the value of each audio sample of the .WAV file, and identifying the sample with the largest value. This is the moment of loudest sound. Convert the position of this largest value into an offset of seconds from the start of that recording.

Now, merge the recordings, applying an offset to each so that the clap occurs at the same moment for all all recordings. The recordings should now be in sync. Mix the recordings to taste, and save the mixed recording.

[Note: this is a speculative answer, based on reading Virtual Choir by Jed Scott, and CB Multitrack Editing 101 by Christopher Bill. Most of the virtual choir instructions I found on the web did not ask the contributors to record any sync moment at all. I guess they just have a human align all the parts by ear.]

| improve this answer | |
  • I wrote my own answer, but I don't think it is very good. I'm hoping it will encourage someone with a better solution to write it as an answer, so that I can learn from it and upvote it. – Jim DeLaHunt May 14 at 23:46
  • this is uncanny - I am currently attempting exactly the same thing! I had thought of something similar to what you're suggesting above, but it does sound a little cumbersome asking everyone to unplug/plug their headphones. – Bill Peet May 15 at 4:28
  • 1
    Another thing I'd thought of was having a piece of software that simultaneously played the backing track and recorded the video/audio - obviously it would start both together, so all tracks would be automatically aligned. I haven't found any software that does this however, at least not a simple/free one that could be distributed to a large number of people. – Bill Peet May 15 at 4:31
  • Would be very glad to hear if you come up with anything else! – Bill Peet May 15 at 4:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.