Syncing hard-coded text with audio

Question

Perhaps too basic for this forum... but it might help someone in the future.

I want to make translated lyric videos, with the lyrics hardcoded onto the video (English on top, my translation in my native language below). The opposite of fancy, just plain text with no effects.

I'm struggling with ensuring that each lyric appears on the exact moment as the audio.

Is there a program or a tool who finds a way to automatically sync text with the audio? Or detects when a new "line" begins? It would be enough to get a txt. file with the timestamps. For some reason I feel the need to be precise by the millisecond, so eye-balling it isn't doing it for me.

Answer 1

This will certainly vary depending on the type of music. If there's not excessive compression, and the instrumentation is quite a bit lower volume than the singer (such as a jazz crooner song, e.g. "The Way You Look Tonight"), you might have some luck with putting Audacity and FFmpeg to work.

• As long as you've installed FFmpeg for Audacity (Edit -> Preferences: Libraries -> FFmpeg Library: Download), you can use "File -> Import -> Audio..." to import audio direct from a video. (NOTE: FFmpeg for Audacity is a separate install from the command line tool FFmpeg.)
• Then, you can select all the audio and choose "Analyze -> Label Sounds". Experiment with the threshold level to make the line detection more accurate. -30dB is much more sensitive than -10dB.
• CTRL+B can manually establish any additional new labels at the cursor position on the timeline.
• Then, File -> Export -> Export Labels... will give you a text file with the start/end times and the labels you chose for the sounds.

From there, you might have either a little bit of grunt work translating the seconds to the hh:mm:ss,mmm style of timestamp required by, for example, the .SRT format for subtitles. Or hopefully you have a bit of scripting or reg-ex replace knowhow :-) to automate that process. Then, mix it in. For example, I use this command to mix in subtitles onto a video:

ffmpeg -i in.avi -vf subtitles=lyrics.srt -max_muxing_queue_size 9999 CONCERT.AVI

The -max_muxing_queue_size 9999 is related to an obscure problem with ffmpeg that may have been fixed since I last used this feature.

Of course, it will also vary by the tools you use. It will help to know your environment and favored toolchain. Good luck!