Adding many sounds to a video file at specific frames

Question

I have a video of a TAS I've created that unfortunately does not contain sound. I would like to add sound to the video before releasing it, but I obviously don't want to have to go through and place hundreds or thousands of sounds manually.

An example of a finished product, with all sounds that are present having been added manually, can be seen here:

I can modify the source code of the game, so a list of exact frames that sounds are played on is trivial to create. I'm currently using Adobe Premiere Pro CC 2015, but I'm happy to switch to different software if it provides the functionality I want.

FFmpeg looks like it might be able to do what I want, but I don't know enough about using it and haven't been able to find any examples of using it for what I want to do. I'd also rather not create a plugin (and haven't been able to find any information on scripting) for Premiere or anything like that, though I'd prefer that to doing it manually.

Thanks in advance!

Answer 1

Alright, more googling found that ffmpeg is definitely the solution to my issue!

ffmpeg -i inputvideo.mp4 -i inputsound.wav -filter_complex "[1:0]adelay=leftchanneldelay|rightchanneldelay[delayed];[0:1][delayed]amix[mixout]" -map 0:v -map [mixout] -c:v copy -c:a aac outputvideo.mp4

This line allows me to easily determine where in the video (down to the millisecond, which is more than precise enough with each frame equaling 20 milliseconds) I wish to place the sound; wrap it in a loop inside a batch file (or whichever method you prefer) and off you go!

Edit: A more fully-featured solution written in Java, which only creates the audio file (to allow you to add it manually):

package javaapplication4;

import java.io.IOException;

public class JavaApplication4 {
static String delay;

public static void main(String[] args) throws IOException, InterruptedException {
    String a = "0";
    String c = "1";
    String temp = "";
    String basePath = "C:\\Users\\Desktop\\Stuff\\FFMpeg\\";
    String fileIn = "blockbreak";
    int[] delays = {2740,5760,51381,61040,84520,109600,113660,156640,311260,312820,314100,443800,735120,740540};
    Runtime.getRuntime().exec(basePath + "ffmpeg.exe -y -i " + basePath + fileIn +".flac -filter_complex \"[0:0]adelay=" + delays[0] + "[mixout]\" -map [mixout] -c:a flac " + basePath + "output0.flac").waitFor();
    for (int i = 1; i <= delays.length - 1; i++) {
        delay = "" + delays[i];
        System.out.println("" + (i) + "/" + (delays.length - 1));
        Runtime.getRuntime().exec(basePath + "ffmpeg.exe -y -i " + basePath + "output" + a + ".flac -i " + basePath + fileIn +".flac -filter_complex \"[1:0]adelay=" + delay + "[delayed];[delayed][0:0]amix=inputs=2:duration=longest[mixin];[mixin]volume=6.0201dB[mixout]\" -map [mixout] -c:a flac " + basePath + "output" + c + ".flac").waitFor();
        temp = a;
        a = c;
        c = temp;
    }

}

}

I increase the volume of the sound by 6.021db each time as ffmpeg seems to lower the volume when encoding; this isn't a perfect number, but it's close enough that the difference in volume between the first and last sounds in a file that's a few thousand sounds long ends up being around 1 db.