I'm new to video editing, and I need to sync up the audio and video from two different sources for a Virtual Reality game. Basically, I have the VR itself running in Unity on one computer, with the game output being screen captured (I have screen captures both from OBS and and in-game Unity screen capture). There is no audio output on this computer. All of the audio output happens in a program called Max/MSP on my MacBook, where the audio is being recorded internally. These two computers are linked over an ethernet cable, so there is essentially 0 latency. All of the audio was recorded at 44.1 KHz, and all of the video was recorded at 30 fps, with a constant frame rate.
When I pull up the footage in Premiere, there are two things that are confusing me.
The video files are shorter than the audio files by about 7% (though it varies, so it’s not a consistent error), even though everything lined up exactly when I played it in real time. This would lead me to believe it’s a frame rate or sample rate issue, however I recorded everything consistently at 44.1 and at a constant 30 FPS. Just to be sure, I converted the audio again and encoded the video at a constant frame rate with Handbrake, and still got the same results.
Even when I’ve slowed down the video to sync up with the audio at one point in the video, it’s still not synced at other points. If I choose a point to sync it, it will go out of sync usually within 10-30 seconds, though again this varies.
Inspecting the audio files individually, there is definitely no speeding up, slowing down or buffers being dropped; the audio was recorded correctly and I believe accurately represents the length of the take. This leads me to believe there was some sort of issue with the screen capture, maybe because my CPU and GPU usage were very high during the recording. Maybe some frames are being dropped (Which would make sense having the video being shorter), though I would think programs like premiere and final cut would account for that. Or it could be a Unity bug. Although again, I don’t think this is the case because I have screen captures with both Unity and OBS and both have issues. Beyond that I really have no clue.
The only solution I've been able to muster is adjusting the speed of the video, but since the audio goes out of sync every 20 or 30 seconds, I have to constantly clip the videos and readjust. The videos are 20-30 minutes in length, and I have about 30 to edit, so it would be a long and tedious process to do this for all of them (One video took me several hours).
I would be extremely grateful to hear any input on what the issue might be, and possible solutions. Unfortunately I'm not able to record any more gameplay, so I have to fix everything I have in post. Thank you!