MediaRecorder. As the name implies, this can be used to record the contents of a stream, both video and audio. It has annoying limitations though. Most importantly, you can't change streams while recording. If you attempt to replace the current audio stream with another, recording immediately stops. So I had to come up with a workaround, or the user can only show videos without sound.
MediaRecorder only seems to save into WebM with H.256 encoding and Opus audio. There are a few other codecs, but they all unreliable and often crash.
At this point I can record video from the camera, and there's no problem with superimposing short video clips over the camera image, but I can't record the audio of these files. I can't turn off the mic and switch to a new audio stream, and then back, or the API shuts down the recorder.
The solution I came up with is the following:
- Record the presentation.
- When the user shows a video clip, mark the timestamp, and record the clip's audio stream with a second
MediaRecorderinstance. (No, I can't take it from the original file, because the user may pause it, seek into it, stop it prematurely, etc.)
- When the user hides the video, save the timestamp again, and save the separate audio recording.
- After the user finishes recording the presentation, save the "main video".
- Using the saved timestamps, assemble an ffmpeg concatlist. Here's an example:
file video.webm inpoint 00:00:00.000 outpoint 00:00:16.105 file vhP9u_NDj0nEPS6EAtmCf.webm <-- yes, that's a random filename file video.webm inpoint 00:00:21.222 outpoint 00:00:23.688 file mjV66jcoSUUxTq1GbQItH.webm file video.webm inpoint 00:00:27.879 outpoint 00:00:31.142 file Ig55EcInUVC8uduqMye8U.webm file video.webm inpoint 00:00:35.863 outpoint 00:00:37.991
ffmpeg to put together a new video. Take the main video and the small audio files recorded on the way, and concatenate them into a new audio track:
ffmpeg -f concat -i concatenate.txt -vn -ac 2 concat_audio.webm
Then attach this new audio track to the main video, and save it as a new one:
ffmpeg -i video.webm -i concat_audio.webm -c copy video_final.webm
Sounds good - doesn't work. Looks like WebM doesn't like to be used like this!
The output video is almost always garbled in some way. Timings are off, often by several seconds.
At most times the concatenated audio is shorter than the actual video. This certainly isn't the fault of my timestamps, they're very accurate. (It's easy to verify if they are, as the inserted videos begin exactly at the saved times.)
I tried to convert the audio clips into
.wav and the video into
.avi before concatenation. In most cases, the video got messed up. In most such cases it pauses at a certain frame, and that's it.
So, my question is: Is there a better way to do this?
How can I make WebM files more "stable"? I know they're meant for streaming. What format and codec should I convert them into for the concatenation to work?
Can ffmpeg replace parts of the audio stream in a video, without having to chop up and reassemble the entire shebang?
Using a video editing tool to put the video together is out of question. It has to be done by the application, and without requiring immense knowledge of video formats and codecs.