Resuming a partially-completed encode with FFmpeg

Question

Summary

Given a partially completed encode that:

• Was encoded using h264 in the Matroska container
• Was cut short by manually stopping FFmpeg by typing 'q' in the console

...what are the ways to resume encoding this file, preferably by appending the extant file?

Details

I have some large encoding projects that I do that can take up to a week of encoding in some cases. In ideal circumstances, the computer doing the encoding can stay on, uninterrupted and left to do the work. In realistic situations though, sometimes that computer needs to stop encoding and be shut off or restarted for a multitude of reasons. Because of this, I'm looking for a way to make encoding using FFmpeg on that computer more "bullet-proof" by anticipating possible interruptions and having a way to use FFmpeg by itself to pick up where it left off later.

What I'm Doing Now

When this happens, I recover what I can of the encoded file, examine it in video editing software, find a good splicing point, then use AviSynth to create a script that will start encoding a new segment where the previous one left off. I then use other software to trim the existing encode so that the splice will be seamless, then use FFmpeg's demuxer to combine the individual h264 segments later into a completed file.

This is all a lot of manual work that can be very slow that I'd like to avoid if there exists a more automated solution.

Ideas

Segmenting: This seems like the most promising solution, but there are gaps I'm not sure how to fill. This answer seems to cover a lot of it. You can tell FFmpeg to output segments, broken up at I-Frames at rough intervals, that can be remuxed later. What I don't know from looking at that answer and at FFmpeg's documentation is how I would start encoding again at a specific segment. It seems like there might be a way to give FFmpeg a file that tells it where each segment should start via frame or timecode, but how do I make it start somewhere in the middle of that list?

This option is not ideal, however, in that I will need to effectively have double the space of the resultant encode (which will frequently be in the 100s of gigabytes in size) available since I will need to mux the segments into a single file. It would be nice if I could somehow cleanly cut off an ongoing encode and have FFmpeg reopen that file and continue where it left off.

Virtual Machines: This has the potential to work since I could just freeze the machine's state before a restart, but all of my encoding space exists on network-attached storage, which I'm not sure will work in the context of a virtual machine being suspended and resumed, since the VM doesn't actually control the drives.

Answer 1

I've come up with a "good enough" solution to this problem, as it doesn't appear possible at the time of writing to actually pick up a partial encode that was cut off arbitrarily and accurately concatenate the rest of the video to it.

A Solution Using Segmenting

First, use FFmpeg's segmenting functionality to output segments at some interval for which losing such an interval is an acceptable loss. Also output a .csv file with segment start and end times. An example command that achieves this is:

ffmpeg.exe -i .\source.avi -c:v libx264 -crf 20 -c:a aac -pix_fmt yuv420p -f segment -segment_time 30 -segment_list segments.csv -reset_timestamps 1 seg%03d.mkv

Assume that during the encoding of the fourth segment (seg003.mkv), it becomes necessary to stop the encoder and reset the machine. Sending a q keystroke to FFMpeg during the encoding of a segment stops it arbitrarily at some point and leaves that segment incomplete.

The segment that was interrupted is effectively lost and can be deleted. Opening the segments.csv file reveals

seg000.mkv,0.000000,33.390000
seg001.mkv,33.390000,62.586000
seg002.mkv,62.586000,91.782000
seg003.mkv,91.782000,98.655000

Using this information, we know that seg003.mkv started at 91.782 seconds into the source. In order to pick up where we left off, we use the same command as above to encode segmented output, but also seek (-ss) to the start of our fourth segment, and change the numbering (-segment_start_number):

ffmpeg.exe -ss 91.782000 -i source.avi -c:v libx264 -crf 20 -c:a aac -pix_fmt yuv420p -f segment -segment_time 30 -segment_list segments2.csv -segment_start_number 3 -reset_timestamps 1 seg%03d.mkv

FFmpeg will pick up at the beginning of the fourth segment, and correctly name it seg003.mkv. Notice that the name of the output .csv file has been changed so that the original does not get overwritten. This new .csv file will start at 0.00, so further interruptions will require adding times in this file to the previous file to find a starting point.

Finally, when all segments are ready to be concatenated, we construct an .ffcat file as outlined in the examples on the wiki page:

file seg000.mkv
file seg001.mkv
file seg002.mkv
file seg003.mkv
file seg004.mkv
file seg005.mkv

and run this command:

ffmpeg.exe -f concat -i .\segments.ffcat -c copy output.mkv

This outputs the full, concatenated encode. The option -fflags +genpts is mentioned in the linked answer of the question, and may be necessary to achieve perfect output, but I have not found it to make any difference in my testing.

Answer 2

You could run ffmpeg in a KVM Virtual Machine, which should run nearly as fast native hardware. Then you could pause, snapshot, and resume the VM as necessary.