ffmpeg to convert and merge 3 videos to a single widescreen video

Question

I have three .mov files with video in Apple ProRes (apcn) and audio in PCM S24 LE (in24), of identical length and dimensions (1920x1080), frame rate (23.976024) and each with the same audio track. I'd like to:

• convert to H.264 video and AAC audio, retaining very good quality (for public exhibition), and

• generate a single widescreen video in H.264/AAC, i.e. it'll be 1920px times 3 wide and 1080px high.

For generating the widescreen I've worked out I can use this command assuming I've prepared the individual mp4 files:

ffmpeg -i left.mp4 -i centre.mp4 -i right.mp4 
   -filter_complex "[0:v:0][1:v:0][2:v:0]hstack=inputs=3" triple.mp4

Main Question:

Should I do the process in two steps -- (a) convert files, (b) make widescreen file -- or does that mean the widescreen video is re-encoded therefore it's worse quality? I'd like to keep the separate files so if there's no other impact then I'd prefer to do it as two steps. If I should do the conversion to H.264 at the same time as making the widescreen how do I combine the different commands?

Sub-Questions:

1. When preparing the widescreen I came across an alternative method using overlay like below. Is there any advantage either way? It seems like the overlay should be slower but in testing seemed about the same.

   ffmpeg -i left.mp4 -i centre.mp4 -i right.mp4 -filter_complex "[0:v:0]pad=iw*3:ih[bg]; [bg][1:v:0]overlay=w[leftcentre]; [leftcentre][2:v:0]overlay=w*2" triple.mp4

2. When preparing the widescreen, I think ffmpeg chooses the audio from just one of the input files. Since the audio is the same on each it doesn't matter to me which one. Is that right or is it in some way combining them?

3. Am I right in thinking a term like [1:v:0] in the filter_complex expression means 'second input, first video track only'?

4. Here's my command to convert the .mov files, which seems to work as desired. Is there anything obviously wrong with it?

   ffmpeg -i left.mov -c:v libx264 -preset medium -tune film -crf 16 -c:a aac -b:a 256k left.mp4

Answer 1

Quality-wise, it's better to use the original MOVs, but if the MP4-based stacking looks ok, that's the final arbiter.

ffmpeg -i left.mov -i centre.mov -i right.mov 
-filter_complex "[0:v:0][1:v:0][2:v:0]hstack=inputs=3"
-c:v libx264 -tune film -crf 16 -b:a 256k triple.mp4

When using the originals, it doesn't matter whether you do it in one step or two. For streamlining purposes, I would do it in two - then the two encodes aren't competing for memory or CPU.

1. The stack filters are somewhat more efficient. They require all inputs to have the same size in the dimension normal to the join direction i.e. same height for hstack. Overlay is a generic filter, so it doesn't. Also, stack filters will freeze on the first and last frame of inputs which start late or finish early.

2. In digital files, audio and video are independent i.e. they exist within the same file and are synced via timestamps but logically they are distinct entities. ffmpeg relies on -map options to select streams for inclusion in the output. In the absence of any -map option, it picks one audio - the "best" - from among all the inputs. See https://ffmpeg.org/ffmpeg.html#Stream-selection

3. [1:v:0] ... "means 'second input, first video track only'?" --> Yes.

4. Looks fine. The suitability of the tune value depends on the material.