3

I am doing some tests with the FFMPEG concat filter (DASH experiments), and am seeing that I start losing lipsync after a few concatenations.

Is it expected that one can concatenate independent MP4-boxed streams?

This seems curious that loss of sync should happen, as I am doing the following:

  • Transcoding all sources into the same bitrate, frame rate, GOP size, dimensions and codec.
  • Using MP4Box to create a set of small N-second slices, each being a single GOP.
  • Interleaving the slices with the concat codec from each asset (v1,slice 1;v2,slice 1;v1,slice2...)

Because I am very careful to precondition all the sources to be "in sync", I would expect that concatenating these files should not cause any audio presentation issues, and should look just like a DASH asset (the actual purpose of this test).

UPDATE: I am starting to wonder if it has something to do with some kind of presentation time stamp not being updated correctly, either on the cut or the reassemble side. It always seems that the audio is cutting before the video, and always for the same amount, regardless of the GOP size (being careful of course to make the GOP size a whole fraction of the overall slice size).

As per Mulvya's request, here are the steps I am performing:

Note that I am purposely setting it to one GOP per slice, on the theory that a GOP size smaller than a segment size might be causing the problem. Didn't seem to be the issue.

ffmpeg -i 360a.mp4 
       -vf scale=640:360,setsar=1/1 -r 30 -b:v 1000k
       -c:v libx264 -x264opts keyint=300:min-keyint=300:scenecut=-1
       -c:a aac -strict -2 
       360a-a.mp4

(-strict -2: as you can see, I'm using the stock release built on a Mac, which seems to scream about AAC a lot. Could this be the issue?) Note that using the copy codec for audio didn't make a difference.

mp4box -split 10 360a-a.mp4

ffmpeg -i 360b.mp4 
       -vf scale=640:360,setsar=1/1 -r 30 -b:v 1000k
       -c:v libx264 -x264opts keyint=300:min-keyint=300:scenecut=-1
       -c:a aac -strict -2
       360b-a.mp4

mp4box -split 10 360b-a.mp4

mp4box -cat 360a-a_001.mp4 -cat 360b-a_001.mp4
       -cat 360a-a_002.mp4 -cat 360b-a_002.mp4
       ...
       -new output.mp4

I also tried two forms of the concat operation:

ffmpeg -f concat -i files-18516 -movflags +faststart  -strict -2 -c:v libx264 -c:a aac/copy abc.mp4

And the "complex" version (that word REALLY fits! :-) ):

ffmpeg -i 360a_001.mp4 -i 360b_001.mp4 -i 360a_002.mp4 -i 360b_002.mp4 -i 360a_003.mp4 -i 360b_003.mp4 -i 360a_004.mp4 -i 360b_004.mp4 -filter_complex  [0:0] [0:1]  [1:0] [1:1]  [2:0] [2:1]  [3:0] [3:1]  [4:0] [4:1]  [5:0] [5:1]  [6:0] [6:1]  [7:0] [7:1] concat=n=8:v=1:a=1 [v] [a] -map [v] -map [a] -strict -2 -c:v libx264 -c:a aac/copy abc.mp4

And here are the input and common output formats as reported by FFMPEG. As you can see, the two files were already pretty similar. The extra step is to handle files with different aspect ratios, etc. Note that the video might get stretched...the point here is to test audio issues, not really the video, so I didn't bother padding.

File 360a.mp4:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '360a.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.50.100
  Duration: 00:00:30.53, start: 0.033333, bitrate: 1169 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x360, 1033 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

File 360b.mp4:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '360b.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.50.100
  Duration: 00:00:31.43, start: 0.033333, bitrate: 1120 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x360, 981 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

Common Output Format:

Output #0, mp4, to '360a/b-a.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.48.100
    Stream #0:0(und): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=-1--1, 1000 kb/s, 30 fps, 15360 tbn, 30 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc55.69.100 libx264
    Stream #0:1(und): Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc55.69.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))

Each slice looks essentially like:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '360a-a_001.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isom
    creation_time   : 2015-04-26 19:23:40
  Duration: 00:00:10.00, start: 0.000000, bitrate: 1183 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 1046 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 129 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Improve this question
16
  • Can you post the commands you used? Both MP4Box and ffmpeg.
    – Gyan
    Commented Apr 25, 2015 at 15:26
  • @Mulvya : done.
    – Mark Gerolimatos
    Commented Apr 26, 2015 at 20:00
  • encoder settings: You shouldn't need to mess with scenecut. If x264 wants to insert a non-IDR I frame to compress better, let it. Your min-keyint setting will make sure any extra I frames are not IDR (keyframes). IDK if this will help compression any on scenecuts, but it shouldn't hurt. And should have no bearing on your a/v sync issue. Audio: ffmpeg's built-in aac encoder is significantly worse than libfdk_aac. If you find you need to xcode your audio, use that. Preferably don't xcode the audio at all. (-c:a copy), once you get your a/v sync sorted.
    – Peter Cordes
    Commented Apr 26, 2015 at 20:54
  • Is a/v in sync in the split files, before remuxing them into a merged video? e.g. if you just play one of the 10-sec files with your player of choice.
    – Peter Cordes
    Commented Apr 26, 2015 at 20:55
  • 1
    Why use mp4box to split and concat? ffmpeg offers segment and concat options as well. Split using "-f segment -segment_time 10" and end with "360a-a%03d.mp4". Then create a text file with one entry on each line like "file '360a-a001.mp4", interleaving the entries in the correct order. Then join using -f concat -i text.txt and also with -flags +genpts -async 1
    – Gyan
    Commented Apr 27, 2015 at 5:34

1 Answer 1

Reset to default
4

Split using -f segment -segment_time 10 and end with 360a-a%03d.mp4. Then create a text file with one entry on each line like file '360a-a001.mp4, interleaving the entries in the correct order. Then join using -f concat -i text.txt and also with -fflags +genpts -async 1

The -fflags +genpts -async 1 is necessary to ensure that the concatenated audio segments will switch over at the same time as the video segment.

Improve this answer
2
  • 1
    -fflags, its two "f"s and not one
    – Abdallah Alsamman
    Commented Feb 27, 2017 at 16:42
  • 1
    For me the -async 1 was the key. I had all the other pieces, but without that things were still getting messed up.
    – Eric
    Commented Sep 29, 2017 at 4:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.