1

I am trying to creeate a tiled video using FFmpeg - a result of putting several videos together, tiled, using -filter_complex with overlays.

Following the how-to here, I have created a script that can tile any number of videos.

The issue is, FFmpeg is being killed during the process. It seems that as few as 4 videos can eat all system resources and I think the Linux core kills FFmpeg.

Killed 8235 fps= 20 q=28.0 size=  107008kB time=00:59:22.17 bitrate= 246.1kbits/s speed=8.86x

The file sizes range but they are nothing too large - typically, FullHD. The files are on an EncFS virtual drive. I have 16 GB ram + 16 GB swap, and the fast SSD (Samsung EVO 850).

How can I prevent this? Is there some way to make FFmpeg use some on-disk buffer? Or perhaps some other approach other than -filter-compled?

I was thinking about joining 2 videos at a time and making it bottom-up, but that would be really very slow and tedious, even if scripted.

ffmpeg \
>      -i '1.mp4' \
>      -i '2.mp4' \
>      -i '3.mp4' \
>      -i '4.mp4' \
>      -i '5.mp4' \
>      -i '6.mkv' \
>     -filter_complex "nullsrc=size=1440x540 [lay0];
>           [0:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_1_1]; 
>           [1:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_1_2]; 
>           [2:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_1_3]; 
>           [3:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_2_1]; 
>           [4:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_2_2]; 
>           [5:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_2_3]; 
>           [lay0][TILE_1_1] overlay=shortest=1:x=0:y=0 [lay1]; 
>           [lay1][TILE_1_2] overlay=shortest=1:x=480:y=0 [lay2]; 
>           [lay2][TILE_1_3] overlay=shortest=1:x=960:y=0 [lay3]; 
>           [lay3][TILE_2_1] overlay=shortest=1:x=0:y=270 [lay4]; 
>           [lay4][TILE_2_2] overlay=shortest=1:x=480:y=270 [lay5]; 
>           [lay5][TILE_2_3] overlay=shortest=1:x=960:y=270     "      -c:v libx264 ./tiled.mp4
ffmpeg version 4.1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug 
   --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 
   --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libaom --enable-libfribidi 
   --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg 
   --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc 
   --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[h264 @ 0x6722b80] A non-intra slice in an IDR NAL unit.
[h264 @ 0x6722b80] decode_slice_header error
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41avc1qt  
    creation_time   : 2015-05-03T12:48:13.000000Z
    encoder         : vlc 2.1.6 stream output
    encoder-eng     : vlc 2.1.6 stream output
  Duration: 01:29:40.93, start: 0.000000, bitrate: 583 kb/s
    Stream #0:0(eng): Audio: mp2 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2015-05-03T12:48:13.000000Z
      handler_name    : SoundHandler
    Stream #0:1(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 2230 kb/s, 15 fps, 15 tbr, 1000k tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2015-05-03T12:48:13.000000Z
      handler_name    : VideoHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '2.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: isom3gp4
    creation_time   : 2014-05-16T10:47:46.000000Z
  Duration: 00:33:58.60, start: 0.000000, bitrate: 18114 kb/s
    Stream #1:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, 17919 kb/s, SAR 1:1 DAR 16:9, 19.85 fps, 24.83 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-05-16T10:47:46.000000Z
      handler_name    : VideoHandle
      encoder         :                                
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #1:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 190 kb/s (default)
    Metadata:
      creation_time   : 2014-05-16T10:47:46.000000Z
      handler_name    : SoundHandle
Input #2, mov,mp4,m4a,3gp,3g2,mj2, from '3.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2018-08-31T09:08:36.000000Z
  Duration: 00:09:44.19, start: 0.000000, bitrate: 7147 kb/s
    Stream #2:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 7017 kb/s, 29.81 fps, 29.81 tbr, 29809 tbn, 59.62 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #2:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2018-08-31T12:37:17.000000Z
      handler_name    : IsoMedia File Produced by Google, 5-11-2011
[h264 @ 0x6749140] A non-intra slice in an IDR NAL unit.
[h264 @ 0x6749140] decode_slice_header error
Input #3, mov,mp4,m4a,3gp,3g2,mj2, from '4.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41avc1qt  
    creation_time   : 2015-10-08T10:04:06.000000Z
    encoder         : vlc 2.1.6 stream output
    encoder-eng     : vlc 2.1.6 stream output
  Duration: 00:58:58.61, start: 0.000000, bitrate: 1099 kb/s
    Stream #3:0(eng): Audio: mp2 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2015-10-08T10:04:06.000000Z
      handler_name    : SoundHandler
    Stream #3:1(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 966 kb/s, 15.07 fps, 15 tbr, 1000k tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2015-10-08T10:04:06.000000Z
      handler_name    : VideoHandler
Input #4, mov,mp4,m4a,3gp,3g2,mj2, from '5.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2016-09-23T09:25:07.000000Z
    com.android.version: 6.0.1
  Duration: 00:18:43.67, start: 0.000000, bitrate: 17643 kb/s
    Stream #4:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 17500 kb/s, SAR 1:1 DAR 16:9, 29.98 fps, 29.97 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2016-09-23T09:25:07.000000Z
      handler_name    : VideoHandle
    Stream #4:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 155 kb/s (default)
    Metadata:
      creation_time   : 2016-09-23T09:25:07.000000Z
      handler_name    : SoundHandle
Input #5, matroska,webm, from '6.mkv':
  Metadata:
    encoder         : android
  Duration: 00:05:29.41, start: 0.000000, bitrate: 22503 kb/s
    Stream #5:0: Audio: flac, 96000 Hz, stereo, s32 (24 bit) (default)
    Stream #5:1: Video: h264 (Baseline), yuvj420p(pc, smpte170m, progressive), 1920x1080, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn, 2k tbc (default)
File './tiled.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
  Stream #0:1 (h264) -> setpts (graph 0)
  Stream #1:0 (h264) -> setpts (graph 0)
  Stream #2:0 (h264) -> setpts (graph 0)
  Stream #3:1 (h264) -> setpts (graph 0)
  Stream #4:0 (h264) -> setpts (graph 0)
  Stream #5:1 (h264) -> setpts (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:0 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264 @ 0x85cd140] A non-intra slice in an IDR NAL unit.
[h264 @ 0x85cd140] decode_slice_header error
[h264 @ 0x72c9100] A non-intra slice in an IDR NAL unit.
[h264 @ 0x72c9100] decode_slice_header error
[swscaler @ 0xa2858c0] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0x85b9d40] using SAR=1/1
[libx264 @ 0x85b9d40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x85b9d40] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x85b9d40] 264 - core 157 r2935 545de2f - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to './tiled.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 0
    compatible_brands: mp41avc1qt  
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1440x540 [SAR 1:1 DAR 8:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2015-05-03T12:48:13.000000Z
      handler_name    : SoundHandler
      encoder         : Lavc58.35.100 aac
Killed 8235 fps= 20 q=28.0 size=  107008kB time=00:59:22.17 bitrate= 246.1kbits/s speed=8.86x
Improve this question

1 Answer 1

Reset to default
0

It seems that overlays are not suitable for this.

I have replaced overlay with xstack, and so far, the tiling with 5x5 tiles continues to run.

ffmpeg \
>      -i 'input1.mp4' \
... 24 other inputs ...
>     -filter_complex "
>           [0:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_1_1]; 
>           [1:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_1_2]; 
...
>           [24:v] setpts=PTS-STARTPTS, scale=480x270 [TILE_5_5]; 
>     [TILE_1_1][TILE_1_2][TILE_1_3][TILE_1_4][TILE_1_5][TILE_2_1][TILE_2_2][TILE_2_3][TILE_2_4][TILE_2_5][TILE_3_1][TILE_3_2][TILE_3_3][TILE_3_4][TILE_3_5][TILE_4_1][TILE_4_2][TILE_4_3][TILE_4_4][TILE_4_5][TILE_5_1][TILE_5_2][TILE_5_3][TILE_5_4][TILE_5_5]xstack=inputs=25:layout=0_0|480_0|960_0|1440_0|1920_0|0_270|480_270|960_270|1440_270|1920_270|0_540|480_540|960_540|1440_540|1920_540|0_810|480_810|960_810|1440_810|1920_810|0_1080|480_1080|960_1080|1440_1080|1920_1080"  
> -c:v libx264 ./tiled.mp4
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.