0

I'm overlaying the same HD footage multiple times on a 4K image sequence but I need to offset the start with a holding loop on the first frame. I've been using filters to split and then loop the first frame with different offsets, but when I hstack/vstack and overlay the s1trans and s4trans streams that have the longer loop skip a few frames when they come out of the loop.

-framerate 30 -t 60.0 -i 4K_%04d.jpg
-framerate 30 -t 15.0 -i HD_%04d.jpg 

-filter_complex "
[1]split=4[trans1][trans2][trans3][trans4];
[trans1]loop=296:1:0[s1trans];
[trans2]loop=278:1:0[s2trans];
[trans3]loop=269:1:0[s3trans];
[trans4]loop=291:1:0[s4trans];
[s1trans][s2trans]hstack[top];
[s3trans][s4trans]hstack[bottom];
[top][bottom]vstack[trans];
[0][trans]overlay=format=rgb"

If I instead bring in the HD footage four times and apply the same loop it has no issue and I get the result I'm expecting.

-framerate 30 -t 60.0 -i 4K_%04d.jpg
-framerate 30 -t 15.0 -i HD_%04d.jpg 
-framerate 30 -t 15.0 -i HD_%04d.jpg 
-framerate 30 -t 15.0 -i HD_%04d.jpg 
-framerate 30 -t 15.0 -i HD_%04d.jpg 

-filter_complex "
[1]loop=296:1:0[s1trans];
[2]loop=278:1:0[s2trans];
[3]loop=269:1:0[s3trans];
[4]loop=291:1:0[s4trans];
[s1trans][s2trans]hstack[top];
[s3trans][s4trans]hstack[bottom];
[top][bottom]vstack[trans];
[0][trans]overlay=format=rgb"

Is there any issue with doing it in this way? I'd assume it would be worse for performance. Am I missing something when doing the split and loop?

Console output:

ffmpeg.exe -y -start_number 101 -framerate 30 -t 60.0 -i *\Cam_0001.%04d.jpg -framerate 30 -t 15.0 -i *\Shutter_Wall_Opening_Slow_%03d.png -framerate 30 -loop 1 -i \FilmGrain_%04d.png -filter_complex "[0:v]split=3[in1][in2][orig];[in1]crop=960:270:0:0[top];[in2]crop=960:270:960:0[bottom];[top][bottom]vstack[s1];[1:v]scale=480x270[trans];[trans]split=4[s1trans][s2trans][s3trans][s4trans];[s1trans]loop=296:1:0[s1trans];[s2trans]loop=278:1:0[s2trans];[s3trans]loop=269:1:0[s3trans];[s4trans]loop=291:1:0[s4trans];[s1trans][s2trans]hstack[top];[s3trans][s4trans]hstack[bottom];[top][bottom]vstack[trans];[s1][trans]overlay=format=rgb[s1];[s1]setsar=1:1[s1];[2:v]setsar=1:1[grain];[s1][grain]blend=shortest=1:all_mode=overlay:all_opacity=0.03[s1]" -map [s1] -r 30 -c:v libx264 -crf 18 -bufsize 24M -profile:v high -level 4.1 -pix_fmt yuv420p "*SQ001_SH9999_Render_v002.mp4" -map [orig] -r 30 -c:v libx264 -crf 18 -bufsize 24M -profile:v high -level 4.1 -pix_fmt yuv420p "*SQ001_SH9999_Render_v002_ORIG.mp4" ffmpeg version N-87130-g2b9fd15 Copyright (c) 2000-2017 the FFmpeg developers built with gcc 7.1.0 (GCC) configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib libavutil 55. 74.100 / 55. 74.100 libavcodec 57.104.100 / 57.104.100 libavformat 57. 79.100 / 57. 79.100 libavdevice 57. 8.100 / 57. 8.100 libavfilter 6.101.100 / 6.101.100 libswscale 4. 7.103 / 4. 7.103 libswresample 2. 8.100 / 2. 8.100 libpostproc 54. 6.100 / 54. 6.100 Input #0, image2, from '\Cam_0001.%04d.jpg': Duration: 00:01:03.33, start: 0.000000, bitrate: N/A Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1920x270 [SAR 1:1 DAR 64:9], 30 fps, 30 tbr, 30 tbn, 30 tbc Input #1, image2, from '\Shutter_Wall_Opening_Slow_%03d.png': Duration: 00:00:15.00, start: 0.000000, bitrate: N/A Stream #1:0: Video: png, rgba(pc), 1920x1080 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc Input #2, image2, from '\FilmGrain_%04d.png': Duration: 00:00:08.33, start: 0.000000, bitrate: N/A Stream #2:0: Video: png, rgb24(pc), 960x540, 30 fps, 30 tbr, 30 tbn, 30 tbc [Parsed_setsar_14 @ 0000000004b9c120] num:den syntax is deprecated, please use num/den or named options instead [Parsed_setsar_15 @ 0000000004b9bc40] num:den syntax is deprecated, please use num/den or named options instead Stream mapping: Stream

0:0 (mjpeg) -> split Stream #1:0 (png) -> scale Stream #2:0 (png) -> setsar blend -> Stream #0:0 (libx264) split:output2 -> Stream #1:0 (libx264) Press [q] to stop, [?] for help [Parsed_setsar_14 @ 000000000c549ea0] num:den syntax is deprecated, please use num/den or

named options instead [Parsed_setsar_15 @ 000000000c549c40] num:den syntax is deprecated, please use num/den or named options instead [swscaler @ 000000000c4eb060] deprecated pixel format used, make sure you did set range correctly [libx264 @ 00000000046c54a0] VBV bufsize set but maxrate unspecified, ignored [libx264 @ 00000000046c54a0] using SAR=1/1 [libx264 @ 00000000046c54a0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 00000000046c54a0] profile High, level 4.1 [libx264 @ 00000000046c54a0] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=17 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to '*SQ001_SH9999_Render_v002.mp4':
Metadata: encoder : Lavf57.79.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 960x540 [SAR 1:1 DAR 16:9], q=-1--1, 30 fps, 15360 tbn, 30 tbc (default) Metadata: encoder : Lavc57.104.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 24000000 vbv_delay: -1 [libx264 @ 0000000004bdff80] VBV bufsize set but maxrate unspecified, ignored [libx264 @ 0000000004bdff80] using SAR=1/1 [libx264 @ 0000000004bdff80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0000000004bdff80] profile High, level 4.1 [libx264 @ 0000000004bdff80] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=8 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #1, mp4, to '*SQ001_SH9999_Render_v002_ORIG.mp4':
Metadata: encoder : Lavf57.79.100 Stream #1:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x270 [SAR 1:1 DAR 64:9], q=-1--1, 30 fps, 15360 tbn, 30 tbc (default) Metadata: encoder : Lavc57.104.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 24000000 vbv_delay: -1 frame= 1800 fps= 52 q=-1.0 Lq=-1.0 size= 7417kB time=00:00:59.90 bitrate=1014.4kbits/s speed=1.73x video:13455kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown [libx264 @ 00000000046c54a0] frame I:8 Avg QP:14.88 size: 73240 [libx264 @ 00000000046c54a0] frame P:454 Avg QP:17.52 size: 8941 [libx264 @ 00000000046c54a0] frame B:1338 Avg QP:19.94 size: 2188 [libx264 @ 00000000046c54a0] consecutive B-frames: 0.9% 0.0% 0.0% 99.1% [libx264 @ 00000000046c54a0] mb I I16..4: 16.5% 48.0% 35.5% [libx264 @ 00000000046c54a0] mb P I16..4: 2.3% 3.1% 1.3% P16..4: 41.4% 12.0% 9.7% 0.0% 0.0% skip:30.2% [libx264 @ 00000000046c54a0] mb B I16..4: 0.4% 0.1% 0.0% B16..8: 39.5% 3.0% 0.5% direct: 1.0% skip:55.6% L0:46.3% L1:51.1% BI: 2.6% [libx264 @ 00000000046c54a0] 8x8 transform intra:43.0% inter:82.2% [libx264 @ 00000000046c54a0] coded y,uvDC,uvAC intra: 38.2% 63.8% 37.3% inter: 10.7% 13.6% 1.5% [libx264 @ 00000000046c54a0] i16 v,h,dc,p: 57% 35% 3% 5% [libx264 @ 00000000046c54a0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 11% 54% 8% 2% 2% 2% 4% 4% [libx264 @ 00000000046c54a0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 38% 21% 3% 2% 3% 3% 3% 3% [libx264 @ 00000000046c54a0] i8c dc,h,v,p: 39% 31% 28% 3% [libx264 @ 00000000046c54a0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 00000000046c54a0] ref P L0: 55.4% 4.7% 24.6% 15.3% [libx264 @ 00000000046c54a0] ref B L0: 78.9% 16.1% 5.0% [libx264 @ 00000000046c54a0] ref B L1: 89.7% 10.3% [libx264 @ 00000000046c54a0] kb/s:1009.59 [libx264 @ 0000000004bdff80] frame I:8 Avg QP:14.90 size: 16952 [libx264 @ 0000000004bdff80] frame P:454 Avg QP:18.13 size: 7030 [libx264 @ 0000000004bdff80] frame B:1338 Avg QP:19.81 size: 2151 [libx264 @ 0000000004bdff80] consecutive B-frames: 0.9% 0.0% 0.0% 99.1% [libx264 @ 0000000004bdff80] mb I I16..4: 24.9% 47.7% 27.4% [libx264 @ 0000000004bdff80] mb P I16..4: 4.3% 5.0% 1.9% P16..4: 30.3% 11.0% 6.0% 0.0% 0.0% skip:41.5% [libx264 @ 0000000004bdff80] mb B I16..4: 0.6% 0.2% 0.1% B16..8: 33.1% 3.8% 0.6% direct: 1.1% skip:60.5% L0:43.4% L1:52.7% BI: 3.9% [libx264 @ 0000000004bdff80] 8x8 transform intra:41.0% inter:82.3% [libx264 @ 0000000004bdff80] coded y,uvDC,uvAC intra: 26.4% 33.8% 17.6% inter: 9.5% 4.8% 0.9% [libx264 @ 0000000004bdff80] i16 v,h,dc,p: 56% 35% 4% 5% [libx264 @ 0000000004bdff80] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 11% 49% 7% 1% 2% 1% 4% 3% [libx264 @ 0000000004bdff80] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 39% 22% 3% 1% 1% 2% 2% 2% [libx264 @ 0000000004bdff80] i8c dc,h,v,p: 45% 25% 29% 1% [libx264 @ 0000000004bdff80] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0000000004bdff80] ref P L0: 62.9% 5.9% 20.5% 10.7% [libx264 @ 0000000004bdff80] ref B L0: 84.9% 12.2% 2.9% [libx264 @ 0000000004bdff80] ref B L1: 93.5% 6.5% [libx264 @ 0000000004bdff80] kb/s:827.29

  • In the log, do you see errors like framesync msgs or the like? – Gyan Mar 14 '18 at 12:54
  • I'm pretty sure it didn't skip anything. Windows overwrites the frame output though, I've attached the log here (there's few extra commands in there as this is part of something bigger) – Nick Weeden Mar 14 '18 at 15:04
  • console log attached, it doesn't show anything being dropped. It looks like its in the hstack that it's doing it, as one input from each hstack plays correctly and the other doesn't. – Nick Weeden Mar 14 '18 at 15:26
  • there are potentially two sources of trouble here a)frame buffering, or post-looping TS discontinuities Add setpts=N/30/TB after each loop filter to see if it's the second. – Gyan Mar 14 '18 at 16:24
  • That actually adds in a bit more weirdness, in that trans 4 pops in but it's synchronised with trans 2's timing. – Nick Weeden Mar 15 '18 at 11:06

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.