I am creating MPEG-TS segments for HLS playback from multiple ffmpeg processes (it will be used for parallel encoding at a later stage). The commands are as follows — they can be run in sequence for testing purposes:
#!/usr/bin/env bash
rm seg*.ts master.m3u8
ffmpeg \
-f lavfi -i testsrc=s=320x240:r=30 -f lavfi -i sine=440 \
-vf 'drawtext=text=%{n}:fontsize=72:r=60:x=(w-tw)/2: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000099' \
-pix_fmt yuv420p \
-preset ultrafast -c:v:0 libx264 -x264-params "nal-hrd=cbr:force-cfr=1" -b:v:0 4M -maxrate:v:0 4M -minrate:v:0 4M -bufsize:v:0 8M -g 60 -sc_threshold 0 -keyint_min 60 \
-f hls -hls_time 2 -hls_playlist_type event \
-hls_flags independent_segments+append_list+omit_endlist \
-hls_segment_type mpegts -hls_list_size 0 \
-hls_segment_filename seg_01_%02d.ts -master_pl_name master.m3u8 -start_number 0 \
-muxdelay 0 \
-muxpreload 0 \
-output_ts_offset 0 \
-t 10 \
master.m3u8
ffmpeg \
-f lavfi -i testsrc=s=320x240:r=30 -f lavfi -i sine=440 \
-vf 'drawtext=text=%{n}:fontsize=72:r=60:x=(w-tw)/2: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000099' \
-pix_fmt yuv420p \
-preset ultrafast -c:v:0 libx264 -x264-params "nal-hrd=cbr:force-cfr=1" -b:v:0 4M -maxrate:v:0 4M -minrate:v:0 4M -bufsize:v:0 8M -g 60 -sc_threshold 0 -keyint_min 60 \
-f hls -hls_time 2 -hls_playlist_type event \
-hls_flags independent_segments+append_list \
-hls_segment_type mpegts -hls_list_size 0 \
-hls_segment_filename seg_02_%02d.ts -master_pl_name master.m3u8 -start_number 0 \
-muxdelay 0 \
-muxpreload 0 \
-output_ts_offset 10 \
-ss 10 \
-to 20 \
master.m3u8
You can load this in your browser if you simply run a web server like python3 -m http.server with:
<html>
<head></head>
<body>
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<video id="video" controls autoplay></video>
<script>
var video = document.getElementById("video");
var videoSrc = "master.m3u8";
var hls = new Hls({
debug: true,
});
hls.loadSource(videoSrc);
hls.attachMedia(video);
</script>
</body>
</html>
Now, the video seems to play fine. Looking at the frame counter, I see that no frames are lost (although it seems like the very last frame of segment set 1 is displayed a little bit shorter).
The audio, however, has a small dropout at the switch from segment set 1 to 2.
The generated M3U8 has an #EXT-X-DISCONTINUITY tag before switching to the second segment set.
I noted that the start times of the segments don't quite correspond when comparing the audio and video streams. To test this, I run ffprobe and look at the start_time and start_pts values:
for s in *.ts; do ffprobe -loglevel error -hide_banner -select_streams v -show_streams -of json "$s" | jq --arg input "$s" '.streams[0] | {$input, duration, start_time, start_pts}'; done | jq -s .
This is the output:
[
{
"input": "seg_01_00.ts",
"duration": "2.000000",
"start_time": "0.023222",
"start_pts": 2090
},
// ...
{
"input": "seg_01_04.ts",
"duration": "2.000000",
"start_time": "8.023222",
"start_pts": 722090
},
// ...
{
"input": "seg_02_05.ts",
"duration": "2.000000",
"start_time": "10.000000",
"start_pts": 900000
},
// ...
]
The second set is forced to start at 10 seconds, while the last segment of the first set actually extends to 8.023222 + 2 = 10.023222 seconds. This is due to the non-negative offset of the first segment of the first set.
For audio, the timestamps and durations are completely different:
[
{
"input": "seg_01_00.ts",
"duration": "2.043344",
"start_time": "0.000000",
"start_pts": 0
},
// ...
{
"input": "seg_01_04.ts",
"duration": "1.996900",
"start_time": "8.034111",
"start_pts": 723070
},
{
"input": "seg_02_05.ts",
"duration": "2.043344",
"start_time": "9.976778",
"start_pts": 897910
},
// ...
]
Is there any way to fix this, so that the segments generated by the second ffmpeg command can be stitched during playback, without choppy audio?
I have tried another approach, which consists of generating the audio stream separately before starting to encode the video. The audio and video playlists are written into dedicated .m3u8 files which I am simply referencing from a master.m3u8 playlist. This one I have to manually generate, because otherwise ffmpeg would overwrite it. The whole approach is shown here.
The problem is that the video won't start in hls.js:
[log] > Unknown video PTS for cc 0, waiting for video PTS before demuxing audio frag 1 of [0 ,10],track 0
This can be demoed here.
Once I remux everything into new sets of segments, as suggested by @Gyan, it works again. Demo is here.
However, this is not a workable solution here, since I need to be able to generate the HLS segments on the fly, and append to the playlists later.
