I've tried researching this very thoroughly, but have not come across a similar issue.
To sum up my problem first, I'm sending 3 second .mkv clips into Amazon Kinesis Video. Then I pull back random intervals of those clips. These pulled clips have “bad” timecode data. The clips keep the original Block timestamps (e.x. between 0 and 3 seconds). The Cluster Timecode is set to 0.
When I pull the video, I get something with block timecodes like this: [00:00:02:20, 00:00:02:80, 00:00:03:00, 00:00:00:20, 00:00:00:80] (It restarts). When played in something like VLC, it will play until it reaches a timecode less than the previous and then stop.
I'm not sure if this is a AWS Kinesis Video issue, or just a general mkv issue.
I record the original clips in Python from a series of images at 15fps with the following code:
self.video_writer = cv2.VideoWriter('path/to/some/file.mkv',
cv2.VideoWriter_fourcc(*'XVID'), 15, self.size)
self.video_writer.write(self.input)
I then run ffmpeg on it with the following command before sending:
ffmpeg -i 'path/to/some/file.mkv' -c:v libx264 -r 15 -codec:a aac \
-vf scale=960:720 -y -async 1 -shortest 'path/to/some/encoded-file.mkv'
I THEN send the video using the Java producer library provided by AWS with the important part here:
dataClient.putMedia(new PutMediaRequest()
.withStreamName(STREAM_NAME)
.withFragmentTimecodeType(FragmentTimecodeType.RELATIVE)
.withPayload(inputStream)
.withProducerStartTimestamp(
new Date([timestamp in milliseconds of start of video])
),
responseHandler);
Everything streams to AWS just fine, and it's visible in the console viewer
I then go to download the video, also using the AWS provided Java consumer library:
FragmentSelector fragmentSelector = new FragmentSelector().withFragmentSelectorType(FragmentSelectorType.PRODUCER_TIMESTAMP).withTimestampRange(getTimestampRange(request));
...
ListFragmentsResult result = this.archivedMedia.listFragments(fragmentsRequest);
...
fragments.sort(Comparator.comparing(Fragment::getProducerTimestamp));
List<String> fragmentIds = fragments.stream().map(Fragment::getFragmentNumber).collect(Collectors.toList());
final GetMediaForFragmentListResult mediaForFragmentListResult = this.archivedMedia.getMediaForFragmentList(
new GetMediaForFragmentListRequest()
.withFragments(fragmentIds)
.withStreamName(streamName)
);
OutputStream fileOutputStream = Files.newOutputStream(path);
BufferedOutputStream outputStream = new BufferedOutputStream(fileOutputStream);
OutputSegmentMerger outputSegmentMerger = OutputSegmentMerger.createDefault(outputStream);
StreamingMkvReader mkvStreamReader = StreamingMkvReader.createDefault(new InputStreamParserByteSource(mediaForFragmentListResult.getPayload()));
try {
mkvStreamReader.apply(outputSegmentMerger);
} catch (MkvElementVisitException e) {
...
}
Again, everything seems to work, until I go to play the video. It will max out at that original 3 seconds of clip, even less if the video started halfway through one of those clips. Putting the video into MKVToolNix shows me the following info (notice the track timestamps near the bottom)
+ EBML head size 47 data size 35
|+ EBML version: 1 size 4 data size 1
|+ EBML read version: 1 size 4 data size 1
|+ Maximum EBML ID length: 4 size 4 data size 1
|+ Maximum EBML size length: 8 size 4 data size 1
|+ Document type: matroska size 11 data size 8
|+ Document type version: 4 size 4 data size 1
|+ Document type read version: 2 size 4 data size 1
+ Segment: size unknown size is unknown
|+ Seek head (subentries will be skipped) size 72 data size 66
|+ EBML void: size 148 size 157 data size 148
|+ Segment information size 87 data size 75
| + Timestamp scale: 1000000 size 7 data size 3
| + Multiplexing application: Lavf57.83.100 size 16 data size 13
| + Writing application: Lavf57.83.100 size 16 data size 13
| + Segment UID: 0x14 0x0e 0xa4 0x81 0x58 0x75 0x40 0x7e 0x15 0xd2 0x28 0x97 0xa4 0xe0 0x27 0x4d size 19 data size 16
| + Duration: 00:00:02.134000000 size 11 data size 8
|+ Tracks size 160 data size 148
| + Track size 142 data size 133
| + Track number: 1 (track ID for mkvmerge & mkvextract: 0) size 3 data size 1
| + Track UID: 1 size 4 data size 1
| + Lacing flag: 0 size 3 data size 1
| + Language: und size 7 data size 3
| + Codec ID: V_MPEG4/ISO/AVC size 17 data size 15
| + Track type: video size 3 data size 1
| + Default duration: 00:00:00.066666666 (15.000 frames/fields per second for a video track) size 8 data size 4
| + Codec's private data: size 42 (H.264 profile: High @L3.1) size 45 data size 42
|+ Tags size 211 data size 199
| + Tag size 68 data size 58
| + Targets size 14 data size 4
| + Track UID: 1 size 4 data size 1
| + Simple size 44 data size 34
| + Name: DURATION size 11 data size 8
| + String: 00:00:02.134000000 size 23 data size 20
|+ Tags size 243 data size 235
| + Tag size 235 data size 229
| + Simple size 97 data size 91
| + Name: AWS_KINESISVIDEO_FRAGMENT_NUMBER size 38 data size 32
| + String: [aws fragment number removed] size 53 data size 47
| + Simple size 65 data size 59
| + Name: AWS_KINESISVIDEO_SERVER_TIMESTAMP size 39 data size 33
| + String: 1568225058.665 size 20 data size 14
| + Simple size 67 data size 61
| + Name: AWS_KINESISVIDEO_PRODUCER_TIMESTAMP size 41 data size 35
| + String: 1568225048.750 size 20 data size 14
|+ Cluster size 209722 data size 209710
| + Cluster timestamp: 00:00:00.000000000 size 3 data size 1
| + Simple block: key, track number 1, 1 frame(s), timestamp 00:00:00.000000000 size 94684 data size 94680
| + Frame size 94676
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.267000000 size 32150 data size 32146
| + Frame size 32142
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.133000000 size 156 data size 153
| + Frame size 149
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.067000000 size 227 data size 224
| + Frame size 220
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.200000000 size 49 data size 47
| + Frame size 43
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.533000000 size 27221 data size 27217
| + Frame size 27213
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.400000000 size 469 data size 466
| + Frame size 462
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.333000000 size 237 data size 234
| + Frame size 230
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.467000000 size 221 data size 218
| + Frame size 214
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.800000000 size 24236 data size 24232
| + Frame size 24228
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.667000000 size 395 data size 392
| + Frame size 388
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.600000000 size 250 data size 247
| + Frame size 243
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.733000000 size 276 data size 273
| + Frame size 269
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.067000000 size 1724 data size 1721
| + Frame size 1717
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.933000000 size 207 data size 204
| + Frame size 200
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:00.867000000 size 206 data size 203
| + Frame size 196
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.533000000 size 228 data size 225
| + Frame size 221
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.867000000 size 7294 data size 7291
| + Frame size 7287
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.733000000 size 181 data size 178
| + Frame size 174
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.667000000 size 160 data size 157
| + Frame size 153
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.800000000 size 52 data size 50
| + Frame size 46
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:02.067000000 size 59 data size 57
| + Frame size 53
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:01.933000000 size 74 data size 72
| + Frame size 68
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:02.000000000 size 42 data size 40
| + Frame size 36
So I'm not sure where to go from here. I've tried ffmpeg's -ts abs option as well as -fflags +genpts but those didn't seem to do anything. I'm thinking if I can set the Clusters Timecode element to be the time the video starts in milliseconds, AWS might ingest it properly, or at least break up the clusters to have multiple. I'm fine with a solution that requires some post processing as well.
(Sorry for the long post, but just making sure everyone has all the info needed!)
