I develop an application to record from multiple high-speed (e.g. 200 fps) cameras simultaneously. Typically, the frame rate of these cameras is constant but this is not necessarily the case. Typically, these cameras have synchronized acquisition (from a trigger "GenLock" cable) but this is not necessarily the case. Up to now, I use a self-developed method for ensuring accurate timing is kept across the video clips saved from the different cameras (described below). This works but I would like to use standard tools to be able to open my multi-camera data and have the clips automatically synchronized. Is there a standard to store multi-camera high-speed video so that the relative timing between the cameras and individual frames is maintained?
I understand that timecode tracks in the clips may be able to fill this role nicely for "normal" video. These generally seem to use the SMPTE standard. While I have not used this method myself, I am concerned that only common frame rates (like 24, 25, 29.97 and 30 fps) are supported, especially across different software. I have also read a little bit about the IRIG standard, but I have also not used it.
Does an existing standard file format fill the role I need? Is there a some established solution which may not be "standard"? Can you point me to any example videos stored with relevant data and any example applications which can open such data? Does the solution allow for variable frame rate, or is the frame rate fixed? Ideally, the solution would involve open standards and would be widely compatible and understood by various software.
Here I describe my existing solution. I save video from each camera in a separate file using the MKV format which has the (unique among container formats?) characteristic that the "creation time" in the metadata can be specified to the nanosecond level. This is what I do for the first frame. (I additionally save this information redundantly by making the automatically calculated filename have the creation time also to a high precision.) Then, each subsequent frame uses the PTS (presentation time stamp) to indicate the time it was taken relative to the first frame. This works nicely and individual videos are compatible with existing tools like FFMPEG and VLC. However, this scheme is fragile. Many standard video editing programs (e.g. DaVinci Resolve) do not open MKV files and the creation time metadata is saved only to the second (not nanosecond) upon conversion to MP4 container and consequently I loose synchronization across multiple cameras if I convert them to MP4 containers.
