2

I bought a Xiaomi Mijia MI SPHERE 360 Camera, it has an embedded gyroscope. The iOS app allows exporting stabilized movies combining the video and gyro data.

The camera records .mp4 files of the following format: ISO Media, MPEG v4 system, 3GPP JVT AVC [ISO 14496-12:2005]

I would like to retrieve the gyro information of my movie along the movie. I can imagine there is exif like metadata for each frame where the camera would store specific information but I have not found any way yet to put the hand on it.

How could I retrieve the metadata per frame containing this information?

[Edit] Here is a github project containing 2 short (10sec) clips I made. I recorded in the dark. One is steady (camera laid on the ground) the second one shaking. Both have a useless? audio track.

Improve this question
5
  • Does the iOS app work with a rewrapped file: ffmpeg -i orig.mp4 -c copy new.mp4
    – Gyan
    Commented Apr 6, 2018 at 10:06
  • @Mulvya I made the copy and put it back on the key, the file is seen by the iOS app as corrupted, there is no preview thumbnail and downloading it results in a 360 video without the gyro calibration and without the lens calibration neither. It is like the app takes a FOV of +200° instead of 190°. I, therefore, suspect the file to contain a padding of proprietary data holding the gyro, thumbnail and lens calibration information. Next step… to find it and decrypt.
    – Flavien Volken
    Commented Apr 7, 2018 at 22:52
  • @Mulvya you can post an answer as there is none yet and I've 16 hours remaining to award the bounty (or it will just be lost forever).
    – Flavien Volken
    Commented Apr 9, 2018 at 12:15
  • Added some info as answer.
    – Gyan
    Commented Apr 9, 2018 at 12:53
  • In the photos the data is for sure stored in a proprietary way. Usually such data is stored in custom exif tags, but there are no exif tags in the images even though they contain the information. The exif tag called User comment actually has exactly 36 bytes of binary data, which corresponds to the answer by Flavien Volken.
    – simonmikkelsen
    Commented Sep 4, 2018 at 19:30

2 Answers 2

Reset to default
2

The data are stored within the file disregarding of the .mp4 file standard.

the chunk containing those per frame data starts with: #¿Ttlyd or 23C05474 6C7964in hex enter image description here

Each frame has 36 bytes of data, those seem to be 9 floats of normalized (from -1 to 1) numbers

Those numbers seem to be a 3x3 matrix representing the current coordinate system for this frame.

Improve this answer
2
  • Did you find out if it is really a 3x3 rotation matrix, and what is the format of the floats?
    – Juan Cespedes
    Commented Dec 28, 2018 at 16:13
  • Well… kind of but it's a mess to get the proper axis. Check my repo for a working viewer POC made in angular. github.com/Xample/mi-sphere-gyro-extractor
    – Flavien Volken
    Commented Dec 29, 2018 at 15:44
1
+100

There is a madv box in the udta atom that is not normally seen in MP4s

type:'madv' parent:'udta' sz: 3145728 115 3145835

It has sufficient size for it to be a candidate.

This is a dump of the first ~350 bytes

madv: s=   3145728 (0x00300000), o=   1950821 (0x001dc465)
  000000   00 00 00 ac 63 75 74 6c  00 00 00 00 00 00 00 00  |....cutl........|
  000010   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000020   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000030   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000040   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000050   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000060   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000070   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000080   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000090   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  0000a0   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|
  0000b0   66 6c 74 72 00 00 00 00  00 00 00 88 63 61 69 66  |fltr........caif|
  0000c0   32 30 31 38 30 33 33 31  00 00 00 00 00 00 00 00  |20180331........|
  0000d0   31 31 00 00 30 00 00 00  00 00 00 00 00 00 00 00  |11..0...........|
  0000e0   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  0000f0   51 4a 58 4a 30 31 46 4a  00 00 00 00 00 00 00 00  |QJXJ01FJ........|
  000100   31 2e 33 2e 31 30 34 2e  31 31 31 2e 35 2e 31 37  |1.3.104.111.5.17|
  000110   33 33 32 00 00 00 00 00  00 00 00 00 00 00 00 00  |332.............|
  000120   31 35 39 30 38 2f 30 30  30 30 38 30 36 32 00 00  |15908/00008062..|
  000130   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  000140   00 00 02 5c 67 70 73 78  00 00 00 00 00 00 00 00  |...\gpsx........|
  000150   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Although not knowing the storage format, I can't confirm it or parse it.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.