I am currently working on a personal programming challenge to develop an MP4 decoder using VB.NET and the .NET Framework 4.8.1 as a WinForms application. I have obtained the ISO 14496-12
documentation, which serves as the basic structure for MP4 files.
At this stage, I have successfully prepared various lists containing frame sizes, chunk positions, and information from the stsc and stts boxes. Additionally, I have been able to identify keyframes within the MP4 files.
However, I have come to realize that the ISO 14496-12
documentation is not sufficient for decoding the frame bytes, which are encoded in H.264/AVC. To proceed with decoding, I understand that I need to refer to the ISO/IEC 14496-10
documentation specifically for H.264/AVC.
My question is twofold:
- When decoding the frame bytes according to the documentation, would the output data be in a specific format, such as JPEG (identified by the FF D8 marker), or would it be raw bytes representing pixel data? This clarification would help me. In the former case, I could simply save an image using .NET functions.
- Considering my current progress and the information I have mentioned, do you foresee any need for me to purchase a third documentation source?
Any guidance or suggestions would be appreciated.