How can I convert .MTS file (AVCHD) to .mp4 by ffmpeg without re-encoding H264 video stream?

Question

Note: I posted same question to stackoverflow a short time before, when I have not found this comunity yet. I repost this since the question is more suitable to this community.

1. What I tried

I have some .MTS (AVCHD format) files recoreded with my AVCHD camera. Its specification is as shown below:

$ ffprobe 140612_Canon-00000.MTS 
ffprobe version 2.2.1 Copyright (c) 2007-2014 the FFmpeg developers
(snip)
Input #0, mpegts, from '140612_Canon-00000.MTS':
  Duration: 00:48:58.40, start: 0.800300, bitrate: 5563 kb/s
  Program 1 
    Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), 
      yuv420p, 1440x1080 [SAR 4:3 DAR 16:9], 
      29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 
      stereo, fltp, 256 kb/s

Pay attention to the part of framerate/timebase: 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

Now I’d like to convert this file to .mp4 file, without re-encoding H264 video stream, on the other hand, with transcoding its audio stream to AAC. So I tried the following command:

ffmpeg -i 140612_Canon-00000.MTS -t 60 -y -vcodec copy -acodec libfaac -ab 128k 140612_Canon-00001.MTS.mp4

2. Result

and output file’s specification is as shown below:

$ ffprobe 140612_Canon-00000.MTS.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '140612_Canon-00000.MTS.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf55.33.100

  Duration: 00:01:00.04, start: 0.021333, bitrate: 4590 kb/s

    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 
        1440x1080 [SAR 4:3 DAR 16:9], 4448 kb/s, 
        59.94 fps, 59.94 tbr, 90k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler

    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 
        48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

Look at the part of framerate/timebase: 59.94 fps, 59.94 tbr, 90k tbn, 59.94 tbc. Although ffmpeg just copied the video stream, framerate and timebase has been changed to twice value.

So, when I open and playback the output file with QuickTime Player or VLC Player, the audio has no problem, however, the video stream is not played correctly. The video is played back with having its frame forward and backward quiveringly repeatedly.

3. Question

1. How can I convert .MTS file (AVCHD) to .mp4 by ffmpeg without re-encoding H264 video stream correctly?
2. How can I keep the original framerate/timebase values (fps/tbr/tbn/tbc) when I convert the container with ffmpeg and its -vcodec copy switch.
3. How can I set framerate/timebase values (fps/tbr/tbn/tbc) by ffmpeg’s command line options without re-encoding a video stream.

Any ideas?

---

4. Adding -r 29.97 option

Professor Sparkles gave me an advice to add -r 29.97. I tried that:

ffmpeg -i 140612_Canon-00001.MTS -t 60 -r 29.97 -y -vcodec copy -acodec libfaac -ab 128k 140612_Canon-00001.MTS.mp4

However, the output file has still wrong framerate/timebase:

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 
1440x1080 [SAR 4:3 DAR 16:9], 4448 kb/s, 
59.94 fps, 59.94 tbr, 11988 tbn, 59.94 tbc (default)

---

5. Remux using MP4Box

I tried demux and remux using MP4Box, according to Professor Sparkles’s advice.

brew install mp4box

ffmpeg -i 140612_Canon-00000.MTS -t 60 -y \
  -vcodec copy -an 140612_Canon-00000.MTS.h264

ffmpeg -i 140612_Canon-00000.MTS -t 60 -y \
  -vn -acodec libfaac -ab 128k 140612_Canon-00000.MTS.aac

mp4box -add 140612_Canon-00000.MTS.h264:fps=29.97 \
  -add 140612_Canon-00000.MTS.aac \
  -new 140612_Canon-00000.MTS.mp4

and output was:

$ ffprobe 140612_Canon-00000.MTS.mp4

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '140612_Canon-00000.MTS.mp4':
(snip)
  Duration: 00:02:00.22, start: 0.000000, bitrate: 2293 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), 
          yuv420p, 1440x1080 [SAR 4:3 DAR 16:9], 2228 kb/s, 
          29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2014-07-14 00:38:23
      handler_name    : 140612_Canon-00000.MTS.h264:fps=29.97
       - Imported with GPAC 0.5.0-rev4065

    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, 
          stereo, fltp, 125 kb/s (default)

It looks a bit step forward. Look at the part of framerate/timebase: 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc. Those are match to the original stream, except for tbn (timebase value from container).

However, when I play back the output file with Quicktime Player or VLC, the video is play back at the half speed.

Although the original file has 90k tbn (90000 ticks per second) and the new output file from MP4Box has 30k tbn which is just one-third value of original one, the output file is play back at the half speed.

I don't know the reason why. But I think the rest issue is how I can adjust the tbn value.

---

5-b. MediaInfo's report on the output file

I also tried MediaInfo tool on the output file generated by 5. Remux using MP4Box. The output is here: https://gist.github.com/kaorukobo/c5ab9eaa413dff6cd26a

---

6. Trying avconv

Volodya reported avconv worked well. Also I prepared a short sample movie file (Canon-00006.MTS) recorded with the same camera. Okay, let's try:

brew install avconv
avconv -i Canon-00006.MTS -c:a copy -c:v copy -y Canon-00006.MTS.mp4
ffprobe Canon-00006.MTS.mp4

The output file's ffprobe information is here: https://gist.github.com/kaorukobo/5b53244ade2632ff1211 and its framerate/timebase information is as the following: 59.94 fps, 59.94 tbr, 90k tbn, 59.94 tbc

The output file was played back well with VLC Player as Volodya reported. However, opening it with Quicktime Player X, the video was play back at the normal speed but with having its frame backward quiveringly repeatedly.

---

7. Why "Free AVCHD to MOV" app work?

As I mentioned in my previous comment, Free AVCHD to MOV app's "rewrap to MOV" feature worked well, even though it makes not MP4 but MOV.

The software calls its own ffmpeg (or avconv) program internally, and I saw what options are passed to it. It's as shown below:

/Applications/Free AVCHD to Mov.app/Contents/Resources/bin/com.geranium-soft.convert \
  -i /path/to/140710_Canon-00003.MTS \
  -map 0:0 -map 0:1 -c:a libfaac -vol 256 -b:a 128k -c:v copy \
  -sn -movflags faststart -threads 0 -pix_fmt yuv420p -y \
  /path/to/140710_Canon-00003.mov

I tried passing same options (Extractly same. I set the output container type to MOV and removed even -t 60 switch.) to ffmpeg program and converting. But the result was same as reported so far.

Anyway that great app resolved my issue on "How can I convert .MTS file (AVCHD) to .mp4 without re-encoding H264 video stream?", except at "to .mp4" and "by ffmpeg". But I'm still interested in why that app do well but ffmpeg don't.

Answer 1

Seeing that in the text of your question you have started discussing other utilities, i will assume that you are not interested in sticking with ffmpeg, but rather in getting the job done.

In my experience with libav and MTS i have had no problems with the framerate, the files get remuxed perfectly.

I have just attempted the following with one of my files:

avconv -i 00174.MTS -c:a copy -c:v copy 00174.mp4

The resulting MP4 file played correctly with VLC.

My file is progressive MTS, i don't have any interlaced laying around, but if need be i can do more checking with that.

Report about the file test

The topic starter was able to provide a file, that was remuxed from MTS to MP4 and did not play on that person's machine with QuickTime Player (version unknown). It did, however, play with that individual's VLC player.

I do not have a Mac OS computer, but i have tried it with Ubuntu. I have played it on Ubuntu on VLC (2.0.8) and GNOME Videos (used to be called Totem) (3.8.2); both play perfectly.

I then asked a friend of mine, who is on a Mac to play it. He is on Mavericks (10.9.4), and it played fine with QuickTime Player 10.3 (727.4).

At this time it appears to be a problem with the particular player or a problem with configuration settings on the computer. And it is probably best to attempt to update to the newest version of QTP, possibly with first removing the current version and completely wiping the old configuration.

Another possibility

When i have had an old machine i have had some high bitrate files play incorrectly in some players, and it was container specific. For example, VLC would refuse to play MTS file, it would show a frame, and then show the next one only in a second and a half. GNOME Videos played it fine. But when remuxing it to MKV, both players played it reasonably well. This maybe a question of something similar. A player maybe reading a particular container (MP4 in this case) in a way that requires just enough CPU time, that it begins to choke itself. The jerkiness effect can be then attributed to whatever subprocess takes up CPU finishing, and player dumping all of the frames that were lagging behind very rapidly, after that the bad subprocess again kicks in, and the cycle keeps on going.

In this possibility, the best option is still to try to upgrade the software. With the current multicore processors it would be difficult to test the need to upgrade the hardware without actually getting it, but maybe it is possible to look at the CPU load during the QuickTime Player useage and compare it to VLC. If you see 100% for any one core with QTP, that maybe indicative of this.

Answer 2

As per this ffmpeg bug

Interlaced H.264 packets are split causing MP4 STTS

when remuxing a mpeg-ts containing interlaced H.264 into mp4, both fields of each video frame are split into seperate packets. Software such as Mediainfo uses the STTS to determine the frame rate. It will show as 50fps instead of 25fps

The frame rate mismatch reported here seems to be a result of ffmpeg muxing interlaced MP4 streams according to the specification, as per which, each field is separated into one packet. And thus

"software which uses the sample count in the MP4 file to determine the frame rate is simply wrong." Comment 7

This will not be patched as merging field pairs into one access unit violates the MPEG-4 specification, and hence so do any encoders which do the same.

Note that the muxed output, such as the one below, plays fine for me in Potplayer and VLC.

    ffmpeg version N-76741-g8eadabf Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.2.0 (GCC)

Input #0, mpegts, from '00007.MTS':
  Duration: 00:00:07.01, start: 1.033367, bitrate: 15935 kb/s
  Program 1 
    Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1100]: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 448 kb/s
    Stream #0:2[0x1200]: Subtitle: hdmv_pgs_subtitle ([144][0][0][0] / 0x0090), 1920x1080
[mp4 @ 054cf020] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, mp4, to '00007.MTS.mp4':
  Metadata:
    encoder         : Lavf57.16.100
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 29.97 fps, 29.97 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, 5.1(side), fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.15.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[mp4 @ 054cf020] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 054cf020] pts has no value
    Last message repeated 209 times
frame=  420 fps=0.0 q=-1.0 Lsize=   12478kB time=00:00:07.01 bitrate=14564.2kbits/s    
video:12458kB audio:6kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.111239%
[aac @ 052fd480] Qavg: 64863.176

Answer 3

You might want to try to enforce the original frame rate by using -r 29.97. FFmpeg is probably trying to adjust the framerate for some reason. Your syntax is otherwise correct and shouldn't produce that error.

Regarding your third question. Simply not possible. You can omit frames when using codecs that encode frames individually but thats not the case with h264 but even with such a codec you still modify the video stream in some way. The same goes for increasing frame rate, you have to either add calculated frames or duplicate some frames.

Edit: Regarding the additional info from the comment below. If you need to change the data written in the format header without writing a completely new file you probably want to do that in a hex editor. FFmpeg only has option to alter metadata which doesn't include the stream data. How and where to make your changes in the file depends on the container format.

Another option would be to demux the container and remux the video and audio stream into a new container with your options specified. How much you can specifiy again depends on the container format. The tool MP4Box might be of help with that in this case, you can specify a framerate when muxing raw video streams into a new mp4 using the following syntax:

MP4Box -add input.h264:fps=29.97 -new output.mp4

Answer 4

I know this is an old question, but it just popped up again in the feed so it's new to me. (-:

One thing I don't see mentioned is field order. This is an interlaced file, so that's a consideration. The OP mentions the frames "quivering back and forth" which is always a flag for incorrect field order. If the video is otherwise OK except for the 'quivering', try adding whatever ffmpeg needs to force "top field first", then the opposite if that's still wrong. I'm not familiar enough with ffmpeg details to give the exact flags for that.