I need to modify a movie's displayed subtitles and display as an image overlay

Question

I'm kind of at a loss on how to do this. I have access to gimp/inkscape/ffmpeg/vlc, and I can download font styles as needed off the web.

So I have a movie. I play the movie in vlc player, I display the subtitles.

What I need is to take the displayed subtitles, change what they say and make them into an image overlay. So png RGBA I suppose, so I can overlay them onto a different scene. In other words I want to recreate the font and spacing and all the characteristics displayed in vlc.

Calling it up in ffmpeg: The movie contains the sub file as a .ass file. As well as a single .ttf attachment stream. All I know is that those .ttf streams contain formatting information that make the text look the way it does, I'm unsure what to do with it though. I can separate the .ass stream out of the container with ffmpeg and open it in a text editor. I look at it and I think this is the information I want to use to create new text in the style:

Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding

Style: Main,Open Sans Semibold,36,&H00FFFFFF,&H000000FF,&H00020713,&H00000000,-1,0,0,0,100,100,0,0,1,1.7,0,2,0,0,28,1

I don't know what to do with this information though.

In inkscape I create text and go through my fonts and apply a stroke with differing thickness and nothing comes close to the desired font, which seems to be the default vlc player subtitle font shared across many movies i tryout.

I get the feeling there is a much better and easier way to go about doing this. Can I create an overlay with this info as desired with ffmpeg? That would be great.

Edit: I tried this on the isolated subfile.ass, figuring I could just modify the .ass file with the new text I want if this worked. I got it to work, but the formatting is still a bit off. I need whatever is in the .ttf stream to do its thing here, not sure how to include it in the command.

ffmpeg -ss 00:01:87 -f lavfi -i "[email protected]:size=1920x1080,format=rgba,subtitles=subfile.ass:alpha=1" -frames:v 1 out.png

Edit: I read that the Stream # 0:3: Attachment: ttf is just a font style and doesn't contain positioning or anything other than teh font incase u don't already have it. So the solution would be to grab that file and put it in my windows font folder? I tried to grab it out using:

ffmpeg -i input.mkv -map 0:3 -c:t copy output.ttf 

But it says .ttf isn't the right extension to use. How do you export .ttf files out of a video?

Answer 1

Ok great. So 3 steps really: 1. Make sure you have the .ttf on ur computer installed, if not extract it from the streams

ffmpeg -dump_attachment:t:0 OpenSans-Semibold.ttf -i input.mkv

In my case I had 4 streams and the .ttf was my only t stream so its 0. And the filename to be used is the one listed in the stream info which is OpenSans-Semibold.ttf in my case.

2. Take the sub file, .ass in this case out of the video container. and save it as a .ass file

3. Use this command here on the subfile. Open the subfile in a text editor and see the timestamps, and just change the text for a specific timestamp with the formatting you want. And write the time of that timestamps in the -ss. Then after you render out ur output. Just change the .ass in the text editor to have the new text you want. Save. And repeat the command changing the output to something new each time.

   ffmpeg -ss 00:01:87 -f lavfi -i "[email protected]:size=1920x1080,format=rgba,subtitles=subfile.ass:alpha=1" -frames:v 1 out.png