Creating a video from a text and an audio file

Question

I have a text and an audio file. I want to create a video that simply displays the text with the audio file playing in the background. I have to create many such videos, so I'm looking for an automated solution. What program/script could achieve that?

Answer 1

The Amara website offers a free service that accepts audio and text files, synchronizes them, and lets you download a .srt file (synchronized SubTitle). Source of suggestion: Photography TutsPlus.

YouTube also accepts text files without time coding as SubTitle files and then you can enable auto-sync, you ought to be able to download a .srt file after processing is completed (not tested by me).

---

To do it manually (with some automation) try the following.

If you convert the text file to a .ass or .srt file then you can use the following commands:

ffmpeg.exe -i subtitles.srt subtitles.ass

ffmpeg.exe -i video.mp4 -vf ass=subtitles.ass video-new.mp4

That will show the words hardsubbed on the video synchronized with the audio. The video can be a moving image (video) or a static image.

Using a .ass editor allows more control over color and other font features than using .srt files.

Without using a SubTitle Format file you won't get synchronized text.

You can add the audio to a video and upload it to YouTube, then wait 15 minutes, after which YouTube will have used audio recognition to convert the audio to subtitles (with better than 80% accuracy). Then you can download the .srt file from the Video Manager and edit the SubTitle file by comparing it to your (presumably accurate) text file.

If you were hoping for a program that automatically syncs raw text files with audio and makes a video I have found nothing by Googling (outside of pay services).

There are a few questions over at SO, but they remain unanswered.

I will do some more searching and improve this answer in a few hours.

Not a one-mouseclick solution but it's about as automated as I can suggest, for free (other than Amara).

[I welcome Mulvya to edit and improve this answer.]

Answer 2

I wrote a Python script create a video that simply displays the text with the audio file playing in the background.

Example of file structure before running the script (in this example, the three pairs of audio file + transcript):

C:\Users\Franck\Documents\GitHub\code>tree /f
Folder PATH listing
C:.
│   generate_videos.py
│
├───audio
│       file1.wav
│       file2.wav
│       file3.wav
│
└───transcripts
        file1.txt
        file2.txt
        file3.txt

After running the script (output.mp4 is the concatenation of videos\*.mp4):

C:\Users\Franck\Documents\GitHub\code>tree /f
Folder PATH listing
C:.
│   generate_videos.py
│   output.mp4
│
├───audio
│       file1.wav
│       file2.wav
│       file3.wav
│
├───images
│       file1.png
│       file2.png
│       file3.png
│
├───transcripts
│       file1.txt
│       file2.txt
│       file3.txt
│
└───videos
        file1.mp4
        file2.mp4
        file3.mp4

Code:

from PIL import Image, ImageDraw, ImageFont
import textwrap
from pathlib import Path
import glob
import os
import shlex
import subprocess

def draw_multiple_line_text(image, text, font, text_color, text_start_height):
    '''
    From unutbu on [python PIL draw multiline text on image](https://stackoverflow.com/a/7698300/395857)
    '''
    draw = ImageDraw.Draw(image)
    image_width, image_height = image.size
    y_text = text_start_height
    lines = textwrap.wrap(text, width=40)
    for line in lines:
        line_width, line_height = font.getsize(line)
        draw.text(((image_width - line_width) / 2, y_text),
                  line, font=font, fill=text_color)
        y_text += line_height


def generate_image(transcript_filepath, image_filepath):
    '''

    '''
    #image_width
    image = Image.new('RGB', (800, 600), color = (0, 0, 0))
    fontsize = 30  # starting font size
    font = ImageFont.truetype("arial.ttf", fontsize)
    #text1 = "I try to add text at the bottom of image and actually I've done it, but in case of my text is longer then image width it is cut from both sides, to simplify I would like text to be in multiple lines if it is longer than image width."
    #text2 = "You could use textwrap.wrap to break text into a list of strings, each at most width characters long"

    text1 = open(transcript_filepath, 'r').read()
    text_color = (200, 200, 200)
    text_start_height = 0
    draw_multiple_line_text(image, text1, font, text_color, text_start_height)
    text2 = transcript_filepath
    draw_multiple_line_text(image, text2, font, text_color, 400)

    image.save(image_filepath)


def create_folder_if_not_exists(directory):
    '''
    Create the folder if it doesn't exist already.
    '''
    if not os.path.exists(directory):
        os.makedirs(directory)

def get_basename_without_extension(filepath):
    '''
    Getting the basename of the filepath without the extension
    E.g. 'data/formatted/movie_reviews.pickle' -> 'movie_reviews'
    '''
    return os.path.basename(os.path.splitext(filepath)[0])

def run_command(command):
    print('command: {0}'.format(command))
    process = subprocess.Popen(shlex.split(command), stdout=subprocess.PIPE)
    while True:
        output = process.stdout.readline()
        print(output.strip())
        if output == b'' and process.poll() is not None:
            print('Done running the command.')
            break
        if output:
            print(output.strip())
    rc = process.poll()
    return rc

def main():
    '''
    Generate videos based on image+text
    '''
    transcript_file_extension = 'txt'
    speech_file_extension = 'wav'
    data_folder = 'transcripts'
    transcript_filepaths = list(glob.glob('{1}/**.{0}'.format(transcript_file_extension, data_folder)))
    transcript_filepaths.sort()
    speech_filepaths = list(glob.glob('{1}/**.{0}'.format(speech_file_extension, data_folder)))
    speech_filepaths.sort()
    print('transcript_filepaths: {0}'.format(transcript_filepaths))
    print('speech_filepaths: {0}'.format(speech_filepaths))

    output_folder = 'images'
    audio_folder = 'audio'
    video_output_folder = 'videos'
    create_folder_if_not_exists(output_folder)
    create_folder_if_not_exists(video_output_folder)
    output_folder = os.path.abspath(output_folder)
    print('output_folder: {0}'.format(output_folder))


    ffmpeg_concat_filepath = os.path.abspath('ffmpeg_concat_file.txt')
    ffmpeg_concat_file = open(ffmpeg_concat_filepath, 'w')
    for transcript_filepath  in transcript_filepaths:
        image_filename = get_basename_without_extension(transcript_filepath)
        image_filepath = os.path.join(output_folder, '{0}.png'.format(image_filename))
        generate_image(transcript_filepath, image_filepath)
        audio_filepath = os.path.join(audio_folder, '{0}.wav'.format(image_filename))
        video_filepath = os.path.join(video_output_folder, '{0}.mp4'.format(image_filename))
        os.system('ffmpeg -loop 1 -y -i {0} -i {2} -shortest {1}'.format(image_filepath, video_filepath, audio_filepath))
        ffmpeg_concat_file.write("file '{0}'\n".format(video_filepath))

    ffmpeg_concat_file.close()
    # [How to concatenate two MP4 files using FFmpeg?](https://stackoverflow.com/a/11175851/395857)
    # Note: the following command only works in Microsoft Windows. 
    # See the Stack Exchange link for the command for Linux and Mac OS X
    os.system('ffmpeg -safe 0 -f concat -i {0} -c copy output.mp4'.format(ffmpeg_concat_filepath))




if __name__ == "__main__":
    main()
    #cProfile.run('main()') # if you want to do some profiling

Tested with:

• Python 3.6
• Pillow 5.3.0
• ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers, built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)
• Windows 7 SP1 x64 Ultimate:

It should work on any operating system as long as ffmpeg is available (e.g., in the path on Microsoft Windows).

Video output example: