I have a text and an audio file. I want to create a video that simply displays the text with the audio file playing in the background. I have to create many such videos, so I'm looking for an automated solution. What program/script could achieve that?
-
So, subtitling, basically?– GyanDec 9, 2017 at 15:55
-
@Mulvya I guess that could be one approach, one difference being that we don't have any video as input.– Franck DernoncourtDec 10, 2017 at 1:42
-
ffmpeg can do it. Like Mulvya says I'd use subtitles if you want the te\xt to change over the course of the video.– stibDec 10, 2017 at 6:03
-
ffmpeg can create a blank video as canvas. I mentioned subtitling because I assume you want the displayed text to sync with the audio. If not, you can scroll the text - see superuser.com/q/1273424/114058– GyanDec 10, 2017 at 6:12
2 Answers
The Amara website offers a free service that accepts audio and text files, synchronizes them, and lets you download a .srt file (synchronized SubTitle). Source of suggestion: Photography TutsPlus.
YouTube also accepts text files without time coding as SubTitle files and then you can enable auto-sync, you ought to be able to download a .srt file after processing is completed (not tested by me).
To do it manually (with some automation) try the following.
If you convert the text file to a .ass or .srt file then you can use the following commands:
ffmpeg.exe -i subtitles.srt subtitles.ass
ffmpeg.exe -i video.mp4 -vf ass=subtitles.ass video-new.mp4
That will show the words hardsubbed on the video synchronized with the audio. The video can be a moving image (video) or a static image.
Using a .ass editor allows more control over color and other font features than using .srt files.
Without using a SubTitle Format file you won't get synchronized text.
You can add the audio to a video and upload it to YouTube, then wait 15 minutes, after which YouTube will have used audio recognition to convert the audio to subtitles (with better than 80% accuracy). Then you can download the .srt file from the Video Manager and edit the SubTitle file by comparing it to your (presumably accurate) text file.
If you were hoping for a program that automatically syncs raw text files with audio and makes a video I have found nothing by Googling (outside of pay services).
There are a few questions over at SO, but they remain unanswered.
I will do some more searching and improve this answer in a few hours.
Not a one-mouseclick solution but it's about as automated as I can suggest, for free (other than Amara).
[I welcome Mulvya to edit and improve this answer.]
I wrote a Python script create a video that simply displays the text with the audio file playing in the background.
Example of file structure before running the script (in this example, the three pairs of audio file + transcript):
C:\Users\Franck\Documents\GitHub\code>tree /f
Folder PATH listing
C:.
│ generate_videos.py
│
├───audio
│ file1.wav
│ file2.wav
│ file3.wav
│
└───transcripts
file1.txt
file2.txt
file3.txt
After running the script (output.mp4
is the concatenation of videos\*.mp4
):
C:\Users\Franck\Documents\GitHub\code>tree /f
Folder PATH listing
C:.
│ generate_videos.py
│ output.mp4
│
├───audio
│ file1.wav
│ file2.wav
│ file3.wav
│
├───images
│ file1.png
│ file2.png
│ file3.png
│
├───transcripts
│ file1.txt
│ file2.txt
│ file3.txt
│
└───videos
file1.mp4
file2.mp4
file3.mp4
Code:
from PIL import Image, ImageDraw, ImageFont
import textwrap
from pathlib import Path
import glob
import os
import shlex
import subprocess
def draw_multiple_line_text(image, text, font, text_color, text_start_height):
'''
From unutbu on [python PIL draw multiline text on image](https://stackoverflow.com/a/7698300/395857)
'''
draw = ImageDraw.Draw(image)
image_width, image_height = image.size
y_text = text_start_height
lines = textwrap.wrap(text, width=40)
for line in lines:
line_width, line_height = font.getsize(line)
draw.text(((image_width - line_width) / 2, y_text),
line, font=font, fill=text_color)
y_text += line_height
def generate_image(transcript_filepath, image_filepath):
'''
'''
#image_width
image = Image.new('RGB', (800, 600), color = (0, 0, 0))
fontsize = 30 # starting font size
font = ImageFont.truetype("arial.ttf", fontsize)
#text1 = "I try to add text at the bottom of image and actually I've done it, but in case of my text is longer then image width it is cut from both sides, to simplify I would like text to be in multiple lines if it is longer than image width."
#text2 = "You could use textwrap.wrap to break text into a list of strings, each at most width characters long"
text1 = open(transcript_filepath, 'r').read()
text_color = (200, 200, 200)
text_start_height = 0
draw_multiple_line_text(image, text1, font, text_color, text_start_height)
text2 = transcript_filepath
draw_multiple_line_text(image, text2, font, text_color, 400)
image.save(image_filepath)
def create_folder_if_not_exists(directory):
'''
Create the folder if it doesn't exist already.
'''
if not os.path.exists(directory):
os.makedirs(directory)
def get_basename_without_extension(filepath):
'''
Getting the basename of the filepath without the extension
E.g. 'data/formatted/movie_reviews.pickle' -> 'movie_reviews'
'''
return os.path.basename(os.path.splitext(filepath)[0])
def run_command(command):
print('command: {0}'.format(command))
process = subprocess.Popen(shlex.split(command), stdout=subprocess.PIPE)
while True:
output = process.stdout.readline()
print(output.strip())
if output == b'' and process.poll() is not None:
print('Done running the command.')
break
if output:
print(output.strip())
rc = process.poll()
return rc
def main():
'''
Generate videos based on image+text
'''
transcript_file_extension = 'txt'
speech_file_extension = 'wav'
data_folder = 'transcripts'
transcript_filepaths = list(glob.glob('{1}/**.{0}'.format(transcript_file_extension, data_folder)))
transcript_filepaths.sort()
speech_filepaths = list(glob.glob('{1}/**.{0}'.format(speech_file_extension, data_folder)))
speech_filepaths.sort()
print('transcript_filepaths: {0}'.format(transcript_filepaths))
print('speech_filepaths: {0}'.format(speech_filepaths))
output_folder = 'images'
audio_folder = 'audio'
video_output_folder = 'videos'
create_folder_if_not_exists(output_folder)
create_folder_if_not_exists(video_output_folder)
output_folder = os.path.abspath(output_folder)
print('output_folder: {0}'.format(output_folder))
ffmpeg_concat_filepath = os.path.abspath('ffmpeg_concat_file.txt')
ffmpeg_concat_file = open(ffmpeg_concat_filepath, 'w')
for transcript_filepath in transcript_filepaths:
image_filename = get_basename_without_extension(transcript_filepath)
image_filepath = os.path.join(output_folder, '{0}.png'.format(image_filename))
generate_image(transcript_filepath, image_filepath)
audio_filepath = os.path.join(audio_folder, '{0}.wav'.format(image_filename))
video_filepath = os.path.join(video_output_folder, '{0}.mp4'.format(image_filename))
os.system('ffmpeg -loop 1 -y -i {0} -i {2} -shortest {1}'.format(image_filepath, video_filepath, audio_filepath))
ffmpeg_concat_file.write("file '{0}'\n".format(video_filepath))
ffmpeg_concat_file.close()
# [How to concatenate two MP4 files using FFmpeg?](https://stackoverflow.com/a/11175851/395857)
# Note: the following command only works in Microsoft Windows.
# See the Stack Exchange link for the command for Linux and Mac OS X
os.system('ffmpeg -safe 0 -f concat -i {0} -c copy output.mp4'.format(ffmpeg_concat_filepath))
if __name__ == "__main__":
main()
#cProfile.run('main()') # if you want to do some profiling
Tested with:
- Python 3.6
- Pillow 5.3.0
- ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers, built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)
- Windows 7 SP1 x64 Ultimate:
It should work on any operating system as long as ffmpeg is available (e.g., in the path on Microsoft Windows).
Video output example: