FFMPEG Batch Processing

Question

I have a ffmpeg command that will happily process all video files in a folder one by one, but how can i make it process them simultaneously?

here's my current command:

for i in *.mov; do name=``echo $i | cut -d'.' -f1``; echo $name; TIMECODE=$(ffprobe "$i" -v error -show_entries stream_tags=timecode -of default=noprint_wrappers=1:nokey=1 | awk -F: '{ print $1 "\\:" $2 "\\:" $3 "\\:" $4 }') ; ffmpeg -i "$i" -vf "trim=start_frame=192,setpts=PTS-STARTPTS,scale=1280:-1:force_original_aspect_ratio=decrease,drawtext=fontfile=/usr/share/fonts/truetype/droid/DroidSansMono.ttf: fontsize=28: timecode='00\:00\:00\:00': r=24: x=(w-tw)/2: y=25: fontcolor=white: box=1: boxcolor=0x00000099" -threads 4 -c:v mpeg1video -b:v 5000k -af "atrim=start=8,asetpts=PTS-STARTPTS" -c:a libmp3lame -b:a 192k "${name}_mpeg1.mpeg"

Answer 1

All that you need is to put an & at the end of the ffmpeg command line. This will make the ffmpeg process to run in background, allowing it to be executed simultaneously with other processes.

#!/bin bash
# add this code to a file called encode or whatever you want
# make it executable: chmod ugo+x encode
# and you are ready to run: ./encode

for i in *.mov; do 
    name=``echo $i | cut -d'.' -f1``; 
    echo $name; 
    TIMECODE=$(ffprobe "$i" -v error -show_entries stream_tags=timecode -of default=noprint_wrappers=1:nokey=1 | awk -F: '{ print $1 "\\:" $2 "\\:" $3 "\\:" $4 }') ;
    ffmpeg -i "$i" -vf "trim=start_frame=192,setpts=PTS-STARTPTS,scale=1280:-1:force_original_aspect_ratio=decrease,drawtext=fontfile=/usr/share/fonts/truetype/droid/DroidSansMono.ttf: fontsize=28: timecode='00\:00\:00\:00': r=24: x=(w-tw)/2: y=25: fontcolor=white: box=1: boxcolor=0x00000099" -threads 4 -c:v mpeg1video -b:v 5000k -af "atrim=start=8,asetpts=PTS-STARTPTS" -c:a libmp3lame -b:a 192k "${name}_mpeg1.mpeg" &;

    # limit ffmpeg processes to 5
    while [$(ps | grep ffmpeg | wc -l) -eq 5]; do
        echo "ffmpeg queue is full, waiting...";
        # amount of seconds to wait till we check again
        sleep 10;
    done
done;

Answer 2

Put it into a flexible wrapper script

There's a few benefits to doing it this way, for example, the provided script;

1. Uses find and array variables to handle complex and crazy filenames, and the process management
2. Records the process ID's as it spawns ffmpeg and then periodically checks them if they are active
3. If a file already exists in it's desired output form (i.e. file_mpeg1.mpeg), that file is skipped; this prevents saying running a loop of 100 files, failing for some reason after 90 files, and then re-running the whole thing from the start. (there is however no check at this stage to see if the existing file was complete or not. This goes a bit deeper into error checking / maybe a length compare based on original / an ffprobe, etc... )
4. The flexible template allows you easily manipulate the flow.

I tested it to process 4 random files downloaded off youtube:

As you can see it handles silly filenames quite well.

Here is the wrapper script based on your commands:

The syntax is: ./scriptname.sh [ext] [maxprocesses]

• So, to look to look for *.mov files and run 5 processes:
  • If the script name was script.sh, then you'd run
    • ./script.sh mov 5

Wrapper Script:

#!/usr/bin/env bash

ffprocess () {
    file="$1"
    output="$2" 
    TIMECODE=$(ffprobe "$file" -v error -show_entries stream_tags=timecode -of default=noprint_wrappers=1:nokey=1 | awk -F: '{ print $1 "\\:" $2 "\\:" $3 "\\:" $4 }')
    ffmpeg -i "$file" \
    -vf "trim=start_frame=192,setpts=PTS-STARTPTS,scale=1280:-1:force_original_aspect_ratio=decrease,drawtext=fontfile=/usr/share/fonts/truetype/droid/DroidSansMono.ttf: fontsize=28: timecode='00\:00\:00\:00': r=24: x=(w-tw)/2: y=25: fontcolor=white: box=1: boxcolor=0x00000099" \
    -threads 4 \
    -c:v mpeg1video \
    -b:v 5000k \
    -af "atrim=start=8,asetpts=PTS-STARTPTS" \
    -c:a libmp3lame -b:a 192k \
    "${output}"
}

#This function gets called when a process is no longer valid and needs removing from the array

removeps () {
    trtmp=()
    for tmpps in "${ffpsid[@]}"
    do
        [[ $tmpps != "$1" ]] && trtmp+=($tmpps)
    done
    ffpsid=("${trtmp[@]}")
    unset trtmp
    return
}

#Checks to see if each process is still active
checkffpsid () {
    for ffps in "${ffpsid[@]}" 
    do 
    ps -p $ffps > /dev/null || removeps "$ffps"
done

}


if [[ "$#" -lt 2 ]]; then echo -e "Usage: $0 [ext] [maxproc]\ni.e. $0 .mov 5" ; exit ; fi

ext="$1"
maxproc="$2"
find . -maxdepth 1 -iname "*.$ext" > inputlist.txt   # Find files with ext specified
IFS=$'\n' read -rd '' -a filelist <<<"$(cat inputlist.txt)" 
for file in "${filelist[@]}"
    do
    name=$(basename "$file")
    name="${name%.*}"                                  # remove ext for name var
    outname="${name}_mpeg1.mpeg"                       # Set output name
    checkffpsid 
    if [ -f "$outname" ]
        then 
        echo "$outname" already exists, skipping ...
    else
        until [[ "${#ffpsid[@]}" -le "$maxproc" ]]
        do
            checkffpsid
            sleep 5
        done
        ffprocess "$file" "$outname" & ffpsid+=("$!") 
    fi
done

---

Just a couple notes

1. Your "$TIMECODE" variable never gets used ... This is as per your original script.
2. You're better off letting ffmpeg manage its own threads by passing -threads 0 instead of 4