How to get peak audio quality and lowest video quality?

Question

What are the best options to use to get optimal audio clarity for transcription, and minimal video to be used by the transcription service?

Our videos are all meetings, and we're sending them to a service for transcription services - video quality is of no concern, but audio quality of the speakers is.

Everything I've found is a complicated discussion about channels/tracks/mapping, etc., and frankly everything is about achieving peak video at the same time, so I'm confused about which options work on which tracks, and at this point don't even know what I should be googling.

Clarification The source video can vary, depending on the circumstances of the meeting recorded. It can be an enormous mpeg or a smaller mp4 compressed "somehow". The ffmpeg output VIDEO will be very small, 320:240 or similar, but I need as much of the original AUDIO (improved if possible) as I can get into the output mp4 file for the transcription service to have the best we can provide them.

Answer 1

It's not entirely clear what you mean by improved audio, but here are three ffmpeg-based options that should get you what you need. Just combine the appropriate code below. I've included an audio conversion to high bitrate aac in all of the commands, but this will not actively improve your audio. If you're looking to optimize the audio for voice, check out ffmpeg's highpass, lowpass, and anequalizer audio filters to filter out unwanted high end and low end frequencies, and to enhance vocal frequencies (probably around 1khz and 5kz).

This line processes all of one file type in a folder. Just cd into the appropriate directory. Replace [original file extension] with your file extension, eg. for i in *.mov; do

mkdir output; for i in *.[original file extension]; do

Use that first line, plus one of the following

Option 1) -vn will ditch the video altogether. Consider just sending audio files.

ffmpeg -i [input] -vn -acodec aac -b:a 320k [output];done

Option 2) -vf scale will scale all your videos into a smaller size without messing with the audio.

ffmpeg -i [input] -vf scale=320:240 -crf 30 -acodec aac -b:a 320k [output];done

Option 3) The first line gets the duration of your video $t using ffmpeg. The second line creates black video to replace your video with, then uses -map to combine the new black video and the audio. Then it uses -ss and -t to set a start and end time $t. Replace [original file extension] with your file extension, eg. "$(basename $i .mov).mp4"

You should really only consider using this if your transcription service demands a video file, but either of the previous options is probably a better choice.

t=$(ffmpeg -i "$i" 2>&1 | grep "Duration"|grep -o '00:*.*'|cut -f1 -d","|awk -F: '{seconds=($1*60)*60; seconds=seconds+($2*60); seconds=seconds+$3; print seconds}');

ffmpeg -f lavfi -i color=c=black:s=2x2:r=1/1 -i "$i" -map 0:v -map 1:a -acodec aac -b:a 320k -ss 0 -t $t "output/$(basename $i .[original file extension]).mp4";done

Again, if your transcription service doesn't care about or even want video, you could shortcut all of this and just send them high quality wav files. ffmpeg -i [input.mp4] [output.wav]

Answer 2

I use this command to transcode video lectures primarily composed of static images with only the lecturer's voice overlaid.

ffmpeg -ss 00:06:00 -i input.mp4 -filter_complex '[0:v]fps=fps=1[v1],[v1]interlace[v2],[0:a]dynaudnorm[a]' -c:v libx265 -x265-params bframes=16 -crf 45 -c:a libopus -ac 1 -b:a 24K -ar 16K -map "[v2]" -map "[a]" output.mp4

explanation:

-ss 00:06:00: cut out the first 6:00 minutes from the video. This will obviously reduce the length of your video but also its size. I use it to leave out the duration between the meeting start and when the lecturer starts speaking. there are also -t and -to options to cut from the end of your video.

-i input.mp4: the input file. it is designated [0] when referenced afterwards. This is because some ffmpeg commands take multiple input files ffmpeg -i input1.mp4 -i input2.mp4 ... output.mp4 and they would be numbered as [0] [1] [2] ...

-filter_complex: I am going to define a set of filters applied on the audio and the video of my input file, separately.

[0:v]fps=fps=1[v1]: the first filter takes the video stream from input [0] and reduces it's framerate to 1 frame/second. I call the output video stream [v1] but, you could call it anything.

This will produce a laggy video output. I don't care about that since my videos are static images. I guess you wouldn't too since it's only important to identify the speaker's face. you should experiment a little with the different values of this filter. I recommend something between 0.5 and 10

[v1]interlace[v2]: This will take the video stream [v1], interlace it, and output a video stream called [v2]. Interlacing a video stream means that instead of displaying every line of pixels from that video on the screen every frame, each frame of video now displays a line of pixels and skips the next. and the human eye will make up the difference. the advantage of interlacing is that you could halve the frame rate (further) of your video stream without making the video appear more laggy.

[0:a]dynaudnorm[a]: will take the audio stream of your input file and normalize it. This filter lets ffmpeg decide how to normalize audio itself and allows it to change the characteristics of that audio. This is bad for music but, Ok for speech.

There are many other filters that I do not use but could be beneficial for other use cases. for example crop and cropdetect to crop the height and weight of your video (in case you have large black borders) reducing file size, scale to change height and width without cropping, afftdn, anlmdn, arnndn (Thread from Superuser), and the sox command (tutorial) for noise cancellation. since audio is the only important aspect of your file you could use the silencedetect filter to detect the places in your video where there is no audio. then cut these parts out with a second ffmpeg command.

-map "[v2]" -map "[a]": This tells ffmpeg to use [v2] video streams and [a] audio stream as the audio and video streams of the output file.

-c:v libx265: sets the video stream codec to x265. roughly speaking, codecs decide how bits are translated to video. Better codecs will translate more video details for fewer number of bits. x265 is among the more efficient codecs and transcoding your video from the old x264 to x265 could alone reduce video size by 50%.

-x265-params bframes=16: This is passed directly to the x265 encoder. I don't really understand what this parameter does. but I noticed that it more or less does the same thing as the fps filter. using both of them however reduces file size more than using either of them alone. 16 is the largest possible value for this parameter.

-crf 45: reduces video bitrate (bits/second) to achieve fixed quality. the higher this value, the lower the quality will be. the default value is 28 but I often use values between 40 and 51 (largest possible value) when I don't really care about video quality.

-c:a libopus: This will change the audio codec to Opus. Opus is a very efficient audio codec and has special setting for speech. it decides whether to use these settings or not based on bitrate and sampling rate settings which we discuss shortly.

Note that transcoding audio from a lossy codec (e.g. mp3, aac, opus) to another lossy codec will reduce the quality of your audio. you should not do this often if your audio contains music. human speech however can withstand several rounds of transcoding. This does not happen if you transcode audio from a lossless codec (e.g. wav, flac) to a lossy one. also, note that you can't apply filters (e.g. dynaudnorm) without transcoding.

-ac 1: This converts stereo audio (2 audio streams or 'channels', each for every ear) to mono audio (the same audio plays in both ears). This will halve the audio bitrate without affecting audio quality (unless your audio has stereo music, which is not the case in video meetings).

-b:a 24K: This limits the audio bitrate to 24000 bits. This is very sufficient for human speech if you use Opus. This value will vary according to the codec you use. List of possible bitrates for libopus

-ar 16K: This limits the audio sampling rate to 16000 Hz. This is sufficient (Maybe?) for human speech if you use Opus. List of possible sampling rates

output.mp4: output file name.

Answer 3

Just turn the video bitrate or the CRF, depending on what options your encoder gives you, way down.