Yes, that's all! Of course, here all the constants are confused, never do so in real projects!

Yes, that's all! Of course, here all the constants are hardcoded, never do so in real projects!

Writing fast and dirty video filters

Sometimes we need to somehow filter the video, maybe put some text or graphic on top. Write a full application that decompresses the video stream, imbues the image with new elements or changes the image itself, and then compresses the stream anew, somewhat difficult. Today we will learn how to quickly and dirty make such applications without knowing how to encode the video in general.

To begin with, remember that the images consist of rows and columns of points, which are called pixels. Each point consists of 3 or 4 components, usually Red, Green and Blue signals, and sometimes there is an Alpha channel. Therefore, to change 1 point of the image, we need to change 3-4 components (this is usually 3-4 bytes). So, to change the entire image, we need to change all the rows and columns, in each of 3-4 elements and somehow modifying them, save back. For example, if we have a picture of 256x192 pixels in RGB24 format, then it will have 49152 pixels and 147456 color components (bytes).

Therefore, we just need to read pieces of 147456 bytes, somehow modify them and write them back! It's all! No complicated libraries or algorithms are required! Where do we get these bytes? And we can get them with wonderful utility ffmpeg and its equally wonderful rawvideo mode, the exhaust of which we will give to stdout and receive in stdin your application!

Let's write the simplest application that will slightly change the image:

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>

int main () {

// allocate variables and buffer

uint8_t *pixels = malloc (147456);
uint8_t *pix;
int q;

while (1) {

// read frame
if (fread (pixels, 1,147456, stdin) <= 0) {break;}

// process image
pix = pixels;
for (q = 0; q <49152; q ++) {
*pix++ = *pix * 2; // multiple red to 2
*pix++ = *pix + 120; // shift green channel
*pix++ = *pix + q / 10; // lines in blue channel
}

// write frame back
fwrite (pixels, 1,147456, stdout);
}

return EXIT_SUCCESS;
}

Yes, that's all! Of course, here all the constants are confused, never do so in real projects!

Now, run this and upload a video here:

ffmpeg -i video.mp4 -s 256x192 -f rawvideo -pix_fmt rgb24 - | ./a.out | ffmpeg -s 256x192 -f rawvideo -pix_fmt rgb24 -i - -y out.mp4

And we get the following result:

Let's analyze what parameters in this line participated:

ffmpeg -i video.mp4 -s 256x192 -f rawvideo -pix_fmt rgb24 -

Here we read from the file video.mp4, resize the picture to 256x192 (if your video is already 256x192 in size, then you can not do this), then we specify that it should be converted to rgb24 color space (if it is not already in it, what for can spend quite a lot of resources), and the result we need in the form of uncompressed video rawvideo, the result we record in -, i.e. in stdout.

| ./a.out |

And this is our application. Vertical "tubes" mean that it will take data from the previous application and transfer it to the next

ffmpeg -s 256x192 -f rawvideo -pix_fmt rgb24 -i - -y out.mp4

Here we indicate that the input of this ffmpeg will be uncompressed video in rawvideo format, size of 256x192 and in the rgb24 color space. It is also may need to specify fps, as it is lost from such conversion. We will write output to out.mp4, and even if such file already exists, the -y parameter is responsible for this

Here is a very simple example of changing the image, but you can overlay text, other pictures and anything else. Of course, it's better to write a full-fledged application, or at least patch the already existing filter to the desired state, this will be a more correct solution. But if you need a working solution, while you do not want to write a lot of code, this is the solution for you.