Remove static text from video?

Question

I have a video with text on the side of it and I want to remove it. The good thing is that the video has no changes in that area of the screen for the rest of the video. Meaning that the movement in the video is on the right side of the video while the left side, which has the text has no animation what so ever. Is it possible to remove that text.

Here is a link to the video also, the text is the "press any button" thing.

Thank you!

Answer 1

What you're asking for is called Inpainting and is difficult enough for still images. The result is passable to distracting depending on the size and transparency of the text, the surrounding image, and compression. I found only one mention of inpainting at video stack exchange here. You can see a demonstration of the algorithm here.

Using ffmpeg and mathematica, the following procedure is a quick and dirty solution. The result can be better than blurring, again depending on the characteristics of the video.

Export frames:

ffmpeg -i vid.mp4 D:\data\vid\frames-%06d.png

Create a text mask using the average of every 100 frames or so.

fnames = FileNames["D:/data/vid/frames-*.jpg"];
mfnames = fnames[[1 ;; All ;; Floor[0.005*Length[fnames]]]];
imgs = Import /@ mfnames;
mimg = Mean[imgs]
mask = Dilation[Binarize[mimg, {0.37, 1}], 1]

Note: the parameters in Binarize will need to be tweaked for every video.

If the video contains a fixed camera or other parts similar in color to the text throughout its entire length, you must exclude those parts by multiplying by a 2nd mask (the numbers are positions of the lower left and upper right corners of a rectangle that surround the text to be removed):

Image[Graphics[{
{Black, Rectangle[{0, 0}, ImageDimensions[mask]]},
{White, Rectangle @@ {{377.84411276948595`, 
    161.72470978441106`}, {163.44941956882255`, 
    127.76119402985057`}}},
}, PlotRangePadding -> None],
ImageSize -> ImageDimensions[mask]];
mask = mask*%

Apply inpaint algorithm:

Inpaint[Import[fnames[[1]]], mask]

This line processes all frames and saves them to a new directory:

Export[FileNameJoin[MapAt[# <> ".output" &, FileNameSplit[#], -2]], 
    Inpaint[Import[#], mask]] & /@ fnames

Or if you want it done 1000 times faster use this C++ code:

#include <iostream>
#include <vector>
#include <thread>
#include <opencv2/opencv.hpp>
#include <boost/filesystem.hpp>

// args: D:\data\text_remove\frames_full  D:\data\text_remove\frames_full\mask.png

using namespace std;
namespace fs = boost::filesystem;


void process_files(const vector<string>& infile, const vector<string>& outfile, const cv::Mat& mask, int i0, int i1)
{
    for (int i = i0; i < i1; ++i)
    {
        cv::Mat img = cv::imread(infile[i]);

        if (img.empty())
        {
            cout << "process_files(): imread(): " << "[" << i0 << "," << i1 << ")  : " << i1 - i0 << endl;
            break;
        }

        cv::Mat out;
        cv::inpaint(img, mask, out, 3, cv::INPAINT_NS);

        cv::imwrite(outfile[i], out);
    }
}

void threaded_process_files(const vector<string>& in, const vector<string>& out, const cv::Mat& mask, int threads)
{
    int step = in.size() / threads;
    int rem = in.size() % threads;

    vector<thread> pool;

    for (int i = 0; i < (threads - 1); ++i)
    {
        int i0 = i * step;
        int i1 = (i + 1) * step;
        pool.push_back(thread(process_files, in, out, mask, i0, i1));
    }

    int i0 = (threads - 1) * step;
    int i1 = in.size();
    pool.push_back(thread(process_files, in, out, mask, i0, i1));

    for (auto& th : pool)
        th.join();
}

int main(int argc, char** argv)
{
    if (argc < 2)
    {
        cout << "argv < 2: Missing input directory" << endl;
        return EXIT_FAILURE;
    }

    if (argc < 3)
    {
        cout << "argv < 3: Missing mask image" << endl;
        return EXIT_FAILURE;
    }

    cv::Mat mask(cv::imread(argv[2], cv::IMREAD_GRAYSCALE));

    if (mask.empty())
    {
        cout << "imread(): failed to read mask" << endl;
        return EXIT_FAILURE;
    }

    cout << "mask image: " << mask.size() << endl;

    fs::path infile(argv[1]);
    fs::path ifstem = infile.stem();
    fs::path ifleaf = infile.leaf();
    fs::path ifparent = infile.parent_path();

    if (!fs::is_directory(infile))
    {
        cout << infile << " is not a directory" << endl;
        return EXIT_FAILURE;
    }

    string fstr = ifleaf.string() + ".output";
    fs::path outdir = ifparent / fstr;

    if (!fs::exists(outdir))
    {
        if (!fs::create_directory(outdir))
        {
            cout << "create_directory() failed on: " << outdir << endl;
            return EXIT_FAILURE;
        }
    }

    vector<string> fnames;
    vector<string> fnames_out;

    clock_t t0 = clock();

    cout << "Scanning directory images" << endl;

    int i = 0;
    for (fs::directory_iterator di(infile); di != fs::directory_iterator(); ++di)
    {
        fs::path fpath = di->path();
        fs::path leaf = fpath.leaf();
        fs::path fpath_new = outdir / leaf;

        fnames.push_back(fpath.string());
    }

    cout << (double)(clock() - t0) / CLOCKS_PER_SEC << endl;
    cout << "Processing images" << endl;
    t0 = clock();

    threaded_process_files(fnames, fnames_out, mask, 16);

    clock_t t1 = clock();
    double dt = (double)(t1 - t0) / CLOCKS_PER_SEC;
    cout << dt << endl;
    cout << dt / fnames.size() << endl;

    return EXIT_SUCCESS;
}

Answer 2

Remove? No.

Best thing to do would be to remove the text from the original video project and re-render.

Barring that, you could cover it.

What I would do (and have done before) is pull the video into After Effects and cover it with a small area of blur. Once the blur is applied, pull the brightness down enough so that it blends with the surrounding area.

Alternative (and possibly better since the video would probably allow it) is to do essentially the same thing as above only to cover the small area with a dark rectangle shape in After Effects.