0

I have been researching this for the past couple of days, and all I could find on the subject were a bunch of research papers, a demonstration video of a model that isn't publicly available, and a software that you have to pay hundreds for to make it useful, on a poorly designed and shady looking website. Everything else is for JPEG artifacts, de-noising (a completely separate issue), and video up-scaling. Either that, or it's just questions/answers on how to solve the issue from the source (camera, recording software, or editing software) rather than how to fix existing footage.

I am looking for a software, preferably AI based, that can remove and repair highly compressed videos their compression artifacts. (Discoloration, macro-blocking, banding, loss of detail, etc...) without a serious loss of existing detail or flickering between frames (Caused by frame-by-frame editing).

Does this actually exist? It really seems like it doesn't, and relevant information is hard to find.

Improve this question
2

1 Answer 1

Reset to default
3

If I understand you correctly, then most of the restoration goals will be impossible to be reached. While my knowledge of existing videos tools is not extensive, I am still pretty sure such a software does not exist. As a software developer I have a good grasp of what is impossible and what is possible and what effort you need to do that.

When details are lost, they are lost. How should an AI restore these? Well, any AI has to be trained first and this needs to be done with pairs of a master video in highest quality and its heavily-compressed export in the target codec. The problem is that the export video has less information than the master and there is no way to define a sensible bidirectionaly mapping. Imagine the master contains a scene with a machine with little screws. Another scene shows a machine with rivets. But after a heavily compression you cannot see a difference between screws and rivets in the export video. Due to loss of detail, both screws and rivets now look like grey smudges. How should you decide what is correct if you only have the compressed video? You can only make a guess. And that's what the AI would do. This means you get a restored video that often shows quite different details than the original video, however now and then the AI will make correct guesses too. This is of course theoretical, I highly doubt there is an AI based software for this job as it would be extremely expensive in development and with only a few potentional customers the price tag would be enormous.

Discolouration is also hard to solve because often there's no easy way to differentiate between intended de-saturated areas and compression artifacts.

Macro-blocking can be fixed in theory by applying a blur effect. But you have to sacrifice sharpness. Fine artifical noise can be added to gain a bit of sharpness again. Anyway, this can be done and such a function is probably available in some tools.

Colour bending can be solved in theory without too much effort but I don't know if anybody does offer such a thing.

Update: As has been pointed out in the comments, DaVinci Resolve can do de-bending. Both Premiere and Resolve use some AI based tools to restore a video to fix dust, scratches, de-interlace. But if I infer correctly this does not help much with your problem of restoring a heavily compressed video.

All in all, you cannot undo the artifacts of heavy compression.

Improve this answer
5
  • That's why the ai/machine-learning tech is what I'm looking for. As I understand it, they can be trained by providing a highly compressed video, and the original HD version, which means the results would be much more accurate than a simple algorithm, depending on how many examples its been trained with. This kind of thing exists for JPEG artifacts on images, and it works pretty well. Of course you can't expect it to be 100% accurate (what it would have looked like naturally) but something like that would be quite helpful when complete accuracy isn't necessary. Thanks for the info, though
    – Rylan
    Aug 17, 2020 at 0:36
  • This answer is simply incorrect. DaVinci Resolve and Premiere both already use machine learning approaches for image restoration. They both use it for object removal. Resolve uses it to fix dust and scratches. Resolve has a de-banding and de-flickering algorithm, although I don't think that one uses machine learning, in particular. Resolve also uses ML to de-interlace. The tools are out there, and they'll only get better with time, they're just not as automatic as some would expect, and still require guidance from the user.
    – Jason Conrad
    Jan 9, 2021 at 6:14
  • @Jason Conrad You have a point and I'll update my answer a bit. But I'm not entirely convinced. Restoring a heavy compressed video such that it looks almost the same as the master is an incredible complex task. If it can be done, please write an answer where you explain how you do it. However, if it's impossible now but might be in the far future, this does not help the OP much. From what I know, people working on AI keep promising too much (and have done so for decades) and generally fall far behind their promises. So I'm not as optimistic as you.
    – Matt
    Jan 11, 2021 at 21:09
  • idle.slashdot.org/story/21/02/07/0626249/…
    – Jason Conrad
    Feb 8, 2021 at 4:24
  • Thanks for the link. Looking of one of the restored cartoon I clearly see the improvement and even have an idea how they have done it in general. However, the difficult challenge wasn't tackled: there is still a lot of dirt/dust visible. The images are shaky, but I guess this can be mitigated with already existing stabilizing effects. Still wondering why they haven't done it. Nevertheless, it's a progress. But I'm sceptical if this works on non-cartoon video with digital artefacts resulting from high compression.
    – Matt
    Feb 8, 2021 at 7:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.