2

Some mobile phones are able to compensate for low illumination conditions by integrating every two or more frames into one. The result is a video with variable frame rate, where each image has its particular timestamp. This kind of footage shows correctly illuminated subjects, but you notice the drop in frame rate because the movement is not fluid and also the images become noisier (because you are adding too the noise produced by the electronics). Anyway it seems a technological miracle for me, taking into account that the entrance pupil of such devices may not be larger than 1 mm in diameter.

But let's say you forced the phone to record at a fixed frame rate (e.g. 1280x720p30 with OpenCamera in Android). It should be possible to convert for instance 30p to 15p by adding the values stored in the corresponding pixels of each two frames into one. I know this process is possible because it is usual in other contexts, e.g. dealing with astronomical images.

My tools are ffmpeg and Blender. I would be thankful if someone can restrict the answer to these tools, though other solutions are also welcome because they might put me in the right track to look for suitable alternatives.

Improve this question

3 Answers 3

Reset to default
1

As you note, just adding (integrating) each pair of adjacent frames to make one new frame decreases the frame rate.

But if all you want to do is integrate adjacent frames, consider duplicating the video into a separate timeline, shifted by exactly one frame. Integrating these two streams would give you a new video with the same frame rate, but shorter by one frame overall.

All the rest of the noted issues might surface, but at least the frame rate wouldn't change.

Improve this answer
2
  • This is a clever idea, but have you tested it? I'd better interested if there's any unexpected results.
    – user3643
    Oct 27, 2018 at 3:27
  • @DigiVisionMedia - No, I haven't tested it. If I had the means to test it I would, as anyone could. As I said, any of the other issues mentioned might surface, and I'm sure that 'success' would be content-dependent. But it would accomplish the stated goal without changing the frame rate.
    – Jim Mack
    Oct 27, 2018 at 13:38
0

My own, not fully satisfactory answer that might be helpful:

This very simple line does the trick:

ffmpeg -i input vf "tblend=addition,framestep=2" {encoding options} output

E.g.

ffmpeg -i input.mp4 -vf "tblend=addition,framestep=2" -c:v libx264 -crf 16 -c:a copy output.mp4

input.mp4 was variable frame rate <=30, whereas output.mp4 is 15 fps. I could have specified a constant frame rate for the output file, e.g.

ffmpeg -i input.mp4 -vf "tblend=addition,framestep=2" -c:v libx264 -r 30 -crf 16 -c:a copy output.mp4

I will not accept my own answer, because the result of this process is an ugly superposition of the moving objects between consecutive frames, so that some kind of adaptive blurring is needed, hence I leave the question open for someone to give a better solution.

Improve this answer
5
  • 1
    You asked for adding the values stored in the corresponding pixels of each two frames and that's what your method does. This will result in "ugly superposition". You may just want to use a filter to increase brightness.
    – Gyan
    Nov 30, 2017 at 4:52
  • 1
    Mulvya is the resident ffmpeg expert. -- Decreased Framerate allows the shutter to remain open longer gathering more light and astronomical image stacking relies on the extremely slow movement of the objects in question. --- With an existing video and motion each Frame has to be good enough there's nothing to add or stack. You can use curves to brighten only the dark portions, leaving the bright pixels alone. Does this answer work for you: video.stackexchange.com/a/13110/18314 ?
    – Rob
    Nov 30, 2017 at 5:47
  • @Mulvya Increasing brightness produces very ugly results because of discretization noise. There must be some solution including something similar to frame interpolation or some kind of directional adaptive filter or something.
    – Mephisto
    Nov 30, 2017 at 7:52
  • @Rob Mmm I think you are right. Probably the phone is not adding frames, but rather integrating more time in each frame (and shooting less frames per second too).
    – Mephisto
    Nov 30, 2017 at 7:56
  • You can try to improve tblend method by stabilization image. But don't expect to much, your "low signal across few images" getting distorted by compression (quantization and so on).
    – bukkojot
    Dec 2, 2017 at 9:34
0

With physics! All you have to do is record at 15 frames per second in the first place and use a shutter speed of 1/15th of a second (aka, a 360 degree shutter angle). This effectively does the same thing as combining two 1/30th second, 30 fps frames with software, it just does it by letting light hit the sensor for twice as long in the first place. No fancy math or computations necessary!

Improve this answer
3
  • Please note the words existing video, in the title of the question. Which means that I cannot shoot anything. It comes as it is. Also, it is clear at several points in the question text that it is video recorded with a phone. No fancy options further than the frame rate and only after installing a separate app (that is also mentioned in the text of the question) called OpenCamera.
    – Mephisto
    Nov 30, 2017 at 4:28
  • 1
    It was unclear to me whether you were actually asking about a phone, or using it as an example. Specifically, the language, "some mobile phones" and "let's say you" sound hypothetical. In fact, I completely disagree that "it is clear in several points in the question text that it is video recorded with a phone" because of this vagueness. Maybe list smartphone w/ tools? I admit that I may have overlooked the word "existing" in the title, but since there is no further emphasis on the uniqueness, criticality, authenticity, or whatever of your footage, I assumed it could simply be re-shot.
    – Jason Conrad
    Dec 1, 2017 at 15:07
  • Furthermore, the only "fancy option" that I suggested you change was frame rate. If your camera (phone)/ software combination prohibits you from changing camera settings, you should mention that in the question and not assume that everyone knows what OpenCamera is, or force everyone to guess what model phone you have (because you haven't mentioned). But if it truly is the case that you can't change these settings without installing a separate app, maybe it's time for a new phone anyway.
    – Jason Conrad
    Dec 1, 2017 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.