The Goal
I'm creating short tutorials by recording my screen (720p30fps). The content is just a recording of using some desktop software - so mouse moving and opening dialogs, typing, drawing shapes, etc. No high speed action like in a game, no live footage, no talking head or anything like that. After basic editing of the original screen recording, I want to encode the videos to webm and mp4 formats at exactly the same resolution and framerate that I recorded and I want the playback quality to be as close to the original recording quality as possible. So there should be no visible compression artifacts after final encoding.
Regarding the original recording: Resolution is 1280x720 pixels. Framerate is 30 frames per second. Recording software is Simple Screen Recorder. Encoding is H.264 at Constant Rate Factor of 10 (so close enough to lossless) in a Matroska container. Playback of the original shows no artifacts at all.
Then I add some very basic effects like cross fades and text overlays to the original MKV using blender. Then render out to an image sequence (PNG format). I use high quality image export from Blender, and again the output images show no sign of compression artifacts.
The problem
The problem occurs after I encode the image sequence to an mp4 video using the x264 encoder with ffmpeg. Sometimes, some parts of some of the hard edges / shape outlines flicker. These are not moving objects. They are motionless in the background compared to where the mouse is moving. It seems random. For example, I could have two yellow rectangles side-by-side, each with a black outline and just the top part of one of the rectangle outlines flickers for a few frames, then stops. It's a bit distracting and ugly for what should be a really clean/easy encode based on a screen recording.
I generally see this for the horizontal dark lines / hard edges. Occasionally I've seen similar for vertical edges.
I haven't noticed this problem yet with the output webm files produced by ffmpeg, only with the x264 encoded mp4 files. The ffmpeg command I use is like this:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -pix_fmt yuv420p -movflags +faststart -crf 22 -r 30 -g 300 -y myvideo.mp4
For webm, I use the same basic command and the same crf value, but obviously a different encoder (libvpx-vp9). The webm file is about 9+MB for 2 minutes of footage. The mp4 is about 3.5MB. Generally, the quality between the two is impossible to distinguish, only at those times where the mp4 has some hard line segments flicker.
I've roughly illustrated the problem below using ASCII art, where I'm trying to show one rectangle across four frames. At first it's fine then for some frames part of rectangle's outline draws with some extra thickness in some parts, making it appear to flicker or jiggle a bit. Then it goes back to being perfect again.
-----------------
| |
-----------------
------=====------
| |
-----------------
______=====______
| |
-----------------
-----------------
| |
-----------------
Examples
Compare the following images. Notice in the second image the bottom 2 rectangles have an extra grey line appear above or below part of their black border. I captured these images while VLC was replaying my x264 encoded mp4 at 100% (no scale). This is not a really dramatic example, but it's hopefully enough to show whats going on. Because the extra lines appear and disappear, it creates a kind of flicker effect.
Notes:
- The recording was captured using Simple Screen Recorder on a 1280x720 screen with LibreOffice Draw maximised. You can't see artifacts in the original recording, nor in the PNG image sequence I exported from Blender.
- The command I used to generate the mp4 video was:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -coder ac -pix_fmt yuv420p -movflags +faststart -crf 22 -bf 16 -b_strategy 2 -refs 16 -direct-pred spatial -me_method umh -me_range 16 -r 30 -g 300 -y myvideo.mp4
(It's easier to notice the artifacts if you cycle between the two images at original size)
Questions
What is this kind of compression artifact called?
What flag can I pass ffmpeg to try suppress it?
What I have tried so far
Lots of experimentation with the CRF and speed values. For CRF I have dropped down to 18 and that does seem to reduce the flicker a bit, but not eliminate it. But that rate factor seems way too low to get a decent encoding of content as simple as a low action screen recording in 720p.
I have also tried manually setting b frames and reference frames to values between 1 and 16, varied the gop size between 30 and 300 and tried different values for the -me_method option. And I tried a couple of the deinterlace settings via the yadif option, but I'm fairly sure that's not relevant to this problem and it only seems to make the output video quality worse. I've ended up with ffmpeg commands looking more like this:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -coder ac -pix_fmt yuv420p -movflags +faststart -crf 22 -bf 16 -b_strategy 2 -refs 16 -direct-pred spatial -me_method umh -me_range 16 -r 30 -g 300 -y myvideo.mp4
But so far only varying the CRF value has had any noticeable effect. I don't want to drop that much lower, because it's going to blow the video size out too much.
Update: Test Results
I used the above example with LibreOffice Draw and a 1280x720p screen to record a clip of about 47 seconds to an H.264 encoded MKV file. I imported this to Blender and exported straight back out as a PNG image sequence- no effects or edits at all. Then I ran the image sequence through ffmpeg with the x264 encoder to produce the mp4 video file.
I started varying my original ffmpeg command one parameter at a time to see the difference. I focused on the CRF value mainly, because that was the only way I could definitely see a reduction in the compression artifacts. Even with the CRF set to 10, I could still see the weird flickering/banding on the edges in a couple of places. Finally I dropped the value to 5 and didn't notice any artifacts at all. The ideal value is possibly somewhere between 5 and 10 for this content.
That left me with video that's way too big for what it is. After messing around with B-frame and reference frame values to eke out marginal decreases in file size, I finally went back to varying the gop value and that had a relatively big impact. Setting the gop to 300 instead of 30, basically halved the file size. So my final ffmpeg command was this:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 5 -bf 16 -refs 16 -r 30 -g 300 -me_method umh -me_range 16 -y myvideo.mp4
(Update: I originally posted the above command as -crf 10... now corrected to -crf 5)
The main parameters I dropped were:
-pix_fmt yuv420p(it could degrade the quality)-coder ac(I couldn't see any positive impact with or without this)-b_strategy 2(this actually increased the file size slightly in some cases)-direct-pred spatial(absolutely no impact on the output file quality/size that I could observe)
If I was delivering the output file to YouTube, I'd consider putting back -pix_fmt yuv420p and -coder ac and -flags cgop, just because they recommend those.
These parameters yielded marginal improvement/decrease in file size (as in fractions of 1MB):
-bf 16-refs 16-me_method umh -me_range 16
You can see the variation in file size in the screen below where I have named each output file according the ffmpeg parameters I varied. Anything that doesn't have crf05 in the name manifests the artifacts problem.
As a comparison, I ran a similar command to produce a webm file, but with a much worse/higher CRF value. At a CRF value of 22 the resultant mp4 file was 1.4MB with no noticeable compression artifacts at all. The ffmpeg command I used was:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libvpx-vp9 -b:v 0 -tile-columns 2 -crf 22 -r 30 -g 300 -y myvideo.webm
Update: More Test Results
Following suggestions by stib and Mulvya, I tried the -tune and -trellis options and varied the crf value between 5 and 10. I did likewise for my own experiments with -me_method and -bf options. The output mp4, based on my input mkv video, all had visible, flickering compression artifacts at the same place until I turned the crf down to 6 when using -trellis, and down to 5 with the -bf -refs options or when using -tune. Here are screen shots showing the final bit of artifacting I’m trying to eliminate while using higher crf values. The first screen is the input mkv video, the second is a sample encoded mp4 video created using the -trellis 0 option. You can see the artifacts on the top border of one of the rectangles. (There is a thick gray band in the black background also – that’s not a compression artifact though, I think it’s a dialog in the process of vanishing after clicked cancel in my demo/sample recording.)
Here are the commands and resultant file sizes
-tune animation:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 5 -tune animation -r 30 -g 300 -y myvideo.mp4
Size: 2,757,370
Visible artifacts: No
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 6 -tune animation -r 30 -g 300 -y myvideo.mp4
Size: 2,593,711
Visible artifacts: Yes
-trellis 0:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 6 -pix_fmt yuv420p -x264opts trellis=0 -r 30 -g 300 -y myvideo.mp4
Size: 2,214,534 bytes
Visible artifacts: No
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 7 -pix_fmt yuv420p -x264opts trellis=0 -r 30 -g 300 -y myvideo.mp4
Size: 2,074,544 bytes
Visible artifacts: Yes (though a minuscule amount – literally a few pixels over a fraction of a second)
-fast_skip 0:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 7 -pix_fmt yuv420p -x264opts fast_pskip=0 -r 30 -g 300 -y myvideo.mp4
Size: 2,012,596 bytes
Visible artifacts: No
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 8 -pix_fmt yuv420p -x264opts fast_pskip=0 -r 30 -g 300 -y myvideo.mp4
Size: 1,883,919 bytes
Visible artifacts: Yes (very small/brief amount)
-psy 0 -fast_pskip 0:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 7 -pix_fmt yuv420p -x264opts psy=0:fast_pskip=0 -r 30 -g 300 -y myvideo.mp4
Size: 1,886,919 bytes
Visible artifacts: No
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 8 -pix_fmt yuv420p -x264opts psy=0:fast_pskip=0 -r 30 -g 300 -y myvideo.mp4
Size: 1,764,771 bytes
Visible artifacts: Yes
-bf -refs:
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 5 -bf 16 -refs 16 -r 30 -g 300 -me_method umh -me_range 16 -y myvideo.mp4
Size: 2,257,420 bytes
Visible artifacts: No
ffmpeg -speed 1 -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -crf 6 -bf 16 -refs 16 -r 30 -g 300 -me_method umh -me_range 16 -y myvideo.mp4
Size: 2,106,439 bytes
Visible artifacts: Yes (though a minuscule amount)
-deblock:
ffmpeg -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -preset veryslow -crf 14 -pix_fmt yuv420p -x264opts fast_pskip=0:psy=0:deblock=-3,-3 -r 30 -g 300 -y myvideo.mp4
Size: 982,549 bytes
Visible artifacts: No
ffmpeg -framerate 30 -f image2 -i frames/%04d.png -c:v libx264 -movflags +faststart -preset veryslow -crf 14 -pix_fmt yuv420p -x264opts fast_pskip=0:psy=0:deblock=-3,-3 -bf 16 -refs 16 -me_method umh -me_range 64 -r 30 -g 300 -y myvideo.mp4
Size: 961,460 bytes
Visible artifacts: No
Conclusion
Either the vp9 encoder is much better at handling this kind of video content, or there is some x264 parameter that I'm not aware of that would improve things. Does anyone have any ideas for an option I can give ffmpeg to get a better result with x264/mp4? (And I'm not trying to start some codec war - I need to publish in both formats!)
