When comparing and evaluating the 4K era video codecs 'HEVC', 'AV1' and 'VVC', which codec is most promising for the future?
Converting data or signals according to certain rules is called encoding, and converting them back to the original is called decoding, and the standard that performs this encoding and decoding is called a 'codec.' Videos are a series of still images, and adding audio inevitably results in a large file size, but codecs can compress the file size. Chips and Cheese, a hardware news site, has compared and verified the codecs ' HEVC ,' ' AV1 ,' and ' VVC ,' which emerged in an era when 4K resolution is becoming the norm.
Codecs for the 4K Era: HEVC, AV1, VVC and Beyond – Chips and Cheese
◆What are “HEVC”, “AV1”, and “VVC”?
The most mainstream video codec was MPEG-LA 's H.264/AVC (Advanced Video Coding). You can find out how AVC compresses movies by reading the following article.
How does the widely used video compression codec 'H.264/MPEG-4 AVC' dramatically compress huge movies? - GIGAZINE
To use AVC, a license fee had to be paid to MPEG-LA, but it became widely used after Cisco released OpenH264 under the BSD license . Later, HEVC (High Efficiency Video Coding), which can be considered the successor to AVC, appeared, but YouTube and Firefox decided not to adopt it due to license fees and legal issues. Therefore, AVC is still the mainstream codec even nearly 10 years after HEVC was introduced.
However, when trying to encode a resolution higher than 1080p using AVC, even if you lower the bit rate or frame rate, the file size will be large due to compression issues. Therefore, in today's world where 4K resolution video is becoming the norm, we have to say that H.264/AVC is a somewhat outdated codec.
In 2015, the Alliance for Open Media (AOMedia), an industry consortium led by Google, began developing 'AV1' as a proprietary codec to replace HEVC, and it was officially released in March 2018.
Google and Netflix's video codec 'AV1' officially released, expected to popularize 4K streaming - GIGAZINE
AV1 has a compression rate comparable to HEVC, but is developed as open source and can be used royalty-free. AV1 is already supported by YouTube, Chrome, and Firefox, and is expected to become one of the mainstream codecs that will replace AVC in the future. However, there are only a limited number of devices that can perform AV1 hardware encoding, and at the time of writing, there is still no way to perform AV1 hardware encoding on mobile devices such as smartphones.
And as the successor to HEVC, Versatile Video Coding (VVC) was released in 2020 by the Joint Video Experts Team (JVET). VVC is a codec that aims to improve compression efficiency by 30% to 50% compared to HEVC, supports streaming of 4K, 8K, and 16K resolutions, and is a competing standard with AV1.
MPEG-LA, which developed AVC and HEVC, has developed MPEG-5 Part 1 Essential Video Coding (EVC) as a license-friendly codec to replace HEVC. This EVC includes a royalty-free subset called Low Complexity Enhancement Video Coding (LCEVC), which has compression performance equivalent to HEVC.
◆ Software and materials, etc.
To compare codecs, Chips and Cheese used the most popular encoder/decoder, x265 3.5+95, for HEVC. For AV1, they used SVT-AV1 1.4.1, which offers a good balance between quality and speed, and VideoLAN's dav1d 1.1.0 for the decoder. For VVC, they used VVenC 1.7.0 for the encoder and VVdeC 1.6.1 for the decoder.
In addition, it seems that FFmpeg was used as a tool, but VVC was not implemented in FFmpeg at the time of writing, so a patch was used. In addition, the PC that performed the encoding and decoding was equipped with an AMD Ryzen 7950X , and it was run on a virtual machine using KVM . In addition, encoding was performed with preset settings, and the color depth was 10 bits.
The movie material used for the test was Blender's open movie 'Sintel', which is a 4K resolution movie encoded in AVC. You can watch what kind of movie Sintel is below.
Sintel - Open Movie by Blender Foundation - YouTube
In addition, Netflix's video quality assessment library, Video Multimethod Assessment Fusion (VMAF), is used to assess video quality. VMAF is a machine learning tool that predicts and quantitatively analyzes subjective video quality, and Chips and Cheese has an average VMAF score of 95, which is the highest quality.
Results
The graph below summarizes the compression efficiency of HEVC (blue), AV1 (orange), and VVC (green). The vertical axis is the VMAF score, and the horizontal axis is the bitrate. Looking at the VMAF score, it seems that VVC is able to achieve higher video quality at low bitrates. AV1 and HEVC are able to achieve higher video quality at low bitrates, and HEVC is able to achieve higher video quality at high bitrates.
Below are the times it took to encode and decode a video with a VMAF score of 95. HEVC had the shortest encoding time, at an overwhelming speed of 2 hours and 34 minutes. On the other hand, AV1 took 18 hours and 54 minutes to encode, and VVC took 2 days and 10 hours and 35 minutes. HEVC also showed the fastest decoding results.
And below is the quality evaluation of each frame of the video encoded at a bit rate of 1920 kbps using VMAF. We can see that the variation in quality evaluation from frame to frame is the smallest for HEVC and the largest for VVC. In other words, the result is that 'VVC has the highest compression efficiency and HEVC has the most stable average VMAF score.'
Does a stable VMAF score mean high video quality?
Below is a scene from the encoding material 'Sintel', where the source movie was re-encoded with x264. The 11506th frame in this scene had the lowest VMAF score in VVC and the 9th lowest VMAF score in HEVC.
Sintel - 11506 - Reference
Below is the same scene after encoding the source movie with HEVC and then re-encoding it with x264.
Sintel - 11506 - HEVC
This is what it looks like after encoding with AV1 and re-encoding with x264.
Sintel - 11506 - AV1
Below is the movie encoded with VVC and re-encoded with x264.
Sintel - 11506 - VVC
When evaluated frame by frame, the result is that 'HEVC has the highest video quality and VVC has the lowest,' but when you actually compare the video itself, the difference in image quality is almost negligible. This is because the encoder has a concept called Adaptive Quantization (AQ), says Chips and Cheese. For example, when the human eye is watching a video, it is more aware of the parts with a lot of movement than the parts with little movement. Also, even if the background is drawn in detail, it will look at the flatly drawn foreground. Therefore, the encoder uses AQ to prioritize processing the areas that the human eye pays attention to, improving the video quality of the subject itself at the expense of background details. Therefore, if you check the video quality for each frame, there will be a difference, but when humans look at the actual video, it feels like there is no big difference in image quality.
◆Evaluation
From the above results, Chips and Cheese argues that 'When bias is taken into account, for the same average VMAF score, the new codec achieves subjectively higher overall video quality due to the improvement in AQ. AV1 is roughly equivalent to HEVC in terms of compression efficiency, and sometimes even exceeds it. And VVC remains the best of the three codecs.' However, since VVC is still in development, the bottleneck is that there are only a limited number of players that can play VVC-encoded video, let alone hardware acceleration. In addition, VVC has the same patent issues as HEVC.
In addition, the biggest advantage of AV1, which was born between HEVC and VVC, is that it does not require a license fee, which is very attractive for commercial use, but AV1 performs very complex calculations to avoid patented technologies that AOMedia does not own, which causes encoding to take time and poor power efficiency. In addition, although it calls itself completely open source, intellectual property management company Sisvel has published a patent list and claimed that AOMedia is infringing on the patents of rights holder organizations, so it is unclear whether it will be able to maintain its 'completely open source and royalty-free' status in the future.
'If HEVC were not plagued by patents and licensing fees, we would be entering an era of HEVC dominance,' said Chips and Cheese. 'As a fan of free software, I strongly believe that progress and innovation should not be stifled in pursuit of profit through patents. The future looks bright when you look at the technology itself, but it is also mind-bogglingly complex.'
Related Posts:
