H.265 video encoding

Coding history

Regarding the history of H.265 video coding, I will start with the two major video coding organizations.

The first organization is MPEG, the English full name is Moving Picture Experts Group, translated into Chinese is “Moving Picture Experts Group”. This organization was established in 1988, dedicated to the development of video and audio compression coding technology. Many standards such as MPEG-1, MPEG-2, MPEG-3, MPEG-4, and MPEG-7 have been formulated so far, and the latest MPEG-21 is being formulated.

The standards developed need to be accepted and recognized by all manufacturers and institutions, and cooperation with the government is required. Therefore, the name of MPEG is ISO/IEC JTC 1/SC 29/WG 11 – Coding of moving pictures and audio (ISO/IEC Joint Technical Committee 1, Subcommittee 29, Working Group 11), abbreviated as ISO/IEC Moving Picture Experts Group (MPEG), where ISO is the International Organization for Standardization and IEC is the International Electrotechnical Commission.

The second organization that conducts video coding research is ITU-T, the telecommunications standardization department under the International Telecommunication Union. The International Telecommunication Union (ITU) has a characteristic that any standard can be formulated, and it is highly authoritative. Many international standards require its approval. ITU-T generally uses a capital letter to classify the standards, and it is recommended to classify them as series. For example, the H series is a composite method for video, audio and multimedia systems. Below are H.261, H.262, H.263, H.264, H.265 and other serial numbers.

ISO/IEC MPEG cooperates with ITU-T, and the common coding standards that have been launched are H.262/MPEG-2, H.264/MPEG-4 AVC and H.265/MPEG-H.

Among them, H.265, also known as High Efficiency Video Coding (HEVC), is part 2 of MPEG-H and is regarded as the successor of the ITU-T H.264/MPEG-4 AVC standard.

H.265 encoding was formulated by ISO/IEC MPEG and ITU-T in 2004. The first version of the HEVC/H.265 video compression standard was accepted as the official standard of ITU-T on April 13, 2013.

Technical details

Like H.264 video coding, H.265 adopts the architecture of Hybrid Video Coding, but some new technologies are added to each part or the efficiency of the original coding tools is improved.

Coding tree unit

H.265 supports coding tree units from 8×8 to 64×64 pixels. The coding tree unit may divide the coding unit, the prediction unit, and the conversion unit downward.

Generally speaking, a larger conversion unit can better encode static signals, and a smaller conversion unit can better encode smaller “pulse” signals. In this way, the amount of data is relatively smaller.

Intra-frame coding

One H.265 coding unit can use one of eight prediction modes for prediction, and there are 35 different intra prediction modes (H.264 supports 9).

Inter-frame coding

For motion vector prediction, H.265 is more complicated than H.264. H.265 has two reference tables: L0 and L1, each of which has 16 reference items.

Transcoding

In the encoding process, the prediction unit is the basic unit for prediction, and the transformation unit is the basic unit for transformation and quantization. The separation of these three units makes each processing link of transformation, prediction and coding more flexible.

Loop filter

H.265 has two loop filters, a deblocking filter (DBF) and a sample adaptive offset (SAO) filter.

Unlike H.264 which implements deblocking on 4×4 blocks, H.265 can only implement deblocking on 8×8 grids. This allows parallel processing of deblocking (no filter overlap). The first thing to be deblocked is all the vertical edges in the picture, followed by all the horizontal edges. Use the same filter as H.264.

Entropy coding

The same CABAC (adaptive binary arithmetic coding referred to above) algorithm is used for entropy coding as in H.264, and CAVLC is abandoned.

Parallel coding tool

• Slice
• Tile
• Wavefront Processing

Coding Standards

H.265 defines two “Profiles” including Main and High, and 13 “levels”.

The hierarchy is designed to face different applications and limit its maximum bit rate. Among them, Main is designed for most applications, and High is designed for high-demand applications. The level is a set of restrictions for the bitstream, similar to the profile. When a decoder meets a given level and level, it means that the decoder must also be able to decode the bit stream encoded with that level/level and lower levels/levels.

Coding efficiency

The higher the resolution, the higher the coding efficiency.

The following table shows the ratio of code rate reduction of H.265 encoding at different resolutions compared with H.264 obtained by Qualcomm through experiments:

Resolution	480P	720P	1080P	4K
H.265	-52%	-56%	-62%	-64%

Patent issues

Any equipment manufacturer or H.265 content provider that uses H.265 encoding technology is required to pay a patent license fee.

H.265 currently has 4 patent fee collection organizations, 3 major patent pools plus Technicolor. The three major patent pools are HEVC Advance, Velos Media and MPEG LA.

Industry Application

In the security industry, H.265 encoding has become the mainstream. Network cameras, NVRs, and DVRs have all adopted H.265 encoding and compression on a large scale, and the old H.264 encoding products have been gradually eliminated. Huawei HiSilicon provides many models of DSP for network cameras and SOC for DVR and NVR that support H.265 encoding.

The H.265 patent mentioned above, Huawei has the most core patents. In the field of security, its core technologies for H.265 encoding are SmartP and AVBR technologies. SmartP means intelligent P-frame (forward prediction frame) predictive coding, and AVBR technology means advanced dynamic bit rate control technology.

Many security manufacturers have developed the H.265 encoding technology, and proposed such as Smart H.265, h.265+, H.265++, H.265x, super H.265, etc. The core is still inseparable from H. 265 encoding technology.

A more interesting phenomenon is that although H.265 coding technology has been used on a large scale in the security industry, the rate of using H.265 in the traditional Internet video field is still relatively low. On the one hand, it may be because the security industry is relatively closed and the industrial chain is relatively single, not as open as the Internet video field. There are many.

Other codes

• H.266
• AV1
• VP9