Bagged Tree And ResNet Based Joint End-to-End Fast CTU Partition Decision Algorithm For Video Intra Coding

In the development of video coding standards, advanced ones have greatly improved the bit rate compared with those of previous generation, but also brought a huge increase in coding complexity. Coding standards, such as high efficiency video coding (HEVC), versatile video coding (VVC) and AOMedia video 2 (AV2), get the optimal encoding performance by traversing all possible combinations of coding unit (CU) partition and selecting the combination with minimum coding cost. This process of searching for the best makes up a large part of encoding complexity. To reduce the complexity of coding block partition for many video coding standards, this paper proposes an end-to-end fast algorithm for partition structure decision of coding tree unit (CTU) in intra coding. It can be extended to various coding standards with fine tuning, and is applied to the intra coding of HEVC reference software HM16.7 as an example. In the proposed method, the splitting decision of a CTU is made by a well designed bagged tree model firstly. Then, the partition problem of a 32×32 sized CU is modeled as a 17-output classification task and solved by a well trained residual network (ResNet). Jointly using bagged tree and ResNet, the proposed fast CTU partition algorithm is able to generate the partition quad-tree structure of a CTU through an end-to-end prediction process, instead of multiple decision making procedures at depth level. Besides, several effective and representative datasets are also conducted in this paper to lay the foundation of high prediction accuracy. Compared with the original HM16.7 encoder, experimental results show that the proposed algorithm can reduce the encoding time by 59.79% on average, while the BD-rate loss is as less as 2.02%, which outperforms the results of most of state-of-the-art approaches in the fast intra CU partition area.

Fast CU Partition Decision Based on Texture for H.266/VVC

With the development of multimedia equipment and the increasing demand for high-quality video applications, the traditional video coding standard, H.265/High Efficiency Video Coding (HEVC), can no longer effectively satisfy the requirements. To promote the development of high-quality video, a new generation video coding standard, H.266/Versatile Video Coding (H.266/VVC), is established, and it is the inheritance and development of H.265/HEVC. It not only retains many mature technologies and methods in HEVC but also adds some new coding tools, such as wide-angle prediction and Multitype Tree (MTT) partition structure. The MTT partition structure brings a more flexible partition method of Coding Unit (CU), but the accompanying increase in computational complexity is unacceptable. In order to ensure an effective balance between coding efficiency and coding quality, a fast CU partition algorithm based on texture is proposed in this paper. First, the texture complexity of the neighboring CU is used as a threshold for evaluating the complexity of the current CU, so as to skip the unpromising depth. Then, the gradient features are extracted to determine whether the Quad-Tree (QT) partition is executed. Finally, the improved Canny operator is used to extract edge features, and the partition mode in the horizontal or vertical direction is excluded. The algorithm was embedded in VTM7.0, and the video sequences with different resolutions were tested under general experimental configuration. Simulation experiment results show that the average time saving of this method reached 50.56% compared with the anchor algorithm. At the same time, the average BDBR is increased by 1.31%.

An Early CU Partition Mode Decision Algorithm in VVC Based on Variogram for Virtual Reality 360 Degree Videos

360-degree videos have become increasingly popular with the development of virtual reality (VR) technology. These videos are converted to a 2D image plane format before being encoded with standard encoders. To improve coding efficiency, a new generation video coding standard has been launched to be known as Versatile Video Coding (VVC). However, the computational complexity of VVC makes it time-consuming to compress 360-degree videos of high resolution. The diversity of CU partitioning modes of VVC greatly increases the computational complexity. Through statistical experiments on ERP videos, it is found that the probability of using horizontal partitioning for such videos is greater than that of vertical partitioning. The empirical variogram combined with Mahalanobis distance is proposed to measure texture orientation information. The experimental results show that the algorithm saves 32.13% of the coding time with only 0.66% BDBR increasing.

CNN-LSTM Learning Approach-Based Complexity Reduction for High-Efficiency Video Coding Standard

High-Efficiency Video Coding provides a better compression ratio compared to earlier standard, H.264/Advanced Video Coding. In fact, HEVC saves 50% bit rate compared to H.264/AVC for the same subjective quality. This improvement is notably obtained through the hierarchical quadtree structured Coding Unit. However, the computational complexity significantly increases due to the full search Rate-Distortion Optimization, which allows reaching the optimal Coding Tree Unit partition. Despite the many speedup algorithms developed in the literature, the HEVC encoding complexity still remains a crucial problem in video coding field. Towards this goal, we propose in this paper a deep learning model-based fast mode decision algorithm for HEVC intermode. Firstly, we provide a deep insight overview of the proposed CNN-LSTM, which plays a kernel and pivotal role in this contribution, thus predicting the CU splitting and reducing the HEVC encoding complexity. Secondly, a large training and inference dataset for HEVC intercoding was investigated to train and test the proposed deep framework. Based on this framework, the temporal correlation of the CU partition for each video frame is solved by the LSTM network. Numerical results prove that the proposed CNN-LSTM scheme reduces the encoding complexity by 58.60% with an increase in the BD rate of 1.78% and a decrease in the BD-PSNR of -0.053 dB. Compared to the related works, the proposed scheme has achieved a best compromise between RD performance and complexity reduction, as proven by experimental results.