A Motion Estimation based Algorithm for Encoding Time Reduction in HEVC

High Efficiency Video Coding (HEVC) is a video compression standard that offers 50% more efficiency at the expense of high encoding time contrasted with the H.264 Advanced Video Coding (AVC) standard. The encoding time must be reduced to satisfy the needs of real-time applications. This paper has proposed the Multi- Level Resolution Vertical Subsampling (MLRVS) algorithm to reduce the encoding time. The vertical subsampling minimizes the number of Sum of Absolute Difference (SAD) computations during the motion estimation process. The complexity reduction algorithm is also used for fast coding the coefficients of the quantised block using a flag decision. Two distinct search patterns are suggested: New Cross Diamond Diamond (NCDD) and New Cross Diamond Hexagonal (NCDH) search patterns, which reduce the time needed to locate the motion vectors. In this paper, the MLRVS algorithm with NCDD and MLRVS algorithm with NCDH search patterns are simulated separately and analyzed. The results show that the encoding time of the encoder is decreased by 55% with MLRVS algorithm using NCDD search pattern and 56% with MLRVS using NCDH search pattern compared to HM16.5 with Test Zone (TZ) search algorithm. These results are achieved with a slight increase in bit rate and negligible deterioration in output video quality.

Motion Estimation in HEVC/H.265: Metaheuristic Approach to Improve the Efficiency

An ever increasing use of digital video applications such as video telephony, broadcast and the storage of high and ultra-high definition videos has steered the development of video coding standards. The state of the art video coding standard is High Efficiency Video Coding (HEVC) or otherwise known as H.265. It promises to be 50 percent more efficient than the previous video coding standard H.264. Ultimately, H.265 provides significant improvement in compression at the expense of computational complexity. HEVC encoder is very complex and 50 percent of the encoding consists of Motion Estimation (ME). It uses a Test Zone (TZ) fast search algorithm for its motion estimation, which compares a block of pixels with a few selected blocks in the search region of a referenced frame. However, the encoding time is not suitable to meet the needs of real time video applications. So, there is a requirement to improve the search algorithm and to provide comparable results to TZ search to save a substantial amount of time. In our paper, we aim to study the effects of a meta-heuristic algorithm on motion estimation. One such suitable algorithm for this task is the Firefly Algorithm (FA). FA is inspired by the social behavior of fireflies and is generally used to solve optimization problems. Our results show that implementing FA for ME saves a considerable amount of time with a comparable encoding efficiency.

Application of region-based video surveillance in smart cities using deep learning

Smart video surveillance helps to build more robust smart city environment. The varied angle cameras act as smart sensors and collect visual data from smart city environment and transmit it for further visual analysis. The transmitted visual data is required to be in high quality for efficient analysis which is a challenging task while transmitting videos on low capacity bandwidth communication channels. In latest smart surveillance cameras, high quality of video transmission is maintained through various video encoding techniques such as high efficiency video coding. However, these video coding techniques still provide limited capabilities and the demand of high-quality based encoding for salient regions such as pedestrians, vehicles, cyclist/motorcyclist and road in video surveillance systems is still not met. This work is a contribution towards building an efficient salient region-based surveillance framework for smart cities. The proposed framework integrates a deep learning-based video surveillance technique that extracts salient regions from a video frame without information loss, and then encodes it in reduced size. We have applied this approach in diverse case studies environments of smart city to test the applicability of the framework. The successful result in terms of bitrate 56.92%, peak signal to noise ratio 5.35 bd and SR based segmentation accuracy of 92% and 96% for two different benchmark datasets is the outcome of proposed work. Consequently, the generation of less computational region-based video data makes it adaptable to improve surveillance solution in Smart Cities.

Bagged Tree And ResNet Based Joint End-to-End Fast CTU Partition Decision Algorithm For Video Intra Coding

In the development of video coding standards, advanced ones have greatly improved the bit rate compared with those of previous generation, but also brought a huge increase in coding complexity. Coding standards, such as high efficiency video coding (HEVC), versatile video coding (VVC) and AOMedia video 2 (AV2), get the optimal encoding performance by traversing all possible combinations of coding unit (CU) partition and selecting the combination with minimum coding cost. This process of searching for the best makes up a large part of encoding complexity. To reduce the complexity of coding block partition for many video coding standards, this paper proposes an end-to-end fast algorithm for partition structure decision of coding tree unit (CTU) in intra coding. It can be extended to various coding standards with fine tuning, and is applied to the intra coding of HEVC reference software HM16.7 as an example. In the proposed method, the splitting decision of a CTU is made by a well designed bagged tree model firstly. Then, the partition problem of a 32×32 sized CU is modeled as a 17-output classification task and solved by a well trained residual network (ResNet). Jointly using bagged tree and ResNet, the proposed fast CTU partition algorithm is able to generate the partition quad-tree structure of a CTU through an end-to-end prediction process, instead of multiple decision making procedures at depth level. Besides, several effective and representative datasets are also conducted in this paper to lay the foundation of high prediction accuracy. Compared with the original HM16.7 encoder, experimental results show that the proposed algorithm can reduce the encoding time by 59.79% on average, while the BD-rate loss is as less as 2.02%, which outperforms the results of most of state-of-the-art approaches in the fast intra CU partition area.

A Complexity Reduction Method for VVC Intra Prediction Based on Statistical Analysis and SAE-CNN

Compared with High Efficiency Video Coding (HEVC), the latest video coding standard Versatile Video Coding Standard (VVC), due to the introduction of many novel technologies and the introduction of the Quad-tree with nested Multi-type Tree (QTMT) division scheme in the block division method, the coding quality has been greatly improved. Due to the introduction of the QTMT scheme, the encoder needs to perform rate–distortion optimization for each division mode during Coding Unit (CU) division, so as to select the best division mode, which also leads to an increase in coding time and coding complexity. Therefore, we propose a VVC intra prediction complexity reduction algorithm based on statistical theory and the Size-adaptive Convolutional Neural Network (SAE-CNN). The algorithm combines the establishment of a pre-decision dictionary based on statistical theory and a Convolutional Neural Network (CNN) model based on adaptively adjusting the size of the pooling layer to form an adaptive CU size division decision process. The algorithm can make a decision on whether to divide CUs of different sizes, thereby avoiding unnecessary Rate–distortion Optimization (RDO) and reducing coding time. Experimental results show that compared with the original algorithm, our suggested algorithm can save 35.60% of the coding time and only increases the Bjøntegaard Delta Bit Rate (BD-BR) by 0.91%.

Fast CU Size Decision Method Based on Just Noticeable Distortion and Deep Learning

With the development of broadband networks and high-definition displays, people have higher expectations for the quality of video images, which also brings new requirements and challenges to video coding technology. Compared with H.265/High Efficiency Video Coding (HEVC), the latest video coding standard, Versatile Video Coding (VVC), can save 50%-bit rate while maintaining the same subjective quality, but it leads to extremely high encoding complexity. To decrease the complexity, a fast coding unit (CU) size decision method based on Just Noticeable Distortion (JND) and deep learning is proposed in this paper. Specifically, the hybrid JND threshold model is first designed to distinguish smooth, normal, or complex region. Then, if CU belongs to complex area, the Ultra-Spherical SVM (US-SVM) classifiers are trained for forecasting the best splitting mode. Experimental results illustrate that the proposed method can save about 52.35% coding runtime, which can realize a trade-off between the reduction of computational burden and coding efficiency compared with the latest methods.

Versatile Video Coding for 3.0 Next Generation Digital TV in Brazil

In the past few decades, the video broadcast ecosystem has gone through major changes; Originally transmitted using analog signals, it has been more and more transitioned toward digital, leveraging compression technologies and transport protocols, principally developed by MPEG. Along this way, the introduction of new video formats was achieved with standardization of new compression technologies for their better bandwidth preservation. Notably, SD with MPEG-2, HD with H.264, 4K/UHD with HEVC. In Brazil, the successive generations of digital broadcasting systems were developed by the SBTVD Forum, from TV-1.0 to TV-3.0 nowadays. The ambition of TV-3.0 is significantly higher than that of previous generations as it targets the delivery of IPbased signals for applications, such as 8K, HDR, virtual and augmented reality. To deliver such services, compressed video signals shall fit into a limited bandwidth, requiring even more advanced compression technologies. The Versatile Video Coding standard (H.266/VVC), has been finalized by the JVET committee in 2021 and is a relevant candidate to address the TV3.0 requirements. VVC is versatile by nature thanks to its dedicated tools for efficient compression of various formats, from 8K to 360°, and provides around 50% of bitrate saving compared to its predecessor HEVC. This paper presents the VVC-based compression system that has been proposed to the SBTVD call for proposals for TV-3.0. A technical description of VVC and an evaluation of its coding performance is provided. In addition, an end-to-end live transmission chain is demonstrated, supporting 4K real-time encoding and decoding with a low glass-to-glass latency.