Deep learning-based video coding optimisation of H.265

With the development of technology, the hardware requirement and expectations of user for visual enjoyment are getting higher and higher. The multitype tree (MTT) architecture is proposed by the Joint Video Experts Team (JVET). Therefore, it is necessary to determine not only coding unit (CU) depth but also its split mode in the H.266/Versatile Video Coding (H.266/VVC). Although H.266/VVC achieves significant coding performance on the basis of H.265/High Efficiency Video Coding (H.265/HEVC), it causes significantly coding complexity and increases coding time, where the most time-consuming part is traversal calculation rate-distortion (RD) of CU. To solve these problems, this paper proposes an adaptive CU split decision method based on deep learning and multifeature fusion. Firstly, we develop a texture classification model based on threshold to recognize complex and homogeneous CU. Secondly, if the complex CUs belong to edge CU, a Convolutional Neural Network (CNN) structure based on multifeature fusion is utilized to classify CU. Otherwise, an adaptive CNN structure is used to classify CUs. Finally, the division of CU is determined by the trained network and the parameters of CU. When the complex CUs are split, the above two CNN schemes can successfully process the training samples and terminate the rate-distortion optimization (RDO) calculation for some CUs. The experimental results indicate that the proposed method reduces the computational complexity and saves 39.39% encoding time, thereby achieving fast encoding in H.266/VVC.

Download Full-text

Improved Low-Bitrate HEVC Video Coding Using Deep Learning Based Super-Resolution and Adaptive Block Patching

IEEE Transactions on Multimedia ◽

10.1109/tmm.2019.2919433 ◽

2019 ◽

Vol 21 (12) ◽

pp. 3010-3023 ◽

Cited By ~ 2

Author(s):

Hongwei Lin ◽

Xiaohai He ◽

Linbo Qing ◽

Qizhi Teng ◽

Songfan Yang

Keyword(s):

Deep Learning ◽

Video Coding ◽

Super Resolution

Download Full-text

A Switchable Deep Learning Approach for In-loop Filtering in Video Coding

IEEE Transactions on Circuits and Systems for Video Technology ◽

10.1109/tcsvt.2019.2935508 ◽

2020 ◽

pp. 1-1 ◽

Cited By ~ 3

Author(s):

Dandan Ding ◽

Lingyi Kong ◽

Guangyao Chen ◽

Zoe Liu ◽

Yong Fang

Keyword(s):

Deep Learning ◽

Video Coding ◽

Learning Approach

Download Full-text

Application of region-based video surveillance in smart cities using deep learning

Multimedia Tools and Applications ◽

10.1007/s11042-021-11468-w ◽

2021 ◽

Author(s):

Asma Zahra ◽

Mubeen Ghafoor ◽

Kamran Munir ◽

Ata Ullah ◽

Zain Ul Abideen

Keyword(s):

Deep Learning ◽

Video Coding ◽

Video Surveillance ◽

Smart City ◽

Smart Cities ◽

Video Frame ◽

Visual Data ◽

High Quality ◽

City Environment ◽

Salient Regions

AbstractSmart video surveillance helps to build more robust smart city environment. The varied angle cameras act as smart sensors and collect visual data from smart city environment and transmit it for further visual analysis. The transmitted visual data is required to be in high quality for efficient analysis which is a challenging task while transmitting videos on low capacity bandwidth communication channels. In latest smart surveillance cameras, high quality of video transmission is maintained through various video encoding techniques such as high efficiency video coding. However, these video coding techniques still provide limited capabilities and the demand of high-quality based encoding for salient regions such as pedestrians, vehicles, cyclist/motorcyclist and road in video surveillance systems is still not met. This work is a contribution towards building an efficient salient region-based surveillance framework for smart cities. The proposed framework integrates a deep learning-based video surveillance technique that extracts salient regions from a video frame without information loss, and then encodes it in reduced size. We have applied this approach in diverse case studies environments of smart city to test the applicability of the framework. The successful result in terms of bitrate 56.92%, peak signal to noise ratio 5.35 bd and SR based segmentation accuracy of 92% and 96% for two different benchmark datasets is the outcome of proposed work. Consequently, the generation of less computational region-based video data makes it adaptable to improve surveillance solution in Smart Cities.

Download Full-text

Wyner-Ziv Video Coding using Hadamard Transform and Deep Learning

International Journal of Advanced Computer Science and Applications ◽

10.14569/ijacsa.2016.070779 ◽

2016 ◽

Vol 7 (7) ◽

Author(s):

Jean-Paul Kouma ◽

Ulrik Soderstrom

Keyword(s):

Deep Learning ◽

Video Coding ◽

Hadamard Transform

Download Full-text

VVC In-Loop Filtering Based on Deep Convolutional Neural Network

Computational Intelligence and Neuroscience ◽

10.1155/2021/9912839 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Soulef Bouaafia ◽

Seifeddine Messaoud ◽

Randa Khemiri ◽

Fatma Elzahra Sayadi

Keyword(s):

Neural Network ◽

Deep Learning ◽

Video Coding ◽

Convolutional Neural Network ◽

Video Streaming ◽

High Efficiency ◽

Dynamic Range ◽

Video Quality ◽

Deep Convolutional Neural Network ◽

Frame Rate

With the rapid advancement in many multimedia applications, such as video gaming, computer vision applications, and video streaming and surveillance, video quality remains an open challenge. Despite the existence of the standardized video quality as well as high definition (HD) and ultrahigh definition (UHD), enhancing the quality for the video compression standard will improve the video streaming resolution and satisfy end user’s quality of service (QoS). Versatile video coding (VVC) is the latest video coding standard that achieves significant coding efficiency. VVC will help spread high-quality video services and emerging applications, such as high dynamic range (HDR), high frame rate (HFR), and omnidirectional 360-degree multimedia compared to its predecessor high efficiency video coding (HEVC). Given its valuable results, the emerging field of deep learning is attracting the attention of scientists and prompts them to solve many contributions. In this study, we investigate the deep learning efficiency to the new VVC standard in order to improve video quality. However, in this work, we propose a wide-activated squeeze-and-excitation deep convolutional neural network (WSE-DCNN) technique-based video quality enhancement for VVC. Thus, the VVC conventional in-loop filtering will be replaced by the suggested WSE-DCNN technique that is expected to eliminate the compression artifacts in order to improve visual quality. Numerical results demonstrate the efficacy of the proposed model achieving approximately − 2.85 % , − 8.89 % , and − 10.05 % BD-rate reduction of the luma (Y) and both chroma (U, V) components, respectively, under random access profile.

Download Full-text