Machine-Type Video Communication Using Pretrained Network for Internet of Things

With the increasing demand for internet of things (IoT) applications, machine-type video communications have become an indispensable means of communication. It is changing the way we live and work. In machine-type video communications, the quality and delay of the video transmission should be guaranteed to satisfy the requirements of communication devices at the condition of limited resources. It is necessary to reduce the burden of transmitting video by losing frames at the video sender and then to increase the frame rate of transmitting video at the receiver. In this paper, based on the pretrained network, we proposed a frame rate up-conversion (FRUC) algorithm to guarantee low-latency video transmitting in machine-type video communications. At the IoT node, by periodically discarding the video frames, the video sequences are significantly compressed. At the IoT cloud, a pretrained network is used to extract the feature layers of the transmitted video frames, which is fused into the bidirectional matching to produce the motion vectors (MVs) of the losing frames, and according to the output MVs, the motion-compensated interpolation is implemented to recover the original frame rate of the video sequence. Experimental results show that the proposed FRUC algorithm effectively improve both objective and subjective qualities of the transmitted video sequences.

Homography-guided stereo matching for wide-baseline image interpolation

Image interpolation has a wide range of applications such as frame rate-up conversion and free viewpoint TV. Despite significant progresses, it remains an open challenge especially for image pairs with large displacements. In this paper, we first propose a novel optimization algorithm for motion estimation, which combines the advantages of both global optimization and a local parametric transformation model. We perform optimization over dynamic label sets, which are modified after each iteration using the prior of piecewise consistency to avoid local minima. Then we apply it to an image interpolation framework including occlusion handling and intermediate image interpolation. We validate the performance of our algorithm experimentally, and show that our approach achieves state-of-the-art performance.