NL-VTON: a non-local virtual try-on network with feature preserving of body and clothes

In an image based virtual try-on network, both features of the target clothes and the input human body should be preserved. However, current techniques failed to solve the problems of blurriness on complex clothes details and artifacts on human body occlusion regions at the same time. To tackle this issue, we propose a non-local virtual try-on network NL-VTON. Considering that convolution is a local operation and limited by its convolution kernel size and rectangular receptive field, which is unsuitable for large size non-rigid transformations of persons and clothes in virtual try-on, we introduce a non-local feature attention module and a grid regularization loss so as to capture detailed features of complex clothes, and design a human body segmentation prediction network to further alleviate the artifacts on occlusion regions. The quantitative and qualitative experiments based on the Zalando dataset demonstrate that our proposed method significantly improves the ability to preserve features of bodies and clothes compared with the state-of-the-art methods.

Automatic corrections of human body depth maps using deep neural networks

This paper presents an approach to correcting misclassified pixels in depth maps representing parts of the human body. A misclassified pixel is a pixel of a depth map which, incorrectly, has the ?background? value and does not accurately reflect the distance from the sensor to the body being scanned. A completely automatic, deep learning based solution for depth map correction is proposed. As an input, the solution requires a color image and a corresponding erroneous depth map. The input color image is segmented using deep neural network for human body segmentation. The extracted segments are further used as guidance to find and amend the misclassified pixels on the depth map using a simple average based filter. Unlike other depth map refinement solutions, this paper designs a method for the improvement of the input depth map in terms of completeness instead of precision. The proposed method does not exclude the application of other refinement methods. Instead, it can be used as the first step in a depth map enhancement pipeline to determine approximate depths for erroneous pixels, while other refinement methods can be applied in a second step to improve the accuracy of the recovered depths.

Graph Cut-Based Human Body Segmentation in Color Images Using Skeleton Information from the Depth Sensor

Segmentation of human bodies in images is useful for a variety of applications, including background substitution, human activity recognition, security, and video surveillance applications. However, human body segmentation has been a challenging problem, due to the complicated shape and motion of a non-rigid human body. Meanwhile, depth sensors with advanced pattern recognition algorithms provide human body skeletons in real time with reasonable accuracy. In this study, we propose an algorithm that projects the human body skeleton from a depth image to a color image, where the human body region is segmented in the color image by using the projected skeleton as a segmentation cue. Experimental results using the Kinect sensor demonstrate that the proposed method provides high quality segmentation results and outperforms the conventional methods.