A Foreground-Aware Framework for Local Face Attribute Transfer

In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there are many algorithms automatically and globally transferring the style from one image to another, they fail to respect the semantics of the scene and are unable to allow users to merely transfer the attributes of one or two face organs in the foreground region leaving the background region unchanged. To overcome this problem, we developed a novel framework for semantically meaningful local face attribute transfer, which can flexibly transfer the local attribute of a face organ from the reference image to a semantically equivalent organ in the input image, while preserving the background. Our method involves warping the reference photo to match the shape, pose, location, and expression of the input image. The fusion of the warped reference image and input image is then taken as the initialized image for a neural style transfer algorithm. Our method achieves better performance in terms of inception score (3.81) and Fréchet inception distance (80.31), which is about 10% higher than those of competitors, indicating that our framework is capable of producing high-quality and photorealistic attribute transfer results. Both theoretical findings and experimental results are provided to demonstrate the efficacy of the proposed framework, reveal its superiority over other state-of-the-art alternatives.

Moving Object Detection from Moving Camera Image Sequences Using an Inertial Measurement Unit Sensor

This paper describes a new method for the detection of moving objects from moving camera image sequences using an inertial measurement unit (IMU) sensor. Motion detection systems with vision sensors have become a global research subject recently. However, detecting moving objects from a moving camera is a difficult task because of egomotion. In the proposed method, the interesting points are extracted by a Harris detector, and the background and foreground are classified by epipolar geometry. In this procedure, an IMU sensor is used to calculate the initial fundamental matrix. After the feature point classification, a transformation matrix is obtained from matching background feature points. Image registration is then applied to the consecutive images, and a difference map is extracted to find the foreground region. Finally, a minimum bounding box is applied to mark the detected moving object. The proposed method is implemented and tested with numerous real-world driving videos, which show that it outperforms the previous work.

SSNet: Learning Mid-Level Image Representation Using Salient Superpixel Network

In the standard bag-of-visual-words (BoVW) model, the burstiness problem of features and the ignorance of high-order information often weakens the discriminative power of image representation. To tackle them, we present a novel framework, named the Salient Superpixel Network, to learn the mid-level image representation. For reducing the impact of burstiness occurred in the background region, we use the salient regions instead of the whole image to extract local features, and a fast saliency detection algorithm based on the Gestalt grouping principle is proposed to generate image saliency maps. In order to introduce the high-order information, we propose a weighted second-order pooling (WSOP) method, which is capable of exploiting the high-order information and further alleviating the impact of burstiness in the foreground region. Then, we conduct experiments on six image classification benchmark datasets, and the results demonstrate the effectiveness of the proposed framework with either the handcrafted or the off-the-shelf CNN features.

Lychee Fruit Detection Based on Monocular Machine Vision in Orchard Environment

Due to the change of illumination environment and overlapping conditions caused by the neighboring fruits and other background objects, the simple application of the traditional machine vision method limits the detection accuracy of lychee fruits in natural orchard environments. Therefore, this research presented a detection method based on monocular machine vision to detect lychee fruits growing in overlapped conditions. Specifically, a combination of contrast limited adaptive histogram equalization (CLAHE), red/blue chromatic mapping, Otsu thresholding and morphology operations were adopted to segment the foreground regions of the lychees. A stepwise method was proposed for extracting individual lychee fruit from the lychee foreground region. The first step in this process was based on the relative position relation of the Hough circle and an equivalent area circle (equal to the area of the potential lychee foreground region) and was designed to distinguish lychee fruits growing in isolated or overlapped states. Then, a process based on the three-point definite circle theorem was performed to extract individual lychee fruits from the foreground regions of overlapped lychee fruit clusters. Finally, to enhance the robustness of the detection method, a local binary pattern support vector machine (LBP-SVM) was adopted to filter out the false positive detections generated by background chaff interferences. The performance of the presented method was evaluated using 485 images captured in a natural lychee orchard in Conghua (Area), Guangzhou. The detection results showed that the recall rate was 86.66%, the precision rate was greater than 87% and the F1-score was 87.07%.

A Model-Based Approach of Foreground Region of Interest Detection for Video Codecs

Detecting the Region of Interest (ROI) for video clips is a significant and useful technique both in video codecs and surveillance/monitor systems. In this paper, a new model-based detection method is designed which suits video compression codecs by proposing two models: an “inter” and “intra” model. The “inter” model exploits the motion information represented as blocks by global motion compensation approaches while the “intra” model extracts the objects details through objects filtering and image segmentation procedures. Finally, the detection results are formed through a new clustering with fine-tune approach from the “intra” model assisted with the “inter” model. Experimental results show that the proposed method fits well for real-time video codecs and it achieves a good performance both on detection precision and on computing time. In addition, the proposed method is versatile for a wide range of surveillance videos with different characteristics.

Human Action Recognition Based on Foreground Trajectory and Motion Difference Descriptors

Aimed at the problems of high redundancy of trajectory and susceptibility to background interference in traditional dense trajectory behavior recognition methods, a human action recognition method based on foreground trajectory and motion difference descriptors is proposed. First, the motion magnitude of each frame is estimated by optical flow, and the foreground region is determined according to each motion magnitude of the pixels; the trajectories are only extracted from behavior-related foreground regions. Second, in order to better describe the relative temporal information between different actions, a motion difference descriptor is introduced to describe the foreground trajectory, and the direction histogram of the motion difference is constructed by calculating the direction information of the motion difference per unit time of the trajectory point. Finally, a Fisher vector (FV) is used to encode histogram features to obtain video-level action features, and a support vector machine (SVM) is utilized to classify the action category. Experimental results show that this method can better extract the action-related trajectory, and it can improve the recognition accuracy by 7% compared to the traditional dense trajectory method.

Video Foreground Detection Algorithm Based on Fast Principal Component Pursuit and Motion Saliency

Aiming at the shortcoming of being unsuitable for dynamic background and high computational complexity of the existing RPCA- (robust principal component analysis-) based block-sparse moving object detection method, this paper proposes a two-stage foreground detection framework based on motion saliency for video sequence. At the first stage, the observed image sequence is regarded as the sum of a low-rank background matrix and a sparse outlier matrix, and then the decomposition is solved by the RPCA method via fast PCP (principal component pursuit). At the second stage, the sparse foreground blocks are obtained according to the spectral residuals and the spatial correlation of the foreground region. Finally, the block-sparse RPCA algorithm through fast PCP is used to estimate foreground areas dynamically and to reconstruct the foreground objects. Extensive experiments demonstrate that our method can exclude the interference of background motion and change, simultaneously improving the detection rate of small targets.

A novel method to detect foreground region using morphological operations with block based enhancement for underwater images

Automation of detecting the Foreground Region (FR) or Shape of the object is essential in several computer vision, object recognition applications and poses several challenges in case of underwater images. Although Synthetic Sonar Images produce better quality images scattering of light, color distortion and poor lighting conditions are the few characteristics that effects the natural scene of the captured image. A novel technique for extracting the foreground region from a low quality underwater image is presented in this paper. We have decomposed the image in to multiple levels based on discrete wavelet transforms (DWT) for improving the sharpness or to reduce the fogginess in the image in order to get the clear image. Subsequently, to determine the sharpness of the local patches in the image a block based SSI algorithm is presented. Finally, the segmentation is performed by computing the binary gradient mask with the Sobel edge detection algorithm along with morphological operations. The proposed method is fast, extracting the accurate foreground regions and also detect the smallest particles present in the image. The results are qualitatively compared with the improved fuzzy c-means clustering (FCM), Otsu’s Threshold and FCM thresholding by considering the static background images.

State model based face mask detection

The Automatic Teller Machine plays an important role in the modern economic society. ATM centers are located in remote central which are at high risk due to the increasing crime rate and robbery.These ATM centers assist with surveillance techniques to provide protection. Even after installing the surveillance mechanism, the robbers fool the security system by hiding their face using mask/helmet. Henceforth, an automatic mask detection algorithm is required to, alert when the ATM is at risk. In this work, the Gaussian Mixture Model (GMM) is applied for foreground detection to extract the regions of interest (ROI) i.e. Human being. Face region is acquired from the foreground region through the torso partitioning and applying Viola-Jones algorithm in this search space. Parts of the face such as Eye pair, Nose, and Mouth are extracted and a state model is developed to detect mask.