Grasp detection via visual rotation object detection and point cloud spatial feature scoring

Accurately detecting the appropriate grasp configurations is the central task for the robot to grasp an object. Existing grasp detection methods usually overlook the depth image or only regard it as a two-dimensional distance image, which makes it difficult to capture the three-dimensional structural characteristics of target object. In this article, we transform the depth image to point cloud and propose a two-stage grasp detection method based on candidate grasp detection from RGB image and spatial feature rescoring from point cloud. Specifically, we first adopt the recently proposed high-performance rotation object detection method for aerial images, named R3Det, to grasp detection task, obtaining the candidate grasp boxes and their appearance scores. Then, point clouds within each candidate grasp box are normalized and evaluated to get the point cloud quality scores, which are fused with the established point cloud quantity scoring model to obtain spatial scores. Finally, appearance scores and their corresponding spatial scores are combined to output high-quality grasp detection results. The proposed method effectively fuses three types of grasp scoring modules, thus is called Score Fusion Grasp Net. Besides, we propose and adopt top-k grasp metric to effectively reflect the success rate of algorithm in actual grasp execution. Score Fusion Grasp Net obtains 98.5% image-wise accuracy and 98.1% object-wise accuracy on Cornell Grasp Dataset, which exceeds the performances of state-of-the-art methods. We also use the robotic arm to conduct physical grasp experiments on 15 kinds of household objects and 11 kinds of adversarial objects. The results show that the proposed method still has a high success rate when facing new objects.

Handwritten Signature Verification Method based on Improved Combined Features

As a behavior feature, handwritten signatures are widely used in financial and administrative institutions. The appearance of forged signatures will cause great property losses to customers. This paper proposes a handwritten signature verification method based on improved combined features. According to advanced smart pen technology, when writing a signature, offline images and online data of the signature can be obtained in real time. It is the first time to realize the combination of offline and online. We extract the static and dynamic features of the signature and verify them with support vector machine (SVM) and dynamic time warping (DTW) respectively. We use a small number of samples during the training stage, which solves the problem of insufficient number of samples to a certain extent. We get two decision scores while getting the verification result. Finally, we propose a score fusion method based on accuracy (SF-A), which combines offline and online features through score fusion and utilize the complementarity among classifiers effectively. Experimental results show that using different numbers of training samples to conduct experiments on local data sets, the false acceptance rate (FAR) and false reject rate (FRR) obtained are better than the offline or online verification results.

A Pedestrian Detection Algorithm Based on Score Fusion for Multi-LiDAR Systems

Pedestrian detection plays an essential role in the navigation system of autonomous vehicles. Multisensor fusion-based approaches are usually used to improve detection performance. In this study, we aimed to develop a score fusion-based pedestrian detection algorithm by integrating the data of two light detection and ranging systems (LiDARs). We first evaluated a two-stage object-detection pipeline for each LiDAR, including object proposal and fine classification. The scores from these two different classifiers were then fused to generate the result using the Bayesian rule. To improve proposal performance, we applied two features: the central points density feature, which acts as a filter to speed up the process and reduce false alarms; and the location feature, including the density distribution and height difference distribution of the point cloud, which describes an object’s profile and location in a sliding window. Extensive experiments tested in KITTI and the self-built dataset show that our method could produce highly accurate pedestrian detection results in real-time. The proposed method not only considers the accuracy and efficiency but also the flexibility for different modalities.

Multi-view classification with convolutional neural networks

Humans’ decision making process often relies on utilizing visual information from different views or perspectives. However, in machine-learning-based image classification we typically infer an object’s class from just a single image showing an object. Especially for challenging classification problems, the visual information conveyed by a single image may be insufficient for an accurate decision. We propose a classification scheme that relies on fusing visual information captured through images depicting the same object from multiple perspectives. Convolutional neural networks are used to extract and encode visual features from the multiple views and we propose strategies for fusing these information. More specifically, we investigate the following three strategies: (1) fusing convolutional feature maps at differing network depths; (2) fusion of bottleneck latent representations prior to classification; and (3) score fusion. We systematically evaluate these strategies on three datasets from different domains. Our findings emphasize the benefit of integrating information fusion into the network rather than performing it by post-processing of classification scores. Furthermore, we demonstrate through a case study that already trained networks can be easily extended by the best fusion strategy, outperforming other approaches by large margin.

Enhanced Action Recognition Using Multiple Stream Deep Learning with Optical Flow and Weighted Sum

Various action recognition approaches have recently been proposed with the aid of three-dimensional (3D) convolution and a multiple stream structure. However, existing methods are sensitive to background and optical flow noise, which prevents from learning the main object in a video frame. Furthermore, they cannot reflect the accuracy of each stream in the process of combining multiple streams. In this paper, we present a novel action recognition method that improves the existing method using optical flow and a multi-stream structure. The proposed method consists of two parts: (i) optical flow enhancement process using image segmentation and (ii) score fusion process by applying weighted sum of the accuracy. The enhancement process can help the network to efficiently analyze the flow information of the main object in the optical flow frame, thereby improving accuracy. A different accuracy of each stream can be reflected to the fused score while using the proposed score fusion method. We achieved an accuracy of 98.2% on UCF-101 and 82.4% on HMDB-51. The proposed method outperformed many state-of-the-art methods without changing the network structure and it is expected to be easily applied to other networks.

Multimodal score level fusion for recognition using face and palmprint

The paper proposes a t-norm-based matching score fusion approach for a multimodal heterogenous biometric recognition system. Two trait-based multimodal recognition system is developed by using biometrics traits like palmprint and face. First, palmprint and face are pre-processed, extracted features and calculated matching score of each trait using correlation coefficient and combine matching scores using t-norm based score level fusion. Face database like Face 94, Face 95, Face 96, FERET, FRGC and palmprint database like IITD are operated for training and testing of algorithm. The results of experimentation show that the proposed algorithm provides the Genuine Acceptance Rate (GAR) of 99.7% at False Acceptance Rate (FAR) of 0.1% and GAR of 99.2% at FAR of 0.01% significantly improves the accuracy of a biometric recognition system. The proposed algorithm provides the 0.53% more accuracy at FAR of 0.1% and 2.77% more accuracy at FAR of 0.01%, when compared to existing works.