CSA6D: Channel-Spatial Attention Networks for 6D Object Pose Estimation

Abstract6D object pose estimation plays a crucial role in robotic manipulation and grasping tasks. The aim to estimate the 6D object pose from RGB or RGB-D images is to detect objects and estimate their orientations and translations relative to the given canonical models. RGB-D cameras provide two sensory modalities: RGB and depth images, which could benefit the estimation accuracy. But the exploitation of two different modality sources remains a challenging issue. In this paper, inspired by recent works on attention networks that could focus on important regions and ignore unnecessary information, we propose a novel network: Channel-Spatial Attention Network (CSA6D) to estimate the 6D object pose from RGB-D camera. The proposed CSA6D includes a pre-trained 2D network to segment the interested objects from RGB image. Then it uses two separate networks to extract appearance and geometrical features from RGB and depth images for each segmented object. Two feature vectors for each pixel are stacked together as a fusion vector which is refined by an attention module to generate a aggregated feature vector. The attention module includes a channel attention block and a spatial attention block which can effectively leverage the concatenated embeddings into accurate 6D pose prediction on known objects. We evaluate proposed network on two benchmark datasets YCB-Video dataset and LineMod dataset and the results show it can outperform previous state-of-the-art methods under ADD and ADD-S metrics. Also, the attention map demonstrates our proposed network searches for the unique geometry information as the most likely features for pose estimation. From experiments, we conclude that the proposed network can accurately estimate the object pose by effectively leveraging multi-modality features.

Grayscale Medical Image Segmentation Method Based on 2D&3D Object Detection with Deep Learning

Background: Grayscale medical image segmentation is the key step in clinical computer-aided diagnosis. Model-driven and data-driven image segmentation methods are widely used for their less computational complexity and more accurate feature extraction. However, model-driven methods like thresholding usually suffer from wrong segmentation and noises regions because different grayscale images have distinct intensity distribution property thus pre-processing is always demanded. While data-driven methods with deep learning like encoder-decoder networks always are always accompanied by complex architectures which require amounts of training data. Methods: Combining thresholding method and deep learning, this paper presents a novel method by using 2D&3D object detection technologies. First, interest regions contain segmented object are determined with fine-tuning 2D object detection network. Then, pixels in cropped images are turned as point cloud according to their positions and grayscale values. Finally, 3D object detection network is applied to obtain bounding boxes with target points and boxes’ bottoms and tops represent thresholding values for segmentation. After projecting to 2D images, these target points could composite the segmented object. Results: Three groups of grayscale medical images are used to evaluate the proposed image segmentation method. We obtain the IoU (DSC) scores of 0.92 (0.96), 0.88 (0.94) and 0.94 (0.94) for segmentation accuracy on different datasets respectively. Also, compared with five state of the arts and clinically performed well models, our method achieves higher scores and better performance.Conclusions: The prominent segmentation results demonstrate that the built method based on 2D&3D object detection with deep learning is workable and promising for segmentation task of grayscale medical images.

Updating 3D Planning Data based on Detected Differences between Real and Planning Data of Building Interiors[

This paper presents a system to determine differences between 3D reconstructed interiors and their corresponding 3D planning data with the aim of correcting identified differences and updating the 3D planning data based on these deviations. Therefore, a point-based comparison algorithm was developed with which deviations can be recognized regardless of the topology of the data used. Usually, resolution and topology of a 3D reconstruction do not match the CAD data. Here, our solution overcomes this problem by segmenting and extracting objects relevant for comparison (e.g., doors, windows) from the reconstruction and planning data separately with a subsequent analysis of the proximity of these objects to connected walls within the corresponding data set. Starting from the connection points of a segmented object to its walls, adjacent spatial data is located for a correction of detected differences to update the 3D planning data. The quality of the result of the developed process is shown in different examples localizing doors and windows to find deviations. In addition, detected differences between the planning and the measurement data are visualized and compared with the ground truth state of the building interior.

A Novel Hybrid Segmentation Method with Particle Swarm Optimization and Fuzzy C-Mean Based On Partitioning the Image for Detecting Lung Cancer

Recently, the medical image processing is extensively used in several areas. In earlier detection and treatment of these diseases is very helpful to find out the abnormality issues in that image. Here there are number of methods available for segmentation to detect the lung nodule of computer tomography (CT) image. The main result of this paper, the earlier detection of lung nodules using Pre-processing techniques of top-hat transform, median and adaptive bilateral filter was compared both filtering methods and proved the adaptive bilateral filter is suitable method for CT images. The proposed segmentation technique uses novel strip method and the image is split into number of strips 3, 4, 5 and 6. A marker- watershed method based on PSO and Fuzzy C-mean Clustering method was proposed method. Firstly, the input image was reduced noise reduction and smoothing and the filter image is using strips method and then the image is segmented by marker watershed method. Secondly, the enhanced PSO technique was used to locate the better accurate value of the clustering centers of Fuzzy C-mean Clustering. Final stage, with the accurate value of centers and the enhanced target function and the small region of the segmented object was clustered by Fuzzy C-mean. In segmentation algorithm presented in this paper gives 95% of accuracy rate to detect lung nodules when strip count is 5.

A Novel Surface Descriptor for Automated 3-D Object Recognition and Localization

This paper presents a novel approach to the automated recognition and localization of 3-D objects. The proposed approach uses 3-D object segmentation to segment randomly stacked objects in an unstructured point cloud. Each segmented object is then represented by a regional area-based descriptor, which measures the distribution of surface area in the oriented bounding box (OBB) of the segmented object. By comparing the estimated descriptor with the template descriptors stored in the database, the object can be recognized. With this approach, the detected object can be matched with the model using the iterative closest point (ICP) algorithm to detect its 3-D location and orientation. Experiments were performed to verify the feasibility and effectiveness of the approach. With the measured point clouds having a spatial resolution of 1.05 mm, the proposed method can achieve both a mean deviation and standard deviation below half of the spatial resolution.

Improvement of the Image Sub-Segmentation for Identification and Differentiation of Atypical Regions

A new sub-segmentation method has been proposed in 2009 which, in digital images, help us to identify the typical pixels, as well as the less representative pixels or atypical of each segmented region. This method is based on the Possibilistic Fuzzy c-Means (PFCM) clustering algorithm, as it integrates absolute and relative memberships. Now, the segmentation problem is related to isolate each one of the objects present in an image. However, and considering only one segmented object or region represented by gray levels as its only feature, the totality of pixels is divided in two basic groups, the group of pixels representing the object, and the others that do not represent it. In the former group, there is a sub-group of pixels near the most representative element of the object, the prototype, and identified here as the typical pixels, and a sub-group corresponding to the less representative pixels of the object, which are the atypical pixels, and generally located at the borders of the pixels representing the object. Besides, the sub-group of atypical pixels presents greater tones (brighter or towards the white color) or smaller tones (darker or towards black color). So, the sub-segmentation method offers the capability to identify the sub-region of atypical pixels, although without performing a differentiation between the brighter and the darker ones. Hence, the proposal of this work contributes to the problem of image segmentation with the improvement on the detection of the atypical sub-regions, and clearly recognizing between both kind of atypical pixels, because in many cases only the brighter or the darker atypical pixels are the ones that represent the object of interest in an image, depending on the problem to be solved. In this study, two real cases are used to show the contribution of this proposal; the first case serves to demonstrate the pores detection in soil images represented by the darker atypical pixels, and the second one to demonstrate the detection of microcalcifications in mammograms, represented in this case by the brighter atypical pixels.

Image Segmentation Research Based on GA and Improved Otsu Algorithm

<p>In the face of the problem of high complexity of two-dimensional Otsu adaptive threshold algorithm, a new fast and effective Otsu image segmentation algorithm is proposed based on genetic algorithm. This algorithm replaces the segmentation threshold of the traditional two - dimensional Otsu method by finding the threshold of two one-dimensional Otsu method, it reduces the computational complexity of the partition from O (L4) to O (L). In order to ensure the integrity of the segmented object, the algorithm introduces the concept of small dispersion in class, and the automatic optimization of parameters are achieved by genetic algorithm. Theoretical analysis and experimental results show that the algorithm is not only better than the original two-dimensional Otsu algorithm, but also it has better segmentation effect.</p>