3D Shape Classification Based on Point Convolutional Neural Network Combining Multi-geometric Features

Mesh is an important and powerful type of data for 3D shapes and widely studied in the field of computer vision and computer graphics. Regarding the task of 3D shape representation, there have been extensive research efforts concentrating on how to represent 3D shapes well using volumetric grid, multi-view and point cloud. However, there is little effort on using mesh data in recent years, due to the complexity and irregularity of mesh data. In this paper, we propose a mesh neural network, named MeshNet, to learn 3D shape representation from mesh data. In this method, face-unit and feature splitting are introduced, and a general architecture with available and effective blocks are proposed. In this way, MeshNet is able to solve the complexity and irregularity problem of mesh and conduct 3D shape representation well. We have applied the proposed MeshNet method in the applications of 3D shape classification and retrieval. Experimental results and comparisons with the state-of-the-art methods demonstrate that the proposed MeshNet can achieve satisfying 3D shape classification and retrieval performance, which indicates the effectiveness of the proposed method on 3D shape representation.

Download Full-text

MLVCNN: Multi-Loop-View Convolutional Neural Network for 3D Shape Retrieval

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33018513 ◽

2019 ◽

Vol 33 ◽

pp. 8513-8520 ◽

Cited By ~ 10

Author(s):

Jianwen Jiang ◽

Di Bao ◽

Ziqiang Chen ◽

Xibin Zhao ◽

Yue Gao

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

State Of The Art ◽

Shape Representation ◽

The State ◽

Shape Retrieval ◽

3D Shape Retrieval ◽

3D Shape ◽

Loop Level ◽

Art Methods

3D shape retrieval has attracted much attention and become a hot topic in computer vision field recently.With the development of deep learning, 3D shape retrieval has also made great progress and many view-based methods have been introduced in recent years. However, how to represent 3D shapes better is still a challenging problem. At the same time, the intrinsic hierarchical associations among views still have not been well utilized. In order to tackle these problems, in this paper, we propose a multi-loop-view convolutional neural network (MLVCNN) framework for 3D shape retrieval. In this method, multiple groups of views are extracted from different loop directions first. Given these multiple loop views, the proposed MLVCNN framework introduces a hierarchical view-loop-shape architecture, i.e., the view level, the loop level, and the shape level, to conduct 3D shape representation from different scales. In the view-level, a convolutional neural network is first trained to extract view features. Then, the proposed Loop Normalization and LSTM are utilized for each loop of view to generate the loop-level features, which considering the intrinsic associations of the different views in the same loop. Finally, all the loop-level descriptors are combined into a shape-level descriptor for 3D shape representation, which is used for 3D shape retrieval. Our proposed method has been evaluated on the public 3D shape benchmark, i.e., ModelNet40. Experiments and comparisons with the state-of-the-art methods show that the proposed MLVCNN method can achieve significant performance improvement on 3D shape retrieval tasks. Our MLVCNN outperforms the state-of-the-art methods by the mAP of 4.84% in 3D shape retrieval task. We have also evaluated the performance of the proposed method on the 3D shape classification task where MLVCNN also achieves superior performance compared with recent methods.

Download Full-text

Generalized Sparse Convolutional Neural Networks for Semantic Segmentation of Point Clouds Derived from Tri-Stereo Satellite Imagery

Remote Sensing ◽

10.3390/rs12081289 ◽

2020 ◽

Vol 12 (8) ◽

pp. 1289

Author(s):

Stefan Bachhofner ◽

Ana-Maria Loghin ◽

Johannes Otepka ◽

Norbert Pfeifer ◽

Michael Hornacek ◽

...

Keyword(s):

Neural Network ◽

Decision Tree ◽

Convolutional Neural Network ◽

Semantic Segmentation ◽

Point Clouds ◽

Representation Learning ◽

Geometric Features ◽

Color Information ◽

Geometric Information ◽

Learning Technique

We studied the applicability of point clouds derived from tri-stereo satellite imagery for semantic segmentation for generalized sparse convolutional neural networks by the example of an Austrian study area. We examined, in particular, if the distorted geometric information, in addition to color, influences the performance of segmenting clutter, roads, buildings, trees, and vehicles. In this regard, we trained a fully convolutional neural network that uses generalized sparse convolution one time solely on 3D geometric information (i.e., 3D point cloud derived by dense image matching), and twice on 3D geometric as well as color information. In the first experiment, we did not use class weights, whereas in the second we did. We compared the results with a fully convolutional neural network that was trained on a 2D orthophoto, and a decision tree that was once trained on hand-crafted 3D geometric features, and once trained on hand-crafted 3D geometric as well as color features. The decision tree using hand-crafted features has been successfully applied to aerial laser scanning data in the literature. Hence, we compared our main interest of study, a representation learning technique, with another representation learning technique, and a non-representation learning technique. Our study area is located in Waldviertel, a region in Lower Austria. The territory is a hilly region covered mainly by forests, agriculture, and grasslands. Our classes of interest are heavily unbalanced. However, we did not use any data augmentation techniques to counter overfitting. For our study area, we reported that geometric and color information only improves the performance of the Generalized Sparse Convolutional Neural Network (GSCNN) on the dominant class, which leads to a higher overall performance in our case. We also found that training the network with median class weighting partially reverts the effects of adding color. The network also started to learn the classes with lower occurrences. The fully convolutional neural network that was trained on the 2D orthophoto generally outperforms the other two with a kappa score of over 90% and an average per class accuracy of 61%. However, the decision tree trained on colors and hand-crafted geometric features has a 2% higher accuracy for roads.

Download Full-text

Trilateral convolutional neural network for 3D shape reconstruction of objects from a single depth view

IET Image Processing ◽

10.1049/iet-ipr.2019.0532 ◽

2019 ◽

Vol 13 (13) ◽

pp. 2457-2466 ◽

Cited By ~ 1

Author(s):

Patricio Rivera ◽

Edwin Valarezo Añazco ◽

Mun-Taek Choi ◽

Tae-Seong Kim

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Shape Reconstruction ◽

3D Shape ◽

3D Shape Reconstruction ◽

Depth View

Download Full-text

Bacteria shape classification by the use of region covariance and Convolutional Neural Network

2019 International Joint Conference on Neural Networks (IJCNN) ◽

10.1109/ijcnn.2019.8851958 ◽

2019 ◽

Cited By ~ 5

Author(s):

Dawid Polap ◽

Marcin Wozniak

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Shape Classification ◽

Region Covariance

Download Full-text

SEMANTIC SEGMENTATION OF AERIAL IMAGERY VIA MULTI-SCALE SHUFFLING CONVOLUTIONAL NEURAL NETWORKS WITH DEEP SUPERVISION

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-1-29-2018 ◽

2018 ◽

Vol IV-1 ◽

pp. 29-36 ◽

Cited By ~ 4

Author(s):

K. Chen ◽

M. Weinmann ◽

X. Sun ◽

M. Yan ◽

S. Hinz ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Semantic Segmentation ◽

Aerial Imagery ◽

Geometric Features ◽

Feature Maps ◽

Multi Scale ◽

Intermediate Layers ◽

Segmentation Task ◽

The Impact

<p><strong>Abstract.</strong> In this paper, we address the semantic segmentation of aerial imagery based on the use of multi-modal data given in the form of true orthophotos and the corresponding Digital Surface Models (DSMs). We present the Deeply-supervised Shuffling Convolutional Neural Network (DSCNN) representing a multi-scale extension of the Shuffling Convolutional Neural Network (SCNN) with deep supervision. Thereby, we take the advantage of the SCNN involving the shuffling operator to effectively upsample feature maps and then fuse multiscale features derived from the intermediate layers of the SCNN, which results in the Multi-scale Shuffling Convolutional Neural Network (MSCNN). Based on the MSCNN, we derive the DSCNN by introducing additional losses into the intermediate layers of the MSCNN. In addition, we investigate the impact of using different sets of hand-crafted radiometric and geometric features derived from the true orthophotos and the DSMs on the semantic segmentation task. For performance evaluation, we use a commonly used benchmark dataset. The achieved results reveal that both multi-scale fusion and deep supervision contribute to an improvement in performance. Furthermore, the use of a diversity of hand-crafted radiometric and geometric features as input for the DSCNN does not provide the best numerical results, but smoother and improved detections for several objects.</p>

Download Full-text

Training convolutional neural network from multi-domain contour images for 3D shape retrieval

Pattern Recognition Letters ◽

10.1016/j.patrec.2017.08.028 ◽

2019 ◽

Vol 119 ◽

pp. 41-48 ◽

Cited By ~ 4

Author(s):

Zongxiao Zhu ◽

Cong Rao ◽

Song Bai ◽

Longin Jan Latecki

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Shape Retrieval ◽

3D Shape Retrieval ◽

3D Shape

Download Full-text

Recognition of 3D Shapes Based on 3V-DepthPano CNN

Mathematical Problems in Engineering ◽

10.1155/2020/7584576 ◽

2020 ◽

Vol 2020 ◽

pp. 1-11 ◽

Cited By ~ 1

Author(s):

Junjie Yin ◽

Ningning Huang ◽

Jing Tang ◽

Meie Fang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Shape Descriptor ◽

Recognition System ◽

Fine Tuning ◽

Training Time ◽

3D Shape ◽

3D Shapes ◽

And Training

This paper proposes a convolutional neural network (CNN) with three branches based on the three-view drawing principle and depth panorama for 3D shape recognition. The three-view drawing principle provides three key views of a 3D shape. A depth panorama contains the complete 2.5D information of each view. 3V-DepthPano CNN is a CNN system with three branches designed for depth panoramas generated from the three key views. This recognition system, i.e., 3V-DepthPano CNN, applies a three-branch convolutional neural network to aggregate the 3D shape depth panorama information into a more compact 3D shape descriptor to implement the classification of 3D shapes. Furthermore, we adopt a fine-tuning technique on 3V-DepthPano CNN and extract shape features to facilitate the retrieval of 3D shapes. The proposed method implements a good tradeoff state between higher accuracy and training time. Experiments show that the proposed 3V-DepthPano CNN with 3 views obtains approximate accuracy to MVCNN with 12/80 views. But the 3V-DepthPano CNN frame takes much shorter time to obtain depth panoramas and train the network than MVCNN. It is superior to all other existing advanced methods for both classification and shape retrieval.

Download Full-text