3D object detection combining semantic and geometric features from point clouds

Background: 3D object detection based on point clouds in road scenes has attracted much attention recently. The voxel-based methods voxelize the scene to regular grids, which can be processed with the advanced feature learning frameworks based on convolutional layers for semantic feature learning. The point-based methods can extract the geometric feature of the point due to the coordinate reservations. The combination of the two is effective for 3D object detection. However, the current methods use a voxel-based detection head with anchors for classification and localization. Although the preset anchors cover the entire scene, it is not suitable for detection tasks with larger scenes and multiple categories of objects, due to the limitation of the voxel size. Additionally, the misalignment between the predicted confidence and proposals in the Regions of the Interest (ROI) selection bring obstacles to 3D object detection. Methods: We investigate the combination of voxel-based methods and point-based methods for 3D object detection. Additionally, a voxel-to-point module that captures semantic and geometric features is proposed in the paper. The voxel-to-point module is conducive to the detection of small-size objects and avoids the presets of anchors in the inference stage. Moreover, a confidence adjustment module with the center-boundary-aware confidence attention is proposed to solve the misalignment between the predicted confidence and proposals in the regions of the interest selection. Results: The proposed method has achieved state-of-the-art results for 3D object detection in the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) object detection dataset. Actually, as of September 19, 2021, our method ranked 1st in the 3D and Bird Eyes View (BEV) detection of cyclists tagged with difficulty level ‘easy’, and ranked 2nd in the 3D detection of cyclists tagged with ‘moderate’. Conclusions: We propose an end-to-end two-stage 3D object detector with voxel-to-point module and confidence adjustment module.

An airfoil geometric-feature extraction and discrepant data fusion learning method

The perception of geometric-features of airfoils is the basis in aerodynamic area for performance prediction, parameterization, aircraft inverse design, etc. There are three approaches to percept the geometric shape of airfoils, namely manual design of airfoil geometry parameters, polynomial definition and deep learning. The first two methods directly define geometric-features or polynomials of airfoil curves, but the number of extracted features is limited. Deep learning algorithms can extract a large number of potential features (called latent features). However, the features extracted by deep learning lack explicit geometrical meaning. Motivated by the advantages of polynomial definition and deep learning, we propose a geometric-feature extraction method (named Bézier-based feature extraction, BFE) for airfoils, which consists of two parts: manifold metric feature extraction and geometric-feature fusion encoder (GF encoder). Manifold metric feature extraction, with the help of the Bézier curve, captures manifold metrics (a sort of geometric-features) from tangent space of airfoil curves, and the GF-encoder combines airfoil coordinate data and manifold metrics together to form novel fused geometric-features. To validate the feasibility of the fused geometric-features, two experiments based on the public UIUC airfoil dataset are conducted. Experiment I is used to extract manifold metrics of airfoils and export the fused geometric-features. Experiment II, based on the Multi-task learning (MTL), is used to fuse the discrepant data (i.e., the fused geometric-features and the flight conditions) to predict the aerodynamic performance of airfoils. The results show that the BFE can generate more smooth and realistic airfoils than Auto-Encoder, and the fused geometric-features extracted by BFE can be used to reduce the prediction errors of C L and C D .

Transient, Sub-Continuum, Heat Conduction in Irregular Geometries

Phonon transfer in irregular shapes is important for assessing the influence of shape effect on thermal transport characteristics of low-scale films. It becomes critical for evaluating the contribution of the scattering phonons to the phonon intensity distribution inside the film. Hence, the sub-continuum ballistic-diffusive model is incorporated to formulate the phonon transport in an irregular geometry of low-size film adopting the transient, frequency-independent, equation of phonon radiative transfer. The discrete ordinate method is used in the numerical discretization of the governing transport equation. It is demonstrated that the geometric feature of the film influences the phonon intensity distribution within the film material. The transport characteristics obtained from the Fourier and the ballistic-diffusive models are markedly different in their spatial and temporal behavior. This is true when the device sizes are of the same order of magnitude as the mean-free path of the heat carriers.

On the Geometric Featureless Visual Velocity of UGV in an Agriculture Scale

The paper discusses a camera-based velocity estimation for unmanned ground vehicles in an agriculture scale. The proposed concept-based method does not require any geometric feature and focuses on a mapping between the captured images only. The paper provides three pilot experiments. First, we check an assumption of the proposed concept by a field experiment. Second, we check the verification by a set of numerical and laboratory experiments. Third, we check the verification by the field experiment. In the sense that the existence and sensitivity of a representation of the mapping are verified experimentally, the feasibility of the proposed concept is confirmed.

Prediction of Mechanical Properties of Three-Dimensional Printed Lattice Structures Through Machine Learning

Lattice structures (LS) manufactured by 3D printing are widely applied in many areas, such as aerospace and tissue engineering, due to their lightweight and adjustable mechanical properties. It is necessary to reduce costs by predicting the mechanical properties of LS at the design stage since 3D printing is exorbitant at present. However, predicting mechanical properties quickly and accurately poses a challenge. To address this problem, this study proposes a novel method that is applied to different LS and materials to predict their mechanical properties through machine learning. First, this study voxelized 3D models of the LS units and then calculated the entropy vector of each model as the geometric feature of the LS units. Next, the porosity, material density, elastic modulus, and unit length of the lattice unit are combined with entropy as the inputs of the machine learning model. The sample set includes 57 samples collected from previous studies. Support vector regression (SVR) was used in this study to predict the mechanical properties. The results indicate that the proposed method can predict the mechanical properties of LS effectively and is suitable for different LS and materials. The significance of this work is that it provides a method with great potential to promote the design process of lattice structures by predicting their mechanical properties quickly and effectively.

LIO-CSI: LiDAR inertial odometry with loop closure combined with semantic information

Accurate and reliable state estimation and mapping are the foundation of most autonomous driving systems. In recent years, researchers have focused on pose estimation through geometric feature matching. However, most of the works in the literature assume a static scenario. Moreover, a registration based on a geometric feature is vulnerable to the interference of a dynamic object, resulting in a decline of accuracy. With the development of a deep semantic segmentation network, we can conveniently obtain the semantic information from the point cloud in addition to geometric information. Semantic features can be used as an accessory to geometric features that can improve the performance of odometry and loop closure detection. In a more realistic environment, semantic information can filter out dynamic objects in the data, such as pedestrians and vehicles, which lead to information redundancy in generated map and map-based localization failure. In this paper, we propose a method called LiDAR inertial odometry (LIO) with loop closure combined with semantic information (LIO-CSI), which integrates semantic information to facilitate the front-end process as well as loop closure detection. First, we made a local optimization on the semantic labels provided by the Sparse Point-Voxel Neural Architecture Search (SPVNAS) network. The optimized semantic information is combined into the front-end process of tightly-coupled light detection and ranging (LiDAR) inertial odometry via smoothing and mapping (LIO-SAM), which allows us to filter dynamic objects and improve the accuracy of the point cloud registration. Then, we proposed a semantic assisted scan-context method to improve the accuracy and robustness of loop closure detection. The experiments were conducted on an extensively used dataset KITTI and a self-collected dataset on the Jilin University (JLU) campus. The experimental results demonstrate that our method is better than the purely geometric method, especially in dynamic scenarios, and it has a good generalization ability.

Measuring Similarity of Deforestation Patterns in Time and Space across Differences in Resolution

This article demonstrated an easily applicable method for measuring the similarity between a pair of point patterns, which applies to spatial or temporal data sets. Such a measurement was performed using similarity-based pattern analysis as an alternative to conventional approaches, which typically utilize straightforward point-to-point matching. Using our approach, in each point data set, two geometric features (i.e., the distance and angle from the centroid) were calculated and represented as probability density functions (PDFs). The PDF similarity of each geometric feature was measured using nine metrics, with values ranging from zero (very contrasting) to one (exactly the same). The overall similarity was defined as the average of the distance and angle similarities. In terms of sensibility, the method was shown to be capable of measuring, at a human visual sensing level, two pairs of hypothetical patterns, presenting reasonable results. Meanwhile, in terms of the method′s sensitivity to both spatial and temporal displacements from the hypothetical origin, the method is also capable of consistently measuring the similarity of spatial and temporal patterns. The application of the method to assess both spatial and temporal pattern similarities between two deforestation data sets with different resolutions was also discussed.