3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Abstract The acoustic-based automatic speech recognition (ASR) technique has been a matured technique and widely seen to be used in numerous applications. However, acoustic-based ASR will not maintain a standard performance for the disabled group with an abnormal face, that is atypical eye or mouth geometrical characteristics. For governing this problem, this article develops a three-dimensional (3D) sensor lip image based pronunciation recognition system where the 3D sensor is efficiently used to acquire the action variations of the lip shapes of the pronunciation action from a speaker. In this work, two different types of 3D lip features for pronunciation recognition are presented, 3D-(x, y, z) coordinate lip feature and 3D geometry lip feature parameters. For the 3D-(x, y, z) coordinate lip feature design, 18 location points, each of which has 3D-sized coordinates, around the outer and inner lips are properly defined. In the design of 3D geometry lip features, eight types of features considering the geometrical space characteristics of the inner lip are developed. In addition, feature fusion to combine both 3D-(x, y, z) coordinate and 3D geometry lip features is further considered. The presented 3D sensor lip image based feature evaluated the performance and effectiveness using the principal component analysis based classification calculation approach. Experimental results on pronunciation recognition of two different datasets, Mandarin syllables and Mandarin phrases, demonstrate the competitive performance of the presented 3D sensor lip image based pronunciation recognition system.

Download Full-text

Neuroscience-Enabled Complex Visual Scene Understanding

10.21236/ada579652 ◽

2012 ◽

Author(s):

Laurent Itti ◽

Nader Noori ◽

Lior Elazary

Keyword(s):

Scene Understanding ◽

Visual Scene ◽

Visual Scene Understanding

Download Full-text

Numerical Analysis of the Bond Strength between Two Methacrylic Polymers by Surface Modification

Materials ◽

10.3390/ma14143927 ◽

2021 ◽

Vol 14 (14) ◽

pp. 3927

Author(s):

Joanna Taczała ◽

Katarzyna Rak ◽

Jacek Sawicki ◽

Michał Krasowski

Keyword(s):

Numerical Analysis ◽

Bond Strength ◽

Angular Position ◽

Fem Analysis ◽

Groove Depth ◽

Chemical Reagent ◽

Shear Process ◽

3D Geometry ◽

The Right ◽

Artificial Teeth

The creation of acrylic dentures involves many stages. One of them is to prepare the surfaces of artificial teeth for connection with the denture plates. The teeth could be rubbed with a chemical reagent, the surface could be developed, or retention hooks could be created. Preparation of the surface is used to improve the bond between the teeth and the plate. Choosing the right combination affects the length of denture use. This work focuses on a numerical analysis of grooving. The purpose of this article is to select the shape and size of the grooves that would most affect the quality of the bond strength. Two types of grooves in different dimensional configurations were analyzed. The variables were groove depth and width, and the distance between the grooves. Finally, 24 configurations were obtained. Models were analyzed in terms of their angular position to the loading force. Finite element method (FEM) analysis was performed on the 3D geometry created, which consisted of two polymer bodies under the shear process. The smallest values of the stresses and strains were characterized by a sample with parallel grooves with the grooving dimensions width 0.20 mm, thickness 0.10 mm, and distance between the grooves 5.00 mm, placed at an angle of 90°. The best dimensions from the parallel (III) and cross (#) grooves were compared experimentally. Specimens with grooving III were not damaged in the shear test. The research shows that the shape of the groove affects the distribution of stresses and strains. Combining the selected method with an adequately selected chemical reagent can significantly increase the strength of the connection.

Download Full-text

Extensive Analyses of Superconducting Cables 3D Geometry With Advanced Tomographic Examinations

IEEE Transactions on Applied Superconductivity ◽

10.1109/tasc.2021.3059600 ◽

2021 ◽

Vol 31 (5) ◽

pp. 1-5

Author(s):

L. Zani ◽

I. Tiseanu ◽

M. Chiletti ◽

D. Dumitru ◽

M. Lungu ◽

...

Keyword(s):

3D Geometry ◽

Superconducting Cables

Download Full-text

Semantic Relation Model and Dataset for Remote Sensing Scene Understanding

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10070488 ◽

2021 ◽

Vol 10 (7) ◽

pp. 488

Author(s):

Peng Li ◽

Dezheng Zhang ◽

Aziguli Wulamu ◽

Xin Liu ◽

Peng Chen

Keyword(s):

Remote Sensing ◽

Scene Understanding ◽

Deep Understanding ◽

Remote Sensing Images ◽

Convolutional Network ◽

Scene Graph ◽

Multi Scale ◽

Relationship Extraction ◽

High Level ◽

Graph Generation

A deep understanding of our visual world is more than an isolated perception on a series of objects, and the relationships between them also contain rich semantic information. Especially for those satellite remote sensing images, the span is so large that the various objects are always of different sizes and complex spatial compositions. Therefore, the recognition of semantic relations is conducive to strengthen the understanding of remote sensing scenes. In this paper, we propose a novel multi-scale semantic fusion network (MSFN). In this framework, dilated convolution is introduced into a graph convolutional network (GCN) based on an attentional mechanism to fuse and refine multi-scale semantic context, which is crucial to strengthen the cognitive ability of our model Besides, based on the mapping between visual features and semantic embeddings, we design a sparse relationship extraction module to remove meaningless connections among entities and improve the efficiency of scene graph generation. Meanwhile, to further promote the research of scene understanding in remote sensing field, this paper also proposes a remote sensing scene graph dataset (RSSGD). We carry out extensive experiments and the results show that our model significantly outperforms previous methods on scene graph generation. In addition, RSSGD effectively bridges the huge semantic gap between low-level perception and high-level cognition of remote sensing images.

Download Full-text

Semantics-Driven Remote Sensing Scene Understanding Framework for Grounded Spatio-Contextual Scene Descriptions

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10010032 ◽

2021 ◽

Vol 10 (1) ◽

pp. 32

Author(s):

Abhishek V. Potnis ◽

Surya S. Durbha ◽

Rajat C. Shinde

Keyword(s):

Remote Sensing ◽

Domain Knowledge ◽

Situational Awareness ◽

Scene Understanding ◽

Graph Representation ◽

Observation Data ◽

Test Scenario ◽

Semantic Web Technologies ◽

Earth Observation Data ◽

Contextual Domain

Earth Observation data possess tremendous potential in understanding the dynamics of our planet. We propose the Semantics-driven Remote Sensing Scene Understanding (Sem-RSSU) framework for rendering comprehensive grounded spatio-contextual scene descriptions for enhanced situational awareness. To minimize the semantic gap for remote-sensing-scene understanding, the framework puts forward the transformation of scenes by using semantic-web technologies to Remote Sensing Scene Knowledge Graphs (RSS-KGs). The knowledge-graph representation of scenes has been formalized through the development of a Remote Sensing Scene Ontology (RSSO)—a core ontology for an inclusive remote-sensing-scene data product. The RSS-KGs are enriched both spatially and contextually, using a deductive reasoner, by mining for implicit spatio-contextual relationships between land-cover classes in the scenes. The Sem-RSSU, at its core, constitutes novel Ontology-driven Spatio-Contextual Triple Aggregation and realization algorithms to transform KGs to render grounded natural language scene descriptions. Considering the significance of scene understanding for informed decision-making from remote sensing scenes during a flood, we selected it as a test scenario, to demonstrate the utility of this framework. In that regard, a contextual domain knowledge encompassing Flood Scene Ontology (FSO) has been developed. Extensive experimental evaluations show promising results, further validating the efficacy of this framework.

Download Full-text

Understanding Natural Disaster Scenes from Mobile Images Using Deep Learning

Applied Sciences ◽

10.3390/app11093952 ◽

2021 ◽

Vol 11 (9) ◽

pp. 3952

Author(s):

Shimin Tang ◽

Zhiqiang Chen

Keyword(s):

Deep Learning ◽

Natural Disaster ◽

Scene Understanding ◽

Computing Methods ◽

Classification Model ◽

Learning Approach ◽

Learning Models ◽

Damage Level ◽

Feature Extractor ◽

Mobile Imaging

With the ubiquitous use of mobile imaging devices, the collection of perishable disaster-scene data has become unprecedentedly easy. However, computing methods are unable to understand these images with significant complexity and uncertainties. In this paper, the authors investigate the problem of disaster-scene understanding through a deep-learning approach. Two attributes of images are concerned, including hazard types and damage levels. Three deep-learning models are trained, and their performance is assessed. Specifically, the best model for hazard-type prediction has an overall accuracy (OA) of 90.1%, and the best damage-level classification model has an explainable OA of 62.6%, upon which both models adopt the Faster R-CNN architecture with a ResNet50 network as a feature extractor. It is concluded that hazard types are more identifiable than damage levels in disaster-scene images. Insights are revealed, including that damage-level recognition suffers more from inter- and intra-class variations, and the treatment of hazard-agnostic damage leveling further contributes to the underlying uncertainties.

Download Full-text

Numerical Homogenization of Multi-Layered Corrugated Cardboard with Creasing or Perforation

Materials ◽

10.3390/ma14143786 ◽

2021 ◽

Vol 14 (14) ◽

pp. 3786

Author(s):

Tomasz Garbowski ◽

Anna Knitter-Piątkowska ◽

Damian Mrówczyński

Keyword(s):

Computational Models ◽

Load Capacity ◽

Torsional Stiffness ◽

Numerical Homogenization ◽

Corrugated Board ◽

3D Geometry ◽

Corrugated Cardboard ◽

Numerical Tools ◽

Packaging Industry ◽

Theoretical Considerations

The corrugated board packaging industry is increasingly using advanced numerical tools to design and estimate the load capacity of its products. This is why numerical analyses are becoming a common standard in this branch of manufacturing. Such trends cause either the use of advanced computational models that take into account the full 3D geometry of the flat and wavy layers of corrugated board, or the use of homogenization techniques to simplify the numerical model. The article presents theoretical considerations that extend the numerical homogenization technique already presented in our previous work. The proposed here homogenization procedure also takes into account the creasing and/or perforation of corrugated board (i.e., processes that undoubtedly weaken the stiffness and strength of the corrugated board locally). However, it is not always easy to estimate how exactly these processes affect the bending or torsional stiffness. What is known for sure is that the degradation of stiffness depends, among other things, on the type of cut, its shape, the depth of creasing as well as their position or direction in relation to the corrugation direction. The method proposed here can be successfully applied to model smeared degradation in a finite element or to define degraded interface stiffnesses on a crease line or a perforation line.

Download Full-text