scholarly journals Sym3DNet: Symmetric 3D Prior Network for Single-View 3D Reconstruction

Sensors ◽  
2022 ◽  
Vol 22 (2) ◽  
pp. 518
Author(s):  
Ashraf Siddique ◽  
Seungkyu Lee
Keyword(s):  
3D Reconstruction ◽  
Real World ◽  
Large Scale ◽  
Critical Role ◽  
Three Dimensional ◽  
Global Symmetry ◽  
Single View ◽  
3D Data ◽  
Multi Level ◽  
Symmetry Structure

The three-dimensional (3D) symmetry shape plays a critical role in the reconstruction and recognition of 3D objects under occlusion or partial viewpoint observation. Symmetry structure prior is particularly useful in recovering missing or unseen parts of an object. In this work, we propose Sym3DNet for single-view 3D reconstruction, which employs a three-dimensional reflection symmetry structure prior of an object. More specifically, Sym3DNet includes 2D-to-3D encoder-decoder networks followed by a symmetry fusion step and multi-level perceptual loss. The symmetry fusion step builds flipped and overlapped 3D shapes that are fed to a 3D shape encoder to calculate the multi-level perceptual loss. Perceptual loss calculated in different feature spaces counts on not only voxel-wise shape symmetry but also on the overall global symmetry shape of an object. Experimental evaluations are conducted on both large-scale synthetic 3D data (ShapeNet) and real-world 3D data (Pix3D). The proposed method outperforms state-of-the-art approaches in terms of efficiency and accuracy on both synthetic and real-world datasets. To demonstrate the generalization ability of our approach, we conduct an experiment with unseen category samples of ShapeNet, exhibiting promising reconstruction results as well.

scholarly journals Crystal structure of a tankyrase 1–telomere repeat factor 1 complex

2016 ◽  
Vol 72 (4) ◽  
pp. 320-327 ◽  
Author(s):  
Bo Li ◽  
Ruihong Qiao ◽  
Zhizhi Wang ◽  
Weihong Zhou ◽  
Xin Li ◽  
...  
Keyword(s):  
Crystal Structure ◽  
Large Scale ◽  
Sparse Matrix ◽  
Critical Role ◽  
Three Dimensional ◽  
Mitotic Cell ◽  
Ankyrin Repeat ◽  
Dimensional Structure ◽  
Telomere Repeat ◽  
Tankyrase 1

Telomere repeat factor 1 (TRF1) is a subunit of shelterin (also known as the telosome) and plays a critical role in inhibiting telomere elongation by telomerase. Tankyrase 1 (TNKS1) is a poly(ADP-ribose) polymerase that regulates the activity of TRF1 through poly(ADP-ribosyl)ation (PARylation). PARylation of TRF1 by TNKS1 leads to the release of TRF1 from telomeres and allows telomerase to access telomeres. The interaction between TRF1 and TNKS1 is thus important for telomere stability and the mitotic cell cycle. Here, the crystal structure of a complex between the N-terminal acidic domain of TRF1 (residues 1–55) and a fragment of TNKS1 covering the second and third ankyrin-repeat clusters (ARC2-3) is presented at 2.2 Å resolution. The TNKS1–TRF1 complex crystals were optimized using an `oriented rescreening' strategy, in which the initial crystallization condition was used as a guide for a second round of large-scale sparse-matrix screening. This crystallographic and biochemical analysis provides a better understanding of the TRF1–TNKS1 interaction and the three-dimensional structure of the ankyrin-repeat domain of TNKS.

scholarly journals PVRNet: Point-View Relation Neural Network for 3D Shape Recognition

2019 ◽  
Vol 33 ◽  
pp. 9119-9126 ◽  
Author(s):  
Haoxuan You ◽  
Yifan Feng ◽  
Xibin Zhao ◽  
Changqing Zou ◽  
Rongrong Ji ◽  
...  
Keyword(s):  
Point Cloud ◽  
Shape Recognition ◽  
Three Dimensional ◽  
Feature Representation ◽  
Research Attention ◽  
Single View ◽  
3D Shape ◽  
3D Data ◽  
Effective Network ◽  
Fusion Feature

Three-dimensional (3D) shape recognition has drawn much research attention in the field of computer vision. The advances of deep learning encourage various deep models for 3D feature representation. For point cloud and multi-view data, two popular 3D data modalities, different models are proposed with remarkable performance. However the relation between point cloud and views has been rarely investigated. In this paper, we introduce Point-View Relation Network (PVRNet), an effective network designed to well fuse the view features and the point cloud feature with a proposed relation score module. More specifically, based on the relation score module, the point-single-view fusion feature is first extracted by fusing the point cloud feature and each single view feature with point-singe-view relation, then the pointmulti- view fusion feature is extracted by fusing the point cloud feature and the features of different number of views with point-multi-view relation. Finally, the point-single-view fusion feature and point-multi-view fusion feature are further combined together to achieve a unified representation for a 3D shape. Our proposed PVRNet has been evaluated on ModelNet40 dataset for 3D shape classification and retrieval. Experimental results indicate our model can achieve significant performance improvement compared with the state-of-the-art models.

scholarly journals Impact Sound Generation for Audiovisual Interaction with Real-World Movable Objects in Building-Scale Virtual Reality

2021 ◽  
Vol 11 (16) ◽  
pp. 7546
Author(s):  
Katashi Nagao ◽  
Kaho Kumon ◽  
Kodai Hattori
Keyword(s):  
Virtual Reality ◽  
Real World ◽  
Large Scale ◽  
Virtual Space ◽  
Sound Generation ◽  
Generation System ◽  
Position Information ◽  
3D Data ◽  
The Impact ◽  
Audiovisual Interaction

In building-scale VR, where the entire interior of a large-scale building is a virtual space that users can walk around in, it is very important to handle movable objects that actually exist in the real world and not in the virtual space. We propose a mechanism to dynamically detect such objects (that are not embedded in the virtual space) in advance, and then generate a sound when one is hit with a virtual stick. Moreover, in a large indoor virtual environment, there may be multiple users at the same time, and their presence may be perceived by hearing, as well as by sight, e.g., by hearing sounds such as footsteps. We, therefore, use a GAN deep learning generation system to generate the impact sound from any object. First, in order to visually display a real-world object in virtual space, its 3D data is generated using an RGB-D camera and saved, along with its position information. At the same time, we take the image of the object and break it down into parts, estimate its material, generate the sound, and associate the sound with that part. When a VR user hits the object virtually (e.g., hits it with a virtual stick), a sound is generated. We demonstrate that users can judge the material from the sound, thus confirming the effectiveness of the proposed method.

The Application of 3D Laser Scanning Technology in Ancient Architecture Protection

2013 ◽  
Vol 353-356 ◽  
pp. 3476-3479
Author(s):  
Jun Lan Zhao ◽  
Ran Wu ◽  
Lei Wang ◽  
Yi Qin Wu
Keyword(s):  
3D Reconstruction ◽  
Data Acquisition ◽  
High Speed ◽  
Laser Scanning ◽  
Large Scale ◽  
High Efficiency ◽  
Image Data ◽  
High Accuracy ◽  
3D Laser Scanning ◽  
3D Data

The study of 3D laser scanning technology in Category Conservation is one of the hot researches in recent years. Through the high-speed laser scanning, catching the 3D data of an object in large-scale with high efficiency, high accuracy and excellent resolution, is a new way in 3D reconstruction and image data acquisition. The method has achieved good results through the experiment.

scholarly journals Hybrid inner-outer algorithm for solving real-world mechanical optimization problems

2021 ◽  
Vol 68 (1) ◽  
pp. 1-18
Author(s):  
Omnia Osman Fadel Abouhabaga ◽  
Mohamed Hassan Gadallah ◽  
Hanan Kamel Kouta ◽  
Mohamed Abass Zaghloul
Keyword(s):  
Real World ◽  
Large Scale ◽  
Optimization Problems ◽  
Critical Role ◽  
Computational Time ◽  
Design Problems ◽  
Engineering Design Problems ◽  
Mechanical Optimization ◽  
Future Work ◽  
Search Domain

AbstractIn the real world, the problems mostly are complex; more precisely, the problems generally are nonlinear or large scale other than if it was mandatory to resolve it under certain constraints, and that is common in engineering design problems. Therefore, the complexity of problem plays a critical role in determining the computational time and cost. Accordingly, a novel algorithm called inner-outer array is proposed in this paper. It depends on the design of parameters and then tolerance design as one of design of experiment stages. In this work, the inner-outer algorithm is used to solve real-world optimization problems to choose the preferable feasible regions of the entire search domain. Numerical results are documented and compared based on four well-known constrained mechanical engineering issues. It can be concluded that the performance of inner-outer algorithm is good to optimize constrained engineering problems, but it still needs some enhancements in the future work.

scholarly journals Three-Dimensional Reconstruction with a Laser Line Based on Image In-Painting and Multi-Spectral Photometric Stereo

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2131
Author(s):  
Liang Lu ◽  
Hongbao Zhu ◽  
Junyu Dong ◽  
Yakun Ju ◽  
Huiyu Zhou
Keyword(s):  
3D Reconstruction ◽  
Real World ◽  
Three Dimensional ◽  
Photometric Stereo ◽  
Laser Line ◽  
Spectral Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Image Dataset ◽  
Dimensional Reconstruction

This paper presents a multi-spectral photometric stereo (MPS) method based on image in-painting, which can reconstruct the shape using a multi-spectral image with a laser line. One of the difficulties in multi-spectral photometric stereo is to extract the laser line because the required illumination for MPS, e.g., red, green, and blue light, may pollute the laser color. Unlike previous methods, through the improvement of the network proposed by Isola, a Generative Adversarial Network based on image in-painting was proposed, to separate a multi-spectral image with a laser line into a clean laser image and an uncorrupted multi-spectral image without the laser line. Then these results were substituted into the method proposed by Fan to obtain high-precision 3D reconstruction results. To make the proposed method applicable to real-world objects, a rendered image dataset obtained using the rendering models in ShapeNet has been used for training the network. Evaluation using the rendered images and real-world images shows the superiority of the proposed approach over several previous methods.

scholarly journals Dynamic Multi-View Hashing for Online Image Retrieval

2017 ◽  
Author(s):  
Liang Xie ◽  
Jialie Shen ◽  
Jungong Han ◽  
Lei Zhu ◽  
Ling Shao
Keyword(s):  
Online Learning ◽  
Real World ◽  
Large Scale ◽  
Image Data ◽  
Superior Performance ◽  
Code Length ◽  
Single View ◽  
Image Organization ◽  
Hash Codes

Advanced hashing technique is essential to facilitate effective large scale online image organization and retrieval, where image contents could be frequently changed. Traditional multi-view hashing methods are developed based on batch-based learning, which leads to very expensive updating cost. Meanwhile, existing online hashing methods mainly focus on single-view data and thus can not achieve promising performance when searching real online images, which are multiple view based data. Further, both types of hashing methods can only produce hash code with fixed length. Consequently they suffer from limited capability to comprehensive characterization of streaming image data in the real world. In this paper, we propose dynamic multi-view hashing (DMVH), which can adaptively augment hash codes according to dynamic changes of image. Meanwhile, DMVH leverages online learning to generate hash codes. It can increase the code length when current code is not able to represent new images effectively. Moreover, to gain further improvement on overall performance, each view is assigned with a weight, which can be efficiently updated during the online learning process. In order to avoid the frequent updating of code length and view weights, an intelligent buffering scheme is also specifically designed to preserve significant data to maintain good effectiveness of DMVH. Experimental results on two real-world image datasets demonstrate superior performance of DWVH over several state-of-the-art hashing methods.

scholarly journals COMPREHENSIVE COMPARISON OF TWO IMAGE-BASED POINT CLOUDS FROM AERIAL PHOTOS WITH AIRBORNE LIDAR FOR LARGE-SCALE MAPPING

2017 ◽  
Vol XLII-2/W7 ◽  
pp. 557-565 ◽  
Author(s):  
E. Widyaningrum ◽  
B. G. H. Gorte
Keyword(s):  
Large Scale ◽  
Three Dimensional ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Topographic Map ◽  
Aerial Photos ◽  
3D Data ◽  
Comparison Results ◽  
Global Matching ◽  
3D Information

The integration of computer vision and photogrammetry to generate three-dimensional (3D) information from images has contributed to a wider use of point clouds, for mapping purposes. Large-scale topographic map production requires 3D data with high precision and accuracy to represent the real conditions of the earth surface. Apart from LiDAR point clouds, the image-based matching is also believed to have the ability to generate reliable and detailed point clouds from multiple-view images. In order to examine and analyze possible fusion of LiDAR and image-based matching for large-scale detailed mapping purposes, point clouds are generated by Semi Global Matching (SGM) and by Structure from Motion (SfM). In order to conduct comprehensive and fair comparison, this study uses aerial photos and LiDAR data that were acquired at the same time. Qualitative and quantitative assessments have been applied to evaluate LiDAR and image-matching point clouds data in terms of visualization, geometric accuracy, and classification result. The comparison results conclude that LiDAR is the best data for large-scale mapping.

scholarly journals A Hierarchical Machine Learning Approach for Multi-Level and Multi-Resolution 3D Point Cloud Classification

2020 ◽  
Vol 12 (16) ◽  
pp. 2598
Author(s):  
Simone Teruggi ◽  
Eleonora Grilli ◽  
Michele Russo ◽  
Francesco Fassi ◽  
Fabio Remondino
Keyword(s):  
Machine Learning ◽  
Point Cloud ◽  
Large Scale ◽  
3D Point Cloud ◽  
Cloud Data ◽  
Cloud Classification ◽  
3D Data ◽  
Structured Information ◽  
Multi Level ◽  
Point Cloud Classification

The recent years saw an extensive use of 3D point cloud data for heritage documentation, valorisation and visualisation. Although rich in metric quality, these 3D data lack structured information such as semantics and hierarchy between parts. In this context, the introduction of point cloud classification methods can play an essential role for better data usage, model definition, analysis and conservation. The paper aims to extend a machine learning (ML) classification method with a multi-level and multi-resolution (MLMR) approach. The proposed MLMR approach improves the learning process and optimises 3D classification results through a hierarchical concept. The MLMR procedure is tested and evaluated on two large-scale and complex datasets: the Pomposa Abbey (Italy) and the Milan Cathedral (Italy). Classification results show the reliability and replicability of the developed method, allowing the identification of the necessary architectural classes at each geometric resolution.

scholarly journals Semantic 3D Reconstruction for Robotic Manipulators with an Eye-In-Hand Vision System

2020 ◽  
Vol 10 (3) ◽  
pp. 1183 ◽  
Author(s):  
Fusheng Zha ◽  
Yu Fu ◽  
Pengfei Wang ◽  
Wei Guo ◽  
Mantian Li ◽  
...  
Keyword(s):  
3D Reconstruction ◽  
Large Scale ◽  
Vision System ◽  
Three Dimensional ◽  
Small Scale ◽  
Intelligent Control System ◽  
Segmented Images ◽  
Dimensional Reconstruction ◽  
Self Driving Cars

Three-dimensional reconstruction and semantic understandings have attracted extensive attention in recent years. However, current reconstruction techniques mainly target large-scale scenes, such as an indoor environment or automatic self-driving cars. There are few studies on small-scale and high-precision scene reconstruction for manipulator operation, which plays an essential role in the decision-making and intelligent control system. In this paper, a group of images captured from an eye-in-hand vision system carried on a robotic manipulator are segmented by deep learning and geometric features and create a semantic 3D reconstruction using a map stitching method. The results demonstrate that the quality of segmented images and the precision of semantic 3D reconstruction are effectively improved by our method.

Sign in / Sign up

Export Citation Format

Share Document