Object Reconstruction Based on Attentive Recurrent Network from Single and Multiple Images

AbstractThe application of traditional 3D reconstruction methods such as structure-from-motion and simultaneous localization and mapping are typically limited by illumination conditions, surface textures, and wide baseline viewpoints in the field of robotics. To solve this problem, many researchers have applied learning-based methods with convolutional neural network architectures. However, simply utilizing convolutional neural networks without taking other measures into account is computationally intensive, and the results are not satisfying. In this study, to obtain the most informative images for reconstruction, we introduce a residual block to a 2D encoder for improved feature extraction, and propose an attentive latent unit that makes it possible to select the most informative image being fed into the network rather than choosing one at random. The recurrent visual attentive network is injected into the auto-encoder network using reinforcement learning. The recurrent visual attentive network pays more attention to useful images, and the agent will quickly predict the 3D volume. This model is evaluated based on both single- and multi-view reconstructions. The experiment results show that the recurrent visual attentive network increases prediction performance in a way that is superior to other alternative methods, and our model has desirable capacity for generalization.

Download Full-text

Computational Geometry-Based Surface Reconstruction for Volume Estimation: A Case Study on Magnitude-Frequency Relations for a LiDAR-Derived Rockfall Inventory

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10030157 ◽

2021 ◽

Vol 10 (3) ◽

pp. 157

Author(s):

Paul-Mark DiFrancesco ◽

David A. Bonneau ◽

D. Jean Hutchinson

Keyword(s):

Computational Geometry ◽

Surface Reconstruction ◽

Power Law ◽

Point Cloud ◽

Point Clouds ◽

Volume Estimation ◽

Estimation Methods ◽

Small Scale ◽

Reconstruction Methods ◽

3D Volume

Key to the quantification of rockfall hazard is an understanding of its magnitude-frequency behaviour. Remote sensing has allowed for the accurate observation of rockfall activity, with methods being developed for digitally assembling the monitored occurrences into a rockfall database. A prevalent challenge is the quantification of rockfall volume, whilst fully considering the 3D information stored in each of the extracted rockfall point clouds. Surface reconstruction is utilized to construct a 3D digital surface representation, allowing for an estimation of the volume of space that a point cloud occupies. Given various point cloud imperfections, it is difficult for methods to generate digital surface representations of rockfall with detailed geometry and correct topology. In this study, we tested four different computational geometry-based surface reconstruction methods on a database comprised of 3668 rockfalls. The database was derived from a 5-year LiDAR monitoring campaign of an active rock slope in interior British Columbia, Canada. Each method resulted in a different magnitude-frequency distribution of rockfall. The implications of 3D volume estimation were demonstrated utilizing surface mesh visualization, cumulative magnitude-frequency plots, power-law fitting, and projected annual frequencies of rockfall occurrence. The 3D volume estimation methods caused a notable shift in the magnitude-frequency relations, while the power-law scaling parameters remained relatively similar. We determined that the optimal 3D volume calculation approach is a hybrid methodology comprised of the Power Crust reconstruction and the Alpha Solid reconstruction. The Alpha Solid approach is to be used on small-scale point clouds, characterized with high curvatures relative to their sampling density, which challenge the Power Crust sampling assumptions.

Download Full-text

Reconstruction Techniques in IceCube using Convolutional and Generative Neural Networks

EPJ Web of Conferences ◽

10.1051/epjconf/201920705005 ◽

2019 ◽

Vol 207 ◽

pp. 05005 ◽

Cited By ~ 2

Author(s):

Mirco Huennefeld

Keyword(s):

Neural Networks ◽

High Energy Physics ◽

High Energy ◽

Reconstruction Method ◽

Network Architectures ◽

Reconstruction Methods ◽

Physics Potential ◽

Reconstruction Performance ◽

Maximum Likelihood Methods ◽

First Results

Reliable and accurate reconstruction methods are vital to the success of high-energy physics experiments such as IceCube. Machine learning based techniques, in particular deep neural networks, can provide a viable alternative to maximum-likelihood methods. However, most common neural network architectures were developed for other domains such as image recogntion. While these methods can enhance the reconstruction performance in IceCube, there is much potential for tailored techniques. In the typical physics use-case, many symmetries, invariances and prior knowledge exist in the data, which are not fully exploited by current network architectures. Novel and specialized deep learning based reconstruction techniques are desired which can leverage the physics potential of experiments like IceCube. A reconstruction method using convolutional neural networks is presented which can significantly increase the reconstruction accuracy while greatly reducing the runtime in comparison to standard reconstruction methods in Ice- Cube. In addition, first results are discussed for future developments based on generative neural networks.

Download Full-text

A Comparison of Volumetric Reconstruction Methods of Archaeological Deposits Using Point-Cloud Data from Ahuahu, Aotearoa New Zealand

Remote Sensing ◽

10.3390/rs13194015 ◽

2021 ◽

Vol 13 (19) ◽

pp. 4015

Author(s):

Joshua Emmitt ◽

Patricia Pillay ◽

Matthew Barrett ◽

Stacey Middleton ◽

Timothy Mackrell ◽

...

Keyword(s):

New Zealand ◽

Computer Aided Design ◽

Three Dimensional ◽

Point Clouds ◽

Terrestrial Lidar ◽

Cloud Data ◽

Aotearoa New Zealand ◽

3D Data ◽

Reconstruction Methods ◽

Localization And Mapping

Collection of 3D data in archaeology is a long-standing practice. Traditionally, the focus of these data has been visualization as opposed to analysis. Three-dimensional data are often recorded during archaeological excavations, with the provenience of deposits, features, and artefacts documented by a variety of methods. Simple analysis of 3D data includes calculating the volumes of bound entities, such as deposits and features, and determining the spatial relationships of artifacts within these. The construction of these volumes presents challenges that originate in computer-aided design (CAD) but have implications for how data are used in archaeological analysis. We evaluate 3D construction processes using data from Waitetoke, Ahuahu Great Mercury Island, Aotearoa, New Zealand. Point clouds created with data collected by total station, photogrammetry, and terrestrial LiDAR using simultaneous localization and mapping (SLAM) are compared, as well as different methods for generating surface area and volumes with triangulated meshes and convex hulls. The differences between methods are evaluated and assessed in relation to analyzing artifact densities within deposits. While each method of 3D data acquisition and modeling has advantages in terms of accuracy and precision, other factors such as data collection and processing times must be considered when deciding on the most suitable.

Download Full-text

Estimation of Dynamical Neuron Nets’ Errors for Measuring Systems

Metrology and instruments ◽

10.33955/2307-2180(3)2018.33-35 ◽

2018 ◽

pp. 33-35

Author(s):

Leit Akhmed Mustafa Al Ravashdekh ◽

I. Ruzhentsev

Keyword(s):

Dynamic Systems ◽

Kalman Filters ◽

Learning Algorithm ◽

Measurement Data ◽

Recurrent Network ◽

Digital Measurement ◽

Network Architectures ◽

Navigation Systems ◽

Measuring Systems ◽

Satellite Navigation Systems

In the work based on the analysis of application of satellite navigation systems for determining position of moving traffic objects is proposed during the processing of the digital measurement data using artificial neural network. To perform modeling of nonlinear dynamic systems, it is suggested to use recurrent network architectures and a learning algorithm based on the theory of Kalman filters.

Download Full-text

3D RECONSTRUCTION FROM MULTI-VIEW MEDICAL X-RAY IMAGES – REVIEW AND EVALUATION OF EXISTING METHODS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-1-w5-319-2015 ◽

2015 ◽

Vol XL-1-W5 ◽

pp. 319-326 ◽

Cited By ~ 3

Author(s):

S. Hosseinian ◽

H. Arefi

Keyword(s):

3D Reconstruction ◽

Imaging Techniques ◽

Weight Bearing ◽

Computer Assisted Surgery ◽

3D Models ◽

Alternative Methods ◽

Computer Assisted ◽

X Ray ◽

Advantages And Disadvantages ◽

Reconstruction Methods

The 3D concept is extremely important in clinical studies of human body. Accurate 3D models of bony structures are currently required in clinical routine for diagnosis, patient follow-up, surgical planning, computer assisted surgery and biomechanical applications. However, 3D conventional medical imaging techniques such as computed tomography (CT) scan and magnetic resonance imaging (MRI) have serious limitations such as using in non-weight-bearing positions, costs and high radiation dose(for CT). Therefore, 3D reconstruction methods from biplanar X-ray images have been taken into consideration as reliable alternative methods in order to achieve accurate 3D models with low dose radiation in weight-bearing positions. Different methods have been offered for 3D reconstruction from X-ray images using photogrammetry which should be assessed. In this paper, after demonstrating the principles of 3D reconstruction from X-ray images, different existing methods of 3D reconstruction of bony structures from radiographs are classified and evaluated with various metrics and their advantages and disadvantages are mentioned. Finally, a comparison has been done on the presented methods with respect to several metrics such as accuracy, reconstruction time and their applications. With regards to the research, each method has several advantages and disadvantages which should be considered for a specific application.

Download Full-text

Forex exchange rate forecasting using deep recurrent neural networks

Digital Finance ◽

10.1007/s42521-020-00019-x ◽

2020 ◽

Vol 2 (1-2) ◽

pp. 69-96 ◽

Cited By ~ 1

Author(s):

Alexander Jakob Dautel ◽

Wolfgang Karl Härdle ◽

Stefan Lessmann ◽

Hsin-Vonn Seow

Keyword(s):

Neural Network ◽

Deep Learning ◽

Exchange Rate ◽

Language Processing ◽

Short Term Memory ◽

Recurrent Network ◽

Network Architectures ◽

Exchange Rate Forecasting ◽

Trading Model ◽

Gated Recurrent Units

Abstract Deep learning has substantially advanced the state of the art in computer vision, natural language processing, and other fields. The paper examines the potential of deep learning for exchange rate forecasting. We systematically compare long short-term memory networks and gated recurrent units to traditional recurrent network architectures as well as feedforward networks in terms of their directional forecasting accuracy and the profitability of trading model predictions. Empirical results indicate the suitability of deep networks for exchange rate forecasting in general but also evidence the difficulty of implementing and tuning corresponding architectures. Especially with regard to trading profit, a simpler neural network may perform as well as if not better than a more complex deep neural network.

Download Full-text

Scalable Models of Antibody Evolution and Benchmarking of Clonal Tree Reconstruction Methods

10.1101/2020.09.17.302505 ◽

2020 ◽

Author(s):

Chao Zhang ◽

Andrey V. Bzikadze ◽

Yana Safonova ◽

Siavash Mirarab

Keyword(s):

B Cell ◽

High Throughput Sequencing ◽

Phylogenetic Reconstruction ◽

Affinity Maturation ◽

Alternative Methods ◽

Tree Reconstruction ◽

Tree Expansion ◽

Reconstruction Methods ◽

Adequate Accuracy ◽

Better Than

AbstractAffinity maturation (AM) of antibodies through somatic hypermutations (SHMs) enables the immune system to evolve to recognize diverse pathogens. The accumulation of SHMs leads to the formation of clonal trees of antibodies produced by B cells that have evolved from a common naive B cell. Recent advances in high-throughput sequencing have enabled deep scans of antibody repertoires, paving the way for reconstructing clonal trees. However, it is not clear if clonal trees, which capture micro-evolutionary time scales, can be reconstructed using traditional phylogenetic reconstruction methods with adequate accuracy. In fact, several clonal tree reconstruction methods have been developed to fix supposed shortcomings of phylogenetic methods. Nevertheless, no consensus has been reached regarding the relative accuracy of these methods, partially because evaluation is challenging. Benchmarking the performance of existing methods and developing better methods would both benefit from realistic models of clonal tree evolution specifically designed for emulating B cell evolution. In this paper, we propose a model for modeling B cell clonal tree evolution and use this model to benchmark several existing clonal tree reconstruction methods. Our model, designed to be extensible, has several features: by evolving the clonal tree and sequences simultaneously, it allows modelling selective pressure due to changes in affinity binding; it enables scalable simulations of millions of cells; it enables several rounds of infection by an evolving pathogen; and, it models building of memory. In addition, we also suggest a set of metrics for comparing clonal trees and for measuring their properties. Our benchmarking results show that while maximum likelihood phylogenetic reconstruction methods can fail to capture key features of clonal tree expansion if applied naively, a very simple postprocessing of their results, where super short branches are contracted, leads to inferences that are better than alternative methods.

Download Full-text

A Novel RGB-D SLAM Algorithm Based on Cloud Robotics

Sensors ◽

10.3390/s19235288 ◽

2019 ◽

Vol 19 (23) ◽

pp. 5288 ◽

Cited By ~ 1

Author(s):

Yanli Liu ◽

Heng Zhang ◽

Chao Huang

Keyword(s):

Feature Extraction ◽

Point Cloud ◽

Experimental Validation ◽

Simultaneous Localization And Mapping ◽

Point Cloud Registration ◽

Cloud Robotics ◽

Localization And Mapping ◽

Computationally Intensive ◽

Pose Optimization ◽

Slam Algorithm

In this paper, we present a novel red-green-blue-depth simultaneous localization and mapping (RGB-D SLAM) algorithm based on cloud robotics, which combines RGB-D SLAM with the cloud robot and offloads the back-end process of the RGB-D SLAM algorithm to the cloud. This paper analyzes the front and back parts of the original RGB-D SLAM algorithm and improves the algorithm from three aspects: feature extraction, point cloud registration, and pose optimization. Experiments show the superiority of the improved algorithm. In addition, taking advantage of the cloud robotics, the RGB-D SLAM algorithm is combined with the cloud robot and the back-end part of the computationally intensive algorithm is offloaded to the cloud. Experimental validation is provided, which compares the cloud robotic-based RGB-D SLAM algorithm with the local RGB-D SLAM algorithm. The results of the experiments demonstrate the superiority of our framework. The combination of cloud robotics and RGB-D SLAM can not only improve the efficiency of SLAM but also reduce the robot’s price and size.

Download Full-text

Self-Optimizing Loop Sifting and Majorization for 3D Reconstruction

10.1115/detc2021-71393 ◽

2021 ◽

Author(s):

Guoxiang Zhang ◽

YangQuan Chen

Keyword(s):

3D Reconstruction ◽

Large Scale ◽

Autonomous Vehicle ◽

Optimization Methods ◽

Indoor Environments ◽

Reconstruction Methods ◽

Localization And Mapping ◽

Loop Detection ◽

Single User ◽

Public Datasets

Abstract Visual simultaneous localization and mapping (vSLAM) and 3D reconstruction methods have gone through impressive progress. These methods are very promising for autonomous vehicle and consumer robot applications because they can map large-scale environments such as cities and indoor environments without the need for much human effort. However, when it comes to loop detection and optimization, there is still room for improvement. vSLAM systems tend to add the loops very conservatively to reduce the severe influence of the false loops. These conservative checks usually lead to correct loops rejected, thus decrease performance. In this paper, an algorithm that can sift and majorize loop detections is proposed. Our proposed algorithm can compare the usefulness and effectiveness of different loops with the dense map posterior (DMP) metric. The algorithm tests and decides the acceptance of each loop without a single user-defined threshold. Thus it is adaptive to different data conditions. The proposed method is general and agnostic to sensor type (as long as depth or LiDAR reading presents), loop detection, and optimization methods. Neither does it require a specific type of SLAM system. Thus it has great potential to be applied to various application scenarios. Experiments are conducted on public datasets. Results show that the proposed method outperforms state-of-the-art methods.

Download Full-text

Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.1803839115 ◽

2018 ◽

Vol 115 (44) ◽

pp. E10467-E10475 ◽

Cited By ~ 26

Author(s):

Nicolas Y. Masse ◽

Gregory D. Grant ◽

David J. Freedman

Keyword(s):

High Performance ◽

Recurrent Network ◽

Network Architectures ◽

Complementary Method ◽

Computational Overhead ◽

Large Numbers ◽

Context Dependent ◽

The Brain ◽

Synaptic Stabilization

Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically causes them to forget previously learned tasks. This phenomenon is the result of “catastrophic forgetting,” in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly nonoverlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks, particularly when combined with weight stabilization. We show that this method works for both feedforward and recurrent network architectures, trained using either supervised or reinforcement-based learning. This suggests that using multiple, complementary methods, akin to what is believed to occur in the brain, can be a highly effective strategy to support continual learning.

Download Full-text