Three-Dimensional Motion and Structure Estimation Using Inertial Sensors and Computer Vision for Augmented Reality

2002 ◽  
Vol 11 (5) ◽  
pp. 474-492 ◽  
Author(s):  
Lin Chai ◽  
William A. Hoff ◽  
Tyrone Vincent
Keyword(s):  
Augmented Reality ◽  
Inertial Sensors ◽  
A Priori ◽  
3D Structure ◽  
Three Dimensional ◽  
Real Data ◽  
Single Camera ◽  
Extended Kalman Filters ◽  
Structure Estimation ◽  
Better Than

A new method for registration in augmented reality (AR) was developed that simultaneously tracks the position, orientation, and motion of the user's head, as well as estimating the three-dimensional (3D) structure of the scene. The method fuses data from head-mounted cameras and head-mounted inertial sensors. Two extended Kalman filters (EKFs) are used: one estimates the motion of the user's head and the other estimates the 3D locations of points in the scene. A recursive loop is used between the two EKFs. The algorithm was tested using a combination of synthetic and real data, and in general was found to perform well. A further test showed that a system using two cameras performed much better than a system using a single camera, although improving the accuracy of the inertial sensors can partially compensate for the loss of one camera. The method is suitable for use in completely unstructured and unprepared environments. Unlike previous work in this area, this method requires no a priori knowledge about the scene, and can work in environments in which the objects of interest are close to the user.

scholarly journals Monocular SLAM for Visual Odometry: A Full Approach to the Delayed Inverse-Depth Feature Initialization Method

2012 ◽  
Vol 2012 ◽  
pp. 1-26 ◽  
Author(s):  
Rodrigo Munguía ◽  
Antoni Grau
Keyword(s):  
Sensory Input ◽  
A Priori ◽  
Real Data ◽  
Visual Odometry ◽  
Visual Features ◽  
Single Camera ◽  
Localization And Mapping ◽  
Monocular Slam ◽  
Freely Moving ◽  
Depth Feature

This paper describes in a detailed manner a method to implement a simultaneous localization and mapping (SLAM) system based on monocular vision for applications of visual odometry, appearance-based sensing, and emulation of range-bearing measurements. SLAM techniques are required to operate mobile robots ina prioriunknown environments using only on-board sensors to simultaneously build a map of their surroundings; this map will be needed for the robot to track its position. In this context, the 6-DOF (degree of freedom) monocular camera case (monocular SLAM) possibly represents the harder variant of SLAM. In monocular SLAM, a single camera, which is freely moving through its environment, represents the sole sensory input to the system. The method proposed in this paper is based on a technique called delayed inverse-depth feature initialization, which is intended to initialize new visual features on the system. In this work, detailed formulation, extended discussions, and experiments with real data are presented in order to validate and to show the performance of the proposal.

scholarly journals MolAR: Bringing Chemical Structures to Life with Augmented Reality and Machine Learning

2021 ◽  
Author(s):  
Sukolsak Sakshuwong ◽  
Hayley Weir ◽  
Umberto Raucci ◽  
Todd J. Martínez
Keyword(s):  
Machine Learning ◽  
Augmented Reality ◽  
3D Structure ◽  
Three Dimensional ◽  
Real Life ◽  
Data Bank ◽  
Molecular Structures ◽  
Chemical Structures ◽  
Food And Drink ◽  
Research And Education

Visualizing three-dimensional molecular structures is crucial to understanding and predicting their chemical behavior. Existing visualization software, however, can be cumbersome to use, and, for many, hand-drawn skeletal structures remain the preferred method of chemical communication. Although convenient, the static, two-dimensional nature of these drawings can be misleading in conveying the molecule’s 3D structure, not to mention that dynamic movement is completely disregarded. Here, we combine machine learning and augmented reality (AR) to develop MolAR, an immersive mobile application for visualizing molecules in real-world scenes. The application uses deep learning to recognize hand-drawn hydrocarbons structures which it converts into interactive 3D molecules in AR. Users can also “hunt” for chemicals in food and drink to uncover molecules in their real-life environment. A variety of interesting molecules are pre-loaded into the application, and users can visualize molecules in PubChem by providing their name or SMILES string and proteins in the Protein Data Bank by providing their PDB ID. MolAR was designed to be used in both research and education settings, providing an almost barrierless platform to visualize and interact with 3D molecular structures in a uniquely immersive way.

A New 3D Coordinates Acquisition Method on Base of Single Camera

2013 ◽  
Vol 475-476 ◽  
pp. 296-300
Author(s):  
Sheng Hong Fan ◽  
Chang Ru Liu ◽  
Xiao Tong Qi ◽  
Lai Wei Jiang ◽  
Ya Jun Wu
Keyword(s):  
Measurement Accuracy ◽  
Three Dimensional ◽  
Single Camera ◽  
Mobile Objects ◽  
Plane Normal ◽  
Double Phase ◽  
Relationship Of ◽  
Acquisition Method ◽  
Better Than ◽  
Object Relative

As is known to us, conventional photogrammetry uses the double phase stereopair for forward intersection, but this paper proposed a new method for 3D coordinates acquisition based on single camera. Under the defined relationship of the object relative position, three-dimensional coordinates can be obtained by space resection through single photo. The principle of this method is simple but practical, especially suitable for measuring the overall mobile objects. Experiments show that within1500mm, plane normal direction is better than 0.02°in the measurement accuracy , and can replace stereopair measurement in certain conditions for its higher measurement accuracy.

The `seed-skewness' integration method generalized for three-dimensional Bragg peaks

2003 ◽  
Vol 36 (6) ◽  
pp. 1475-1479 ◽  
Author(s):  
J. Peters
Keyword(s):  
Statistical Analysis ◽  
High Resolution ◽  
Integration Method ◽  
A Priori ◽  
Reciprocal Lattice ◽  
Three Dimensional ◽  
Real Data ◽  
Area Detector ◽  
Peak Integration ◽  
Electronic Detector

The integration of the three-dimensional profile of each node of the reciprocal lattice without ana priorimodelling of the shape of the reflections is a prerequisite in order to improve the capability of area detectors in diffraction studies. Bolotovskyet al.[J. Appl. Cryst.(1995),28, 86–95] published a new method of area-detector peak integration based on a statistical analysis of pixel intensities and suggested its generalization for processing of high-resolution three-dimensional electronic detector data. This has been done in the present work, respecting the special requirements of data collected from neutron diffraction. The results are compared with other integration methods. It is shown that the seed-skewness method is successful in giving very reliable results and simultaneously optimizes the standard deviations. The integration procedures are applied to real data, which are refined and compared with benchmark results.

Geodynamic inversion with uncertain initial geometries

2021 ◽  
Author(s):  
Arne Spang ◽  
Tobias S. Baumann ◽  
Boris J. P. Kaus
Keyword(s):  
Subduction Zones ◽  
A Priori ◽  
Three Dimensional ◽  
Real Data ◽  
Imaging Data ◽  
Simple Method ◽  
Geometrical Properties ◽  
Geometric Uncertainties ◽  
Geophysical Imaging ◽  
Geological Formations

<p>Advanced numerical methods and increasing computational power have enabled us to incorporate numerical forward models into geodynamic inverse frameworks. We now have several strategies to constrain the rheological properties of the crust and lithosphere. Yet, the initial geometry of geological formations (e.g., salt bodies, magma bodies, subducting slabs) and associated uncertainties are, in most cases, excluded from the inverse problem and assumed to be part of the a priori knowledge. Usually, geometrical properties remain constant, or we employ simplified bodies like planes, spheres or ellipsoids for their parameterization.</p><p>Here, we present a simple method to parameterize complex three-dimensional bodies and incorporate them into geodynamic inverse problems. Our approach enables us to automatically create an entire ensemble of initial geometries within the uncertainty limits of geophysical imaging data. This not only allows us to account for geometric uncertainties, but also enables us to investigate the sensitivity of geophysical data to the geometrical properties of the geological structures.</p><p>We present 3 areas of application for our method, covering salt diapirs, magmatic systems and subduction zones, using both synthetic and real data.</p>

Real-Time Structure Estimation in Dynamic Scenes Using a Single Camera

2013 ◽  
pp. 173-191
Author(s):  
Ashwin P. Dani ◽  
Zhen Kan ◽  
Nic Fischer ◽  
Warren E. Dixon
Keyword(s):  
Moving Objects ◽  
3D Structure ◽  
Estimation Algorithm ◽  
Unknown Input ◽  
Single Camera ◽  
Time Varying ◽  
Moving Camera ◽  
Structure Estimation ◽  
Modeling Uncertainties ◽  
Exponentially Stable

In this chapter, an online method is developed for estimating 3D structure (with proper scale) of moving objects seen by a moving camera. In contrast to traditionally developed batch solutions for this problem, a nonlinear unknown input observer strategy is used where the object’s velocity is considered as an unknown input to the perspective dynamical system. The estimator is exponentially stable, and hence, provides robustness against modeling uncertainties and measurement noise from the camera. The developed method provides first causal, observer based structure estimation algorithm for a moving camera viewing a moving object with unknown time-varying object velocities.

scholarly journals Convolutional Neural Networks Predict Fish Abundance from Underlying Coral Reef Texture

2018 ◽  
Author(s):  
Grace Young ◽  
Vassileios Balntas ◽  
Victor Prisacariu
Keyword(s):  
Spatial Scales ◽  
A Priori ◽  
3D Structure ◽  
Three Dimensional ◽  
Structural Complexity ◽  
3D Models ◽  
Fish Abundance ◽  
Ecological Patterns ◽  
Abundance And Diversity ◽  
First Time

Coral reefs are among the most biodiverse ecosystems on Earth in large part owing to their unique three-dimensional (3D) structure, which provides niches for a variety of species. Metrics of structural complexity have been shown to correlate with the abundance and diversity of fish and other marine organisms, but they are imperfect representations of a surface that can oversimplify key structural elements and bias discoveries. Moreover, they require researchers to make relatively uninformed guesses about the features and spatial scales relevant to species of interest. This paper introduces a machine-learning method for automating inferences about fish abundance from reef 3D models. It demonstrates the capacity of a convolutional neural network (ConvNet) to learn ecological patterns that are extremely subtle, if not invisible, to the human eye. It is the first time in the literature that no a priori assumptions are made about the bathymetry–fish relationship.

Three-dimensional gravity inversion using graph theory to delineate the skeleton of homogeneous sources

Geophysics ◽  
2015 ◽  
Vol 80 (2) ◽  
pp. G53-G66 ◽  
Author(s):  
Rodrigo Bijani ◽  
Cosme F. Ponte-Neto ◽  
Dionisio U. Carlos ◽  
Fernando J. S. Silva Dias
Keyword(s):  
Genetic Algorithm ◽  
Graph Theory ◽  
Minimum Spanning Tree ◽  
Gravity Data ◽  
A Priori ◽  
Three Dimensional ◽  
Real Data ◽  
Gravity Inversion ◽  
Inversion Method ◽  
Dimensional Gravity

We developed a new strategy, based on graph theory concepts, to invert gravity data using an ensemble of simple point masses. Our method consisted of a genetic algorithm with elitism to generate a set of possible solutions. Each estimate was associated to a graph to solve the minimum spanning tree (MST) problem. To produce unique and stable estimates, we restricted the position of the point masses by minimizing the statistical variance of the distances of an MST jointly with the data-misfit function during the iterations of the genetic algorithm. Hence, the 3D spatial distribution of the point masses identified the skeleton of homogeneous gravity sources. In addition, our method also gave an estimation of the anomalous mass of the source. So, together with the anomalous mass, the skeleton could aid other 3D methods with promising geometric a priori parameters. Several tests with different values of regularizing parameter were made to bespeak this new regularizing strategy. The inversion results applied to noise-corrupted synthetic gravity data revealed that, regardless of promising starting models, the estimated distribution of point masses and the anomalous mass offered valuable information about the homogeneous sources in the subsurface. Tests on real data from a portion of Quadrilátero Ferrífero, Minas Gerais state, Brazil, were performed for complementary analysis of the proposed inversion method.

scholarly journals Evaluation and comparison of methods for recapitulation of 3D spatial chromatin structures

2017 ◽  
Vol 20 (4) ◽  
pp. 1205-1214
Author(s):  
Jincheol Park ◽  
Shili Lin
Keyword(s):  
3D Structure ◽  
Three Dimensional ◽  
Simulated Data ◽  
Real Data ◽  
Data Sets ◽  
Contact Map ◽  
Chromosome Conformation ◽  
Contact Maps ◽  
Data Resolution ◽  
Genomic Scale

Abstract How chromosomes fold and how distal genomic elements interact with one another at a genomic scale have been actively pursued in the past decade following the seminal work describing the Chromosome Conformation Capture (3C) assay. Essentially, 3C-based technologies produce two-dimensional (2D) contact maps that capture interactions between genomic fragments. Accordingly, a plethora of analytical methods have been proposed to take a 2D contact map as input to recapitulate the underlying whole genome three-dimensional (3D) structure of the chromatin. However, their performance in terms of several factors, including data resolution and ability to handle contact map features, have not been sufficiently evaluated. This task is taken up in this article, in which we consider several recent and/or well-regarded methods, both optimization-based and model-based, for their aptness of producing 3D structures using contact maps generated based on a population of cells. These methods are evaluated and compared using both simulated and real data. Several criteria have been used. For simulated data sets, the focus is on accurate recapitulation of the entire structure given the existence of the gold standard. For real data sets, comparison with distances measured by Florescence in situ Hybridization and consistency with several genomic features of known biological functions are examined.

Simple Measurement and Annotation Technique of Real Objects in Augmented Reality Environments

2013 ◽  
Author(s):  
Giandomenico Caruso ◽  
Samuele Polistina ◽  
Monica Bordegoni
Keyword(s):  
Augmented Reality ◽  
A Priori ◽  
Three Dimensional ◽  
Video Stream ◽  
Real Object ◽  
Object A ◽  
Virtual Plane ◽  
Real Objects ◽  
Simple Measurement ◽  
3D Scene

The paper describes a technique that allows measuring and annotating real objects in an Augmented Reality (AR) environment. The technique is based on the marker tracking, and aims at enabling the user to define the three-dimensional position of points, within the AR scene, by selecting them directly on the video stream. The technique consists in projecting the points, which are directly selected on the monitor, on a virtual plane defined according to the bi-dimensional marker, which is used for the tracking. This plane can be seen as a virtual depth cue that helps the user to place these points in the desired position. The user can also move this virtual plane to place points within the whole 3D scene. By using this technique, the user can place virtual points around a real object with the aim of taking some measurements of the object, by calculating the minimum distance between the points, or in order to put some annotations on the object. Up to date, these kinds of activities can be carried out by using more complex systems or it is needed to know the shape of the real object a priori. The paper describes the functioning principles of the proposed technique and discusses the results of a testing session carried out with users to evaluate the overall precision and accuracy.

Sign in / Sign up

Export Citation Format

Share Document