Improving RGB-D SLAM in dynamic environments using semantic aided segmentation

Summary Most conventional simultaneous localization and mapping (SLAM) approaches assume the working environment to be static. In a highly dynamic environment, this assumption divulges the impediments of a SLAM algorithm that lack modules that distinctively attend to dynamic objects despite the inclusion of optimization techniques. This work exploits such environments and reduces the effects of dynamic objects in a SLAM algorithm by separating features belonging to dynamic objects and static background using a generated binary mask image. While the features belonging to the static region are used for performing SLAM, the features belonging to non-static segments are reused instead of being eliminated. The approach employs deep neural network or DNN-based object detection module to obtain bounding boxes and then generates a lower resolution binary mask image using depth-first search algorithm over the detected semantics, characterizing the segmentation of the foreground from the static background. In addition, the features belonging to dynamic objects are tracked into consecutive frames to obtain better masking consistency. The proposed approach is tested on both publicly available dataset as well as self-collected dataset, which includes both indoor and outdoor environments. The experimental results show that the removal of features belonging to dynamic objects for a SLAM algorithm can significantly improve the overall output in a dynamic scene.

Robust Tensor Decomposition based Background/Foreground Separation in Noisy Videos and Its Applications in Additive Manufacturing

Background/foreground separation is one of the most fundamental tasks in computer vision, especially for video data. Robust PCA (RPCA) and its tensor extension, namely, Robust Tensor PCA (RTPCA), provide an effective framework for background/foreground separation by decomposing the data into low-rank and sparse components, which contain the background and the foreground (moving objects), respectively. However, in real-world applications, the video data is contaminated with noise. For example, in metal additive manufacturing (AM), the processed X-ray video to study melt pool dynamics is very noisy. RPCA and RTPCA are not able to separate the background, foreground, and noise simultaneously. As a result, the noise will contaminate the background or the foreground or both. There is a need to remove the noise from the background and foreground. To achieve the three terms decomposition, a smooth sparse Robust Tensor Decomposition (SS-RTD) model is proposed to decompose the data into static background, smooth foreground, and noise, respectively. Specifically, the static background is modeled by the low-rank tucker decomposition, the smooth foreground (moving objects) is modeled by the spatiotemporal continuity, which is enforced by the total variation regularization, and the noise is modeled by the sparsity, which is enforced by the L1 norm. An efficient algorithm based on alternating direction method of multipliers (ADMM) is implemented to solve the proposed model. Extensive experiments on both simulated and real data demonstrate that the proposed method significantly outperforms the state-of-the-art approaches for background/foreground separation in noisy cases.<br>

Development of Convective-Scale Static Background Error Covariance within GSI-Based Hybrid EnVar System for Direct Radar Reflectivity Data Assimilation

A convective-scale static background-error covariance (BEC) matrix is further developed to include the capability of direct reflectivity assimilation and evaluated within the GSI-based 3-dimensional variational (3DVar) and hybrid ensemble-variational (EnVar) methods. Specific developments are summarized as follows: 1) Control variables (CVs) are extended to include reflectivity, vertical velocity, and all hydrometeor types. Various horizontal momentum and moisture CV options are included. 2) Cross-correlations between all CVs are established. 3) A storm intensity-dependent binning method is adopted to separately calculate static error matrices for clear-air and storms with varying intensities. The resultant static BEC matrices are simultaneously applied at proper locations guided by the observed reflectivity. 4) The EnVar is extended to adaptively incorporate static BECs based on the quality of ensemble covariances.Evaluation and examination of the new static BECs are first performed on the 8 May 2003 Oklahoma City supercell. Detailed diagnostics and 3DVar examinations suggest zonal/meridian winds and pseudo-relative humidity are selected horizontal momentum and moisture CVs for direct reflectivity assimilation, respectively; inclusion of cross-correlations favors to spinup and maintain the analyzed storms; application of binning improves characteristics and persistence of the simulated storm. Relative to an experiment using the full ensemble BECs (Exp-PureEnVar), incorporating static BECs in hybrid EnVar reduces spinup time and better analyzes reflectivity distributions while the background ensemble is deficient in sampling errors. Compared to both pure 3DVar and Exp-PureEnVar, hybrid EnVar better predicts reflectivity distributions and better maintains strong mesocyclone. Further examination through the 20 May 2013 Oklahoma supercells confirms these results and additionally demonstrates the effectiveness of adaptive hybridization.

Robust Tensor PCA based Background/Foreground Separation in Noisy Videos and Its Applications in Additive Manufacturing

<div>Background/foreground separation is one of the most fundamental tasks in computer vision, especially for video data. Robust PCA (RPCA) and its tensor extension, namely, Robust Tensor PCA (RTPCA), provide an effective framework for background/foreground separation by decomposing the data into low-rank and sparse components, which contain the background and the foreground (moving objects), respectively. However, in real-world applications, the video data is contaminated with noise. For example, in metal additive manufacturing (AM), the processed X-ray video to study melt pool dynamics is very noisy. RPCA and RTPCA are not able to separate the background, foreground, and noise simultaneously. As a result, the noise will contaminate the background or the foreground or both. There is a need to remove the noise from the background and foreground. To achieve the three terms decomposition, a smooth sparse RTPCA (SS-RTPCA) model is proposed to decompose the data into static background, smooth foreground, and noise, respectively. Specifically, the static background is modeled by the low-rank tucker decomposition, the smooth foreground (moving objects) is modeled by the spatio-temporal continuity, which is enforced by the total variation regularization, and the noise is modeled by the sparsity, which is enforced by the `1 norm. An efficient algorithm based on alternating direction method of multipliers (ADMM) is implemented to solve the proposed model. Extensive experiments on both simulated and real data demonstrate that the proposed method significantly outperforms the state-of-the-art approaches for background/foreground separation in noisy cases.</div>

Robust Tensor PCA based Background/Foreground Separation in Noisy Videos and Its Applications in Additive Manufacturing

<div>Background/foreground separation is one of the most fundamental tasks in computer vision, especially for video data. Robust PCA (RPCA) and its tensor extension, namely, Robust Tensor PCA (RTPCA), provide an effective framework for background/foreground separation by decomposing the data into low-rank and sparse components, which contain the background and the foreground (moving objects), respectively. However, in real-world applications, the video data is contaminated with noise. For example, in metal additive manufacturing (AM), the processed X-ray video to study melt pool dynamics is very noisy. RPCA and RTPCA are not able to separate the background, foreground, and noise simultaneously. As a result, the noise will contaminate the background or the foreground or both. There is a need to remove the noise from the background and foreground. To achieve the three terms decomposition, a smooth sparse RTPCA (SS-RTPCA) model is proposed to decompose the data into static background, smooth foreground, and noise, respectively. Specifically, the static background is modeled by the low-rank tucker decomposition, the smooth foreground (moving objects) is modeled by the spatio-temporal continuity, which is enforced by the total variation regularization, and the noise is modeled by the sparsity, which is enforced by the `1 norm. An efficient algorithm based on alternating direction method of multipliers (ADMM) is implemented to solve the proposed model. Extensive experiments on both simulated and real data demonstrate that the proposed method significantly outperforms the state-of-the-art approaches for background/foreground separation in noisy cases.</div>

DOE-SLAM: Dynamic Object Enhanced Visual SLAM

In this paper, we formulate a novel strategy to adapt monocular-vision-based simultaneous localization and mapping (vSLAM) to dynamic environments. When enough background features can be captured, our system not only tracks the camera trajectory based on static background features but also estimates the foreground object motion from object features. In cases when a moving object obstructs too many background features for successful camera tracking from the background, our system can exploit the features from the object and the prediction of the object motion to estimate the camera pose. We use various synthetic and real-world test scenarios and the well-known TUM sequences to evaluate the capabilities of our system. The experiments show that we achieve higher pose estimation accuracy and robustness over state-of-the-art monocular vSLAM systems.

Gravitational dyonic amplitude at one-loop and its inconsistency with the classical impulse

The recent proposal [1, 2] of implementing electric-magnetic duality rotation at the level of perturbative scattering amplitudes and its generalisation to gravitational context where usual gravitational mass is rotated to the NUT parameter of the Taub-NUT spacetime opens up an interesting avenue for studying NUT-charged objects as dynamical entities, in contrast to the usual approach where NUT-charged objects are considered as a static background. We extend the tree-order analysis to one-loop order, and find a disagreement between geodesic motion on Taub-NUT background and impulse computation of scattering amplitudes. As a by-product of our analysis, we find a relation between tidal response parameters and resonance excitation parameters in the language of quantum field theory scattering amplitudes.

Reduced conformal geometrodynamics

The global time in geometrodynamics is defined in a covariant under diffeomorphisms form. An arbitrary static background metric is taken in the tangent space. The global intrinsic time is identified with the mean value of the logarithm of the square root of the ratio of the metric determinants. The procedures of the Hamiltonian reduction and deparametrization of dynamical systems are implemented. The reduced Hamiltonian equations of motion of gravitational field in semi-geodesic coordinate system are written.

Multiple Object Detection and Tracking using Kalman Filter in an Indoor and Outdoor Scene

Tracking target through sequences of images is fundamental problems in vision. In this paper we converse the motion based kalman filter procedure to track the multiple objects for indoor and outdoor scenes. This is of utmost importance for high-performance real -time applications. The mentioned approach is appropriate for indoor & outdoors scenes with static background & overcomes the problem of non-moving objectives fading into the background. The tracking in proposed turned into solely based totally on movement with the belief that each one items move in a immediately line with continuous speed. The motion primarily based Kaman filter monitoring for more than one objects works correctly but requires the camera to be stationary

Nonlinear Dirac Neutrino Oscillations

Neutrino oscillations are a possible way to probe beyond Standard Model physics. The propagation of Dirac neutrinos in a massive medium is governed by the Dirac equation modified with an effective Hamiltonian that de- pends on the number density of surrounding matter fields. At the same time, quantum nonlinearities may contribute to neutrino oscillations by further mod- ifying the Dirac equation. A possible nonlinearity is computationally studied using Mathematica at low energies. We find that the presence of a uniform, static background matter distribution may significantly alter the oscillation am- plitude and wavelength; the considered nonlinearity may further reduce both oscillation amplitude and wavelength. In addition, the presence of matter al- lows the effects of the nonlinearity to be more readily observed for the chosen background densities and neutrino energy.