Sensing-by-Overlaying: A Practical Implementation of a Multiplayer Mixed-Reality Gaming System by Integrating a Dense Point Cloud and a Real-Time Camera

2016 ◽  
Author(s):  
Shuichi Kurabayashi ◽  
Hidetoshi Ishiyama ◽  
Masaru Kanai
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Mixed Reality ◽  
Practical Implementation ◽  
Dense Point

Characterizing real-time dense point cloud capture and streaming on mobile devices

2021 ◽  
Author(s):  
Jinhan Hu ◽  
Aashiq Shaikh ◽  
Alireza Bahremand ◽  
Robert LiKamWa
Keyword(s):  
Real Time ◽  
Mobile Devices ◽  
Point Cloud ◽  
Dense Point

scholarly journals PointLIE: Locally Invertible Embedding for Point Cloud Sampling and Recovery

2021 ◽  
Author(s):  
Weibing Zhao ◽  
Xu Yan ◽  
Jiantao Gao ◽  
Ruimao Zhang ◽  
Jiayan Zhang ◽  
...  
Keyword(s):  
Real Time ◽  
Latent Variable ◽  
Point Cloud ◽  
Fundamental Problem ◽  
Raw Data ◽  
The Arts ◽  
Dense Point ◽  
Directional Learning ◽  
Local Topology ◽  
Time Point

Point Cloud Sampling and Recovery (PCSR) is critical for massive real-time point cloud collection and processing since raw data usually requires large storage and computation. This paper addresses a fundamental problem in PCSR: How to downsample the dense point cloud with arbitrary scales while preserving the local topology of discarded points in a case-agnostic manner (i.e., without additional storage for point relationships)? We propose a novel Locally Invertible Embedding (PointLIE) framework to unify the point cloud sampling and upsampling into one single framework through bi-directional learning. Specifically, PointLIE decouples the local geometric relationships between discarded points from the sampled points by progressively encoding the neighboring offsets to a latent variable. Once the latent variable is forced to obey a pre-defined distribution in the forward sampling path, the recovery can be achieved effectively through inverse operations. Taking the recover-pleasing sampled points and a latent embedding randomly drawn from the specified distribution as inputs, PointLIE can theoretically guarantee the fidelity of reconstruction and outperform state-of-the-arts quantitatively and qualitatively.

Development of environment design support mixed reality system capable of environment estimation using deep learning

Impact ◽  
2020 ◽  
Vol 2020 (2) ◽  
pp. 9-11
Author(s):  
Tomohiro Fukuda
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Computer Games ◽  
Construction Projects ◽  
Mixed Reality ◽  
Semantic Segmentation ◽  
Environment Design ◽  
Aviation Training ◽  
Architecture And Design ◽  
World Environment

Mixed reality (MR) is rapidly becoming a vital tool, not just in gaming, but also in education, medicine, construction and environmental management. The term refers to systems in which computer-generated content is superimposed over objects in a real-world environment across one or more sensory modalities. Although most of us have heard of the use of MR in computer games, it also has applications in military and aviation training, as well as tourism, healthcare and more. In addition, it has the potential for use in architecture and design, where buildings can be superimposed in existing locations to render 3D generations of plans. However, one major challenge that remains in MR development is the issue of real-time occlusion. This refers to hiding 3D virtual objects behind real articles. Dr Tomohiro Fukuda, who is based at the Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering at Osaka University in Japan, is an expert in this field. Researchers, led by Dr Tomohiro Fukuda, are tackling the issue of occlusion in MR. They are currently developing a MR system that realises real-time occlusion by harnessing deep learning to achieve an outdoor landscape design simulation using a semantic segmentation technique. This methodology can be used to automatically estimate the visual environment prior to and after construction projects.

scholarly journals Real-time Track Obstacle Detection from 3D LIDAR Point Cloud

2021 ◽  
Vol 1910 (1) ◽  
pp. 012002
Author(s):  
Chao He ◽  
Jiayuan Gong ◽  
Yahui Yang ◽  
Dong Bi ◽  
Jianpin Lan ◽  
...  
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Obstacle Detection ◽  
3D Lidar

scholarly journals Indoor Path-Planning Algorithm for UAV-Based Contact Inspection

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 642
Author(s):  
Luis Miguel González de Santos ◽  
Ernesto Frías Nores ◽  
Joaquín Martínez Sánchez ◽  
Higinio González Jorge
Keyword(s):  
Path Planning ◽  
Real Time ◽  
Execution Time ◽  
Point Cloud ◽  
Indoor Environments ◽  
Calculation Algorithm ◽  
Time Restrictions ◽  
Planning Algorithm ◽  
Inspection Tasks ◽  
Path Planning Algorithm

Nowadays, unmanned aerial vehicles (UAVs) are extensively used for multiple purposes, such as infrastructure inspections or surveillance. This paper presents a real-time path planning algorithm in indoor environments designed to perform contact inspection tasks using UAVs. The only input used by this algorithm is the point cloud of the building where the UAV is going to navigate. The algorithm is divided into two main parts. The first one is the pre-processing algorithm that processes the point cloud, segmenting it into rooms and discretizing each room. The second part is the path planning algorithm that has to be executed in real time. In this way, all the computational load is in the first step, which is pre-processed, making the path calculation algorithm faster. The method has been tested in different buildings, measuring the execution time for different paths calculations. As can be seen in the results section, the developed algorithm is able to calculate a new path in 8–9 milliseconds. The developed algorithm fulfils the execution time restrictions, and it has proven to be reliable for route calculation.

scholarly journals A Deep Learning Approach to Downscale Geostationary Satellite Imagery for Decision Support in High Impact Wildfires

Forests ◽  
2021 ◽  
Vol 12 (3) ◽  
pp. 294
Author(s):  
Nicholas F. McCarthy ◽  
Ali Tohidi ◽  
Yawar Aziz ◽  
Matt Dennie ◽  
Mario Miguel Valero ◽  
...  
Keyword(s):  
Deep Learning ◽  
Decision Support ◽  
Real Time ◽  
Forest Fires ◽  
Wildland Fire ◽  
Fire Spread ◽  
Low Earth Orbit ◽  
Practical Implementation ◽  
Spatiotemporal Resolution ◽  
Active Fire

Scarcity in wildland fire progression data as well as considerable uncertainties in forecasts demand improved methods to monitor fire spread in real time. However, there exists at present no scalable solution to acquire consistent information about active forest fires that is both spatially and temporally explicit. To overcome this limitation, we propose a statistical downscaling scheme based on deep learning that leverages multi-source Remote Sensing (RS) data. Our system relies on a U-Net Convolutional Neural Network (CNN) to downscale Geostationary (GEO) satellite multispectral imagery and continuously monitor active fire progression with a spatial resolution similar to Low Earth Orbit (LEO) sensors. In order to achieve this, the model trains on LEO RS products, land use information, vegetation properties, and terrain data. The practical implementation has been optimized to use cloud compute clusters, software containers and multi-step parallel pipelines in order to facilitate real time operational deployment. The performance of the model was validated in five wildfires selected from among the most destructive that occurred in California in 2017 and 2018. These results demonstrate the effectiveness of the proposed methodology in monitoring fire progression with high spatiotemporal resolution, which can be instrumental for decision support during the first hours of wildfires that may quickly become large and dangerous. Additionally, the proposed methodology can be leveraged to collect detailed quantitative data about real-scale wildfire behaviour, thus supporting the development and validation of fire spread models.

Real-time, High-resolution Depth Upsampling on Embedded Accelerators

2021 ◽  
Vol 20 (3) ◽  
pp. 1-22
Author(s):  
David Langerman ◽  
Alan George
Keyword(s):  
High Resolution ◽  
Low Power ◽  
Real Time ◽  
Mixed Reality ◽  
Graphics Processing Unit ◽  
Processing Unit ◽  
Reconfigurable Logic ◽  
Depth Sensors ◽  
Time Requirements ◽  
Graphics Processing

High-resolution, low-latency apps in computer vision are ubiquitous in today’s world of mixed-reality devices. These innovations provide a platform that can leverage the improving technology of depth sensors and embedded accelerators to enable higher-resolution, lower-latency processing for 3D scenes using depth-upsampling algorithms. This research demonstrates that filter-based upsampling algorithms are feasible for mixed-reality apps using low-power hardware accelerators. The authors parallelized and evaluated a depth-upsampling algorithm on two different devices: a reconfigurable-logic FPGA embedded within a low-power SoC; and a fixed-logic embedded graphics processing unit. We demonstrate that both accelerators can meet the real-time requirements of 11 ms latency for mixed-reality apps. 1

scholarly journals Software Architecture of a Fog Computing Node for Industrial Internet of Things

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3715
Author(s):  
Ioan Ungurean ◽  
Nicoleta Cristina Gaitan
Keyword(s):  
Internet Of Things ◽  
Real Time ◽  
Development Process ◽  
Fog Computing ◽  
Low Latency ◽  
Practical Implementation ◽  
Industrial Internet Of Things ◽  
Industrial Internet ◽  
Starting Point ◽  
Hard Real Time

In the design and development process of fog computing solutions for the Industrial Internet of Things (IIoT), we need to take into consideration the characteristics of the industrial environment that must be met. These include low latency, predictability, response time, and operating with hard real-time compiling. A starting point may be the reference fog architecture released by the OpenFog Consortium (now part of the Industrial Internet Consortium), but it has a high abstraction level and does not define how to integrate the fieldbuses and devices into the fog system. Therefore, the biggest challenges in the design and implementation of fog solutions for IIoT is the diversity of fieldbuses and devices used in the industrial field and ensuring compliance with all constraints in terms of real-time compiling, low latency, and predictability. Thus, this paper proposes a solution for a fog node that addresses these issues and integrates industrial fieldbuses. For practical implementation, there are specialized systems on chips (SoCs) that provides support for real-time communication with the fieldbuses through specialized coprocessors and peripherals. In this paper, we describe the implementation of the fog node on a system based on Xilinx Zynq UltraScale+ MPSoC ZU3EG A484 SoC.

Sign in / Sign up

Export Citation Format

Share Document