3D Integral Imaging Based Augmented Reality with Deep Learning Implemented by Faster R-CNN

2017 ◽  
pp. 241-247
Author(s):  
Richard Evan Sutanto ◽  
Lenny Pribadi ◽  
Sukho Lee
Keyword(s):  
Deep Learning ◽  
Augmented Reality ◽  
Integral Imaging

scholarly journals Augmented Reality Maintenance Assistant Using YOLOv5

2021 ◽  
Vol 11 (11) ◽  
pp. 4758
Author(s):  
Ana Malta ◽  
Mateus Mendes ◽  
Torres Farinha
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Object Recognition ◽  
Augmented Reality ◽  
Real Time ◽  
Recognition System ◽  
High Accuracy ◽  
Video Streams ◽  
The Neural Network ◽  
Deep Learning Neural Network

Maintenance professionals and other technical staff regularly need to learn to identify new parts in car engines and other equipment. The present work proposes a model of a task assistant based on a deep learning neural network. A YOLOv5 network is used for recognizing some of the constituent parts of an automobile. A dataset of car engine images was created and eight car parts were marked in the images. Then, the neural network was trained to detect each part. The results show that YOLOv5s is able to successfully detect the parts in real time video streams, with high accuracy, thus being useful as an aid to train professionals learning to deal with new equipment using augmented reality. The architecture of an object recognition system using augmented reality glasses is also designed.

scholarly journals Real-time deep learning semantic segmentation during intra-operative surgery for 3D augmented reality assistance

2021 ◽  
Author(s):  
Leonardo Tanzi ◽  
Pietro Piazzolla ◽  
Francesco Porpiglia ◽  
Enrico Vezzetti
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Augmented Reality ◽  
Ad Hoc ◽  
Geodesic Distance ◽  
Semantic Segmentation ◽  
Endoscopic Image ◽  
Operative Surgery ◽  
Endoscopic Videos

Abstract Purpose The current study aimed to propose a Deep Learning (DL) and Augmented Reality (AR) based solution for a in-vivo robot-assisted radical prostatectomy (RARP), to improve the precision of a published work from our group. We implemented a two-steps automatic system to align a 3D virtual ad-hoc model of a patient’s organ with its 2D endoscopic image, to assist surgeons during the procedure. Methods This approach was carried out using a Convolutional Neural Network (CNN) based structure for semantic segmentation and a subsequent elaboration of the obtained output, which produced the needed parameters for attaching the 3D model. We used a dataset obtained from 5 endoscopic videos (A, B, C, D, E), selected and tagged by our team’s specialists. We then evaluated the most performing couple of segmentation architecture and neural network and tested the overlay performances. Results U-Net stood out as the most effecting architectures for segmentation. ResNet and MobileNet obtained similar Intersection over Unit (IoU) results but MobileNet was able to elaborate almost twice operations per seconds. This segmentation technique outperformed the results from the former work, obtaining an average IoU for the catheter of 0.894 (σ = 0.076) compared to 0.339 (σ = 0.195). This modifications lead to an improvement also in the 3D overlay performances, in particular in the Euclidean Distance between the predicted and actual model’s anchor point, from 12.569 (σ= 4.456) to 4.160 (σ = 1.448) and in the Geodesic Distance between the predicted and actual model’s rotations, from 0.266 (σ = 0.131) to 0.169 (σ = 0.073). Conclusion This work is a further step through the adoption of DL and AR in the surgery domain. In future works, we will overcome the limits of this approach and finally improve every step of the surgical procedure.

Deep learning-based smart task assistance in wearable augmented reality

2020 ◽  
Vol 63 ◽  
pp. 101887 ◽  
Author(s):  
Kyeong-Beom Park ◽  
Minseok Kim ◽  
Sung Ho Choi ◽  
Jae Yeol Lee
Keyword(s):  
Deep Learning ◽  
Augmented Reality ◽  
Wearable Augmented Reality ◽  
Task Assistance
Sign in / Sign up

Export Citation Format

Share Document