scholarly journals Real-Time Eyeblink Detector and Eye State Classifier for Virtual Reality (VR) Headsets (Head-Mounted Displays, HMDs)

Sensors ◽  
2019 ◽  
Vol 19 (5) ◽  
pp. 1121 ◽  
Author(s):  
Nassr Alsaeedi ◽  
Dieter Wloka
Keyword(s):  
Virtual Reality ◽  
Real Time ◽  
Motion Vector ◽  
Region Of Interest ◽  
Video Stream ◽  
Detection Algorithm ◽  
Case Scenario ◽  
Current Frame ◽  
Eyelid Closure ◽  
And Performance

The aim of the study is to develop a real-time eyeblink detection algorithm that can detect eyeblinks during the closing phase for a virtual reality headset (VR headset) and accordingly classify the eye’s current state (open or closed). The proposed method utilises analysis of a motion vector for detecting eyelid closure, and a Haar cascade classifier (HCC) for localising the eye in the captured frame. When the downward motion vector (DMV) is detected, a cross-correlation between the current region of interest (eye in the current frame) and a template image for an open eye is used for verifying eyelid closure. A finite state machine is used for decision making regarding eyeblink occurrence and tracking the eye state in a real-time video stream. The main contributions of this study are, first, the ability of the proposed algorithm to detect eyeblinks during the closing or the pause phases before the occurrence of the reopening phase of the eyeblink. Second, realising the proposed approach by implementing a valid real-time eyeblink detection sensor for a VR headset based on a real case scenario. The sensor is used in the ongoing study that we are conducting. The performance of the proposed method was 83.9% for accuracy, 91.8% for precision and 90.40% for the recall. The processing time for each frame took approximately 11 milliseconds. Additionally, we present a new dataset for non-frontal eye monitoring configuration for eyeblink tracking inside a VR headset. The data annotations are also included, such that the dataset can be used for method validation and performance evaluation in future studies.

scholarly journals Robust Real-Time Detection of Laparoscopic Instruments in Robot Surgery Using Convolutional Neural Networks with Motion Vector Prediction

2019 ◽  
Vol 9 (14) ◽  
pp. 2865 ◽  
Author(s):  
Kyungmin Jo ◽  
Yuna Choi ◽  
Jaesoon Choi ◽  
Jong Woo Chung
Keyword(s):  
Neural Networks ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Detection System ◽  
Motion Vector ◽  
Detection Algorithm ◽  
Surgical Instruments ◽  
The Mean ◽  
Real Time Detection ◽  
Motion Vector Prediction

More than half of post-operative complications can be prevented, and operation performances can be improved based on the feedback gathered from operations or notifications of the risks during operations in real time. However, existing surgical analysis methods are limited, because they involve time-consuming processes and subjective opinions. Therefore, the detection of surgical instruments is necessary for (a) conducting objective analyses, or (b) providing risk notifications associated with a surgical procedure in real time. We propose a new real-time detection algorithm for detection of surgical instruments using convolutional neural networks (CNNs). This algorithm is based on an object detection system YOLO9000 and ensures continuity of detection of the surgical tools in successive imaging frames based on motion vector prediction. This method exhibits a constant performance irrespective of a surgical instrument class, while the mean average precision (mAP) of all the tools is 84.7, with a speed of 38 frames per second (FPS).

Hybrid Design Tools in a Social Virtual Reality Using Networked Oculus Rift: A Feasibility Study in Remote Real-Time Interaction

2016 ◽  
Author(s):  
Robert E. Wendrich ◽  
Kris-Howard Chambers ◽  
Wadee Al-Halabi ◽  
Eric J. Seibel ◽  
Olaf Grevenstuk ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Real Time ◽  
Feasibility Study ◽  
Collaborative Design ◽  
Design Tool ◽  
Engineering Industry ◽  
Wire Mesh ◽  
Hybrid Design ◽  
Case Scenario ◽  
Oculus Rift

Hybrid Design Tool Environments (HDTE) allow designers and engineers to use real tangible tools and physical objects and/or artifacts to make and create real-time virtual representations and presentations on-the-fly. Manipulations of the real tangible objects (e.g., real wire mesh, clay, sketches, etc.) are translated into 2-D and/or 3-D digital CAD software and/or virtual instances. The HDTE is equipped with a Loosely Fitted Design Synthesizer (NXt-LFDS) to support this multi-user interaction and design processing. The current study explores for the first time, the feasibility of using a NXt-LFDS in a networked immersive multi-participant social virtual reality environment (VRE). Using Oculus Rift goggles and PC computers at each location linked via Skype, team members physically located in several countries had the illusion of being co-located in a single virtual world, where they used rawshaping technologies (RST) to design a woman’s purse in 3-D virtual representations. Hence, the possibility to print the purse out on the spot (i.e. anywhere within the networked loop) with a 2-D or 3D printer. Immersive affordable Virtual Reality (VR) technology (and 3-D AM) are in the process of becoming commercially available and widely used by mainstream consumers, a major development that could transform the collaborative design process. The results of the current feasibility study suggests that designing products may become considerably more individualized within collaborative multi-user settings and less inhibited during in the coming ‘Diamond Age’ [1] of VR, collaborative networks and with profound implications for the design (e.g. fashion) and engineering industry. This paper presents the proposed system architecture, a collaborative use-case scenario, and preliminary results of the interaction, coordination, cooperation, and communication with immersive VR.

scholarly journals Efficient FPGA Implementation of Human Detection from Video Sequences

2020 ◽  
Vol 9 (6) ◽  
pp. 242-248
Keyword(s):  
Background Subtraction ◽  
Moving Objects ◽  
Median Filter ◽  
Region Of Interest ◽  
Video Stream ◽  
Human Detection ◽  
Video Sequences ◽  
Computer Vision Applications ◽  
And Performance ◽  
Better Than

Detection of Human is a vital and difficult task in computer vision applications like a police investigation, vehicle tracking, and human following. Human detection in video stream is very important in public security management. In such security related cases detecting an object in the video, sequences are very important to understand the behavior of moving objects which normally used in the background subtraction technique. The input data is preprocessed using a modified median filter and Haar transform. The region of interest is extracted using a background subtraction algorithm with remaining spikes removed using threshold technique. The proposed architecture is coded using standard VHDL language and performance is checked in the Spartan-6 FPGA board. The comparison result shows that the proposed architecture is better than the existing method in both hardware and image quality

Multiuser immersive virtual reality application for real-time remote collaboration to enhance design review process in the social distancing era

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Sunseng Tea ◽  
Kriengsak Panuwatwanich ◽  
Rathavoot Ruthankoon ◽  
Manop Kaewmoracharoen
Keyword(s):  
Virtual Reality ◽  
Real Time ◽  
Virtual Environment ◽  
Review Process ◽  
Traditional Approach ◽  
Remote Collaboration ◽  
Design Review ◽  
Content Type ◽  
And Performance ◽  
Aec Industry

Purpose The purpose of this study is to develop and assess the real-time multiuser virtual reality (VR) application that can be used in the design review process. In particular, the application was aimed to accommodate the design review meetings conducted among participants who are in different locations, which has become commonplace during the COVID-19 outbreak. Design/methodology/approach This paper presents a methodology for the development of a real-time multiuser immersive VR application, to support remote collaboration during the design review process. The developed application can immerse remote project participants into the same virtual environment and provide virtual face-to-face discussions. An experiment was conducted with 44 university students to investigate the applicability and performance of the developed application by comparing it with the traditional approach. Findings Results indicated that the group of students who used the developed immersive VR application outperformed the group that used the traditional approach. This was measured by the percentage of correctly identified design errors during a building inspection experiment. Originality/value The difficulty of bringing remote stakeholders together in a virtual environment has impeded the implementation of VR technology in the architecture, engineering and construction (AEC) industry. Most research has focused on the improvement of a single user’s experience. Most of the previous multiuser VR studies were conducted in other industries while similar research in the AEC industry is limited. The study presented in this paper contributes to the AEC industry by presenting the development of multiuser immersive VR applications for real-time remote collaboration and the empirical evidence to substantiate its potential benefits.

scholarly journals Low-Complexity Saliency Detection Algorithm for Fast Perceptual Video Coding

2013 ◽  
Vol 2013 ◽  
pp. 1-15 ◽  
Author(s):  
Pengyu Liu ◽  
Kebin Jia
Keyword(s):  
Visual Perception ◽  
Video Coding ◽  
Saliency Detection ◽  
Motion Vector ◽  
Region Of Interest ◽  
Low Complexity ◽  
Detection Algorithm ◽  
Characteristics Analysis ◽  
Perceptual Video Coding ◽  
Saliency Region

A low-complexity saliency detection algorithm for perceptual video coding is proposed; low-level encoding information is adopted as the characteristics of visual perception analysis. Firstly, this algorithm employs motion vector (MV) to extract temporal saliency region through fast MV noise filtering and translational MV checking procedure. Secondly, spatial saliency region is detected based on optimal prediction mode distributions in I-frame and P-frame. Then, it combines the spatiotemporal saliency detection results to define the video region of interest (VROI). The simulation results validate that the proposed algorithm can avoid a large amount of computation work in the visual perception characteristics analysis processing compared with other existing algorithms; it also has better performance in saliency detection for videos and can realize fast saliency detection. It can be used as a part of the video standard codec at medium-to-low bit-rates or combined with other algorithms in fast video coding.

Real-time location estimation for indoor navigation using a visual-inertial sensor

Sensor Review ◽  
2020 ◽  
Vol 40 (4) ◽  
pp. 455-464
Author(s):  
Zhe Wang ◽  
Xisheng Li ◽  
Xiaojuan Zhang ◽  
Yanru Bai ◽  
Chengcai Zheng
Keyword(s):  
Real Time ◽  
Inertial Sensors ◽  
Inertial Sensor ◽  
Detection Algorithm ◽  
Indoor Navigation ◽  
Location Estimation ◽  
Evaluation Function ◽  
Current Frame ◽  
Content Type ◽  
Adjacent Frame

Purpose The purpose of this study is to use visual and inertial sensors to achieve real-time location. How to provide an accurate location has become a popular research topic in the field of indoor navigation. Although the complementarity of vision and inertia has been widely applied in indoor navigation, many problems remain, such as inertial sensor deviation calibration, unsynchronized visual and inertial data acquisition and large amount of stored data. Design/methodology/approach First, this study demonstrates that the vanishing point (VP) evaluation function improves the precision of extraction, and the nearest ground corner point (NGCP) of the adjacent frame is estimated by pre-integrating the inertial sensor. The Sequential Similarity Detection Algorithm (SSDA) and Random Sample Consensus (RANSAC) algorithms are adopted to accurately match the adjacent NGCP in the estimated region of interest. Second, the model of visual pose is established by using the parameters of the camera itself, VP and NGCP. The model of inertial pose is established by pre-integrating. Third, location is calculated by fusing the model of vision and inertia. Findings In this paper, a novel method is proposed to fuse visual and inertial sensor to locate indoor environment. The authors describe the building of an embedded hardware platform to the best of their knowledge and compare the result with a mature method and POSAV310. Originality/value This paper proposes a VP evaluation function that is used to extract the most advantages in the intersection of a plurality of parallel lines. To improve the extraction speed of adjacent frame, the authors first proposed fusing the NGCP of the current frame and the calibrated pre-integration to estimate the NGCP of the next frame. The visual pose model was established using extinction VP and NGCP, calibration of inertial sensor. This theory offers the linear processing equation of gyroscope and accelerometer by the model of visual and inertial pose.

GPU-Based Real-Time Occlusion in an Immersive Augmented Reality Environment

2009 ◽  
Vol 9 (2) ◽  
Author(s):  
Yuzhu Lu ◽  
Shana Smith
Keyword(s):  
Virtual Reality ◽  
Augmented Reality ◽  
Real Time ◽  
Graphic Processing Unit ◽  
Prototype System ◽  
Processing Unit ◽  
Virtual Objects ◽  
Remote Cameras ◽  
Real Scene ◽  
And Performance

In this paper, we present a prototype system, which uses CAVE-based virtual reality to enhance immersion in an augmented reality environment. The system integrates virtual objects into a real scene captured by a set of stereo remote cameras. We also present a graphic processing unit (GPU)-based method for computing occlusion between real and virtual objects in real time. The method uses information from the captured stereo images to determine depth of objects in the real scene. Results and performance comparisons show that the GPU-based method is much faster than prior CPU-based methods.

scholarly journals An Efficient Hierarchical Video Coding Scheme Combining Visual Perception Characteristics

2014 ◽  
Vol 2014 ◽  
pp. 1-11
Author(s):  
Pengyu Liu ◽  
Kebin Jia
Keyword(s):  
Visual Perception ◽  
Video Coding ◽  
Priority Setting ◽  
Motion Vector ◽  
Region Of Interest ◽  
Low Complexity ◽  
Detection Algorithm ◽  
Coding Scheme ◽  
Encoding Method ◽  
Perception Analysis

Different visual perception characteristic saliencies are the key to constitute the low-complexity video coding framework. A hierarchical video coding scheme based on human visual systems (HVS) is proposed in this paper. The proposed scheme uses a joint video coding framework consisting of visual perception analysis layer (VPAL) and video coding layer (VCL). In VPAL, effective visual perception characteristics detection algorithm is proposed to achieve visual region of interest (VROI) based on the correlation between coding information (such as motion vector, prediction mode, etc.) and visual attention. Then, the interest priority setting for VROI according to visual perception characteristics is completed. In VCL, the optional encoding method is developed utilizing the visual interested priority setting results from VPAL. As a result, the proposed scheme achieves information reuse and complementary between visual perception analysis and video coding. Experimental results show that the proposed hierarchical video coding scheme effectively alleviates the contradiction between complexity and accuracy. Compared with H.264/AVC (JM17.0), the proposed scheme reduces 80% video coding time approximately and maintains a good video image quality as well. It improves video coding performance significantly.

Apartitioned Approach to Real Time Lane Detection at Different Weather Conditions

2013 ◽  
Vol 671-674 ◽  
pp. 2870-2874
Author(s):  
Wei Wei Zhang ◽  
Xiao Lin Song
Keyword(s):  
Real Time ◽  
Region Of Interest ◽  
Weather Conditions ◽  
Detection Algorithm ◽  
Lane Detection ◽  
Far Field ◽  
Computation Complexity ◽  
Bottom Part ◽  
Target Region ◽  
Partitioned Approach

A partitioned approach to real time lane detection is proposed based on the ARM core microprocessor S3C6410. With the help of the dedicated camera interface in S3C6410, the original image can be converted to RGB format and got window-cut in hardware, leaving the target region of interest (ROI). The pixels in ROI are partitioned into two parts to deal with some hostile weather conditions when lane markings in far field are hard to be distinguished from the homogenous road surface. Hough transform is applied into the top part to utilize lane continuum, and the pixel in bottom part is detected in some fixed search bars to reduce computation complexity. Experiments show that the detection algorithm possesses real time performanceand good robustness at different weather conditions.

Sign in / Sign up

Export Citation Format

Share Document