scholarly journals Efficient Live and On-Demand Tiled HEVC 360 VR Video Streaming

2019 ◽  
Vol 13 (03) ◽  
pp. 367-391
Author(s):  
Mattis Jeppsson ◽  
Håvard N. Espeland ◽  
Tomas Kupka ◽  
Ragnar Langseth ◽  
Andreas Petlund ◽  
...  
Keyword(s):  
Real World ◽  
Video Quality ◽  
Field Of View ◽  
Live Streaming ◽  
User Experiences ◽  
Head Mounted Display ◽  
On Demand ◽  
Set Top Box ◽  
Trade Offs ◽  
Bandwidth Savings

360∘ panorama video displayed through Virtual Reality (VR) glasses or large screens offers immersive user experiences, but as such technology becomes commonplace, the need for efficient streaming methods of such high-bitrate videos arises. In this respect, the attention that 360∘ panorama video has received lately is huge. Many methods have already been proposed, and in this paper, we shed more light on the different trade-offs in order to save bandwidth while preserving the video quality in the user’s field-of-view (FoV). Using 360∘ VR content delivered to a Gear VR head-mounted display with a Samsung Galaxy S7 and to a Huawei Q22 set-top-box, we have tested various tiling schemes analyzing the tile layout, the tiling and encoding overheads, mechanisms for faster quality switching beyond the DASH segment boundaries and quality selection configurations. In this paper, we present an efficient end-to-end design and real-world implementation of such a 360∘ streaming system. Furthermore, in addition to researching an on-demand system, we also go beyond the existing on-demand solutions and present a live streaming system which strikes a trade-off between bandwidth usage and the video quality in the user’s FoV. We have created an architecture that combines RTP and DASH, and our system multiplexes a single HEVC hardware decoder to provide faster quality switching than at the traditional GOP boundaries. We demonstrate the performance and illustrate the trade-offs through real-world experiments where we can report comparable bandwidth savings to existing on-demand approaches, but with faster quality switches when the FoV changes.

scholarly journals Colour Calibration of a Head Mounted Display for Colour Vision Research Using Virtual Reality

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Raquel Gil Rodríguez ◽  
Florian Bayer ◽  
Matteo Toscani ◽  
Dar’ya Guarnera ◽  
Giuseppe Claudio Guarnera ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Real World ◽  
Vision Research ◽  
Colour Vision ◽  
Software Implementation ◽  
Field Of View ◽  
Large Field ◽  
Colour Constancy ◽  
Head Mounted Display ◽  
Colour Appearance

AbstractVirtual reality (VR) technology offers vision researchers the opportunity to conduct immersive studies in simulated real-world scenes. However, an accurate colour calibration of the VR head mounted display (HMD), both in terms of luminance and chromaticity, is required to precisely control the presented stimuli. Such a calibration presents significant new challenges, for example, due to the large field of view of the HMD, or the software implementation used for scene rendering, which might alter the colour appearance of objects. Here, we propose a framework for calibrating an HMD using an imaging colorimeter, the I29 (Radiant Vision Systems, Redmond, WA, USA). We examine two scenarios, both with and without using a rendering software for visualisation. In addition, we present a colour constancy experiment design for VR through a gaming engine software, Unreal Engine 4. The colours of the objects of study are chosen according to the previously defined calibration. Results show a high-colour constancy performance among participants, in agreement with recent studies performed on real-world scenarios. Our studies show that our methodology allows us to control and measure the colours presented in the HMD, effectively enabling the use of VR technology for colour vision research.

The Shared View Paradigm in Asymmetric Virtual Reality Setups

i-com ◽  
2020 ◽  
Vol 19 (2) ◽  
pp. 87-101
Author(s):  
Robin Horst ◽  
Fabio Klonowski ◽  
Linda Rau ◽  
Ralf Dörner
Keyword(s):  
Virtual Reality ◽  
Virtual Environments ◽  
Real World ◽  
User Study ◽  
Field Of View ◽  
Virtual Scene ◽  
Head Mounted Display ◽  
Safety Aspects ◽  
Common Technique ◽  
Visualization Techniques

AbstractAsymmetric Virtual Reality (VR) applications are a substantial subclass of multi-user VR that offers not all participants the same interaction possibilities with the virtual scene. While one user might be immersed using a VR head-mounted display (HMD), another user might experience the VR through a common desktop PC. In an educational scenario, for example, learners can use immersive VR technology to inform themselves at different exhibits within a virtual scene. Educators can use a desktop PC setup for following and guiding learners through virtual exhibits and still being able to pay attention to safety aspects in the real world (e. g., avoid learners bumping against a wall). In such scenarios, educators must ensure that learners have explored the entire scene and have been informed about all virtual exhibits in it. According visualization techniques can support educators and facilitate conducting such VR-enhanced lessons. One common technique is to render the view of the learners on the 2D screen available to the educators. We refer to this solution as the shared view paradigm. However, this straightforward visualization involves challenges. For example, educators have no control over the scene and the collaboration of the learning scenario can be tedious. In this paper, we differentiate between two classes of visualizations that can help educators in asymmetric VR setups. First, we investigate five techniques that visualize the view direction or field of view of users (view visualizations) within virtual environments. Second, we propose three techniques that can support educators to understand what parts of the scene learners already have explored (exploration visualization). In a user study, we show that our participants preferred a volume-based rendering and a view-in-view overlay solution for view visualizations. Furthermore, we show that our participants tended to use combinations of different view visualizations.

Augmented Reality Procedure Assistance System for Operator Training and Simulation

2020 ◽  
Vol 64 (1) ◽  
pp. 1176-1180
Author(s):  
Eugene Hayden ◽  
Kang Wang ◽  
Chengjie Wu ◽  
Shi Cao
Keyword(s):  
Augmented Reality ◽  
Situation Awareness ◽  
Mental Workload ◽  
Field Of View ◽  
Task Completion ◽  
Operator Training ◽  
Head Mounted Display ◽  
Subject Design ◽  
Procedural Tasks ◽  
Speed Accuracy

This study explores the design, implementation, and evaluation of an Augmented Reality (AR) prototype that assists novice operators in performing procedural tasks in simulator environments. The prototype uses an optical see-through head-mounted display (OST HMD) in conjunction with a simulator display to supplement sequences of interactive visual and attention-guiding cues to the operator’s field of view. We used a 2x2 within-subject design to test two conditions: with/without AR-cues, each condition had a voice assistant and two procedural tasks (preflight and landing). An experiment examined twenty-six novice operators. The results demonstrated that augmented reality had benefits in terms of improved situation awareness and accuracy, however, it yielded longer task completion time by creating a speed-accuracy trade-off effect in favour of accuracy. No significant effect on mental workload is found. The results suggest that augmented reality systems have the potential to be used by a wider audience of operators.

scholarly journals Extrinsic Camera Calibration with Line-Laser Projection

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1091
Author(s):  
Izaak Van Crombrugge ◽  
Rudi Penne ◽  
Steve Vanlanduit
Keyword(s):  
Camera Calibration ◽  
Real World ◽  
Large Scale ◽  
State Of The Art ◽  
Bundle Adjustment ◽  
Field Of View ◽  
Extrinsic Calibration ◽  
Practical Procedure ◽  
Partial Overlap

Knowledge of precise camera poses is vital for multi-camera setups. Camera intrinsics can be obtained for each camera separately in lab conditions. For fixed multi-camera setups, the extrinsic calibration can only be done in situ. Usually, some markers are used, like checkerboards, requiring some level of overlap between cameras. In this work, we propose a method for cases with little or no overlap. Laser lines are projected on a plane (e.g., floor or wall) using a laser line projector. The pose of the plane and cameras is then optimized using bundle adjustment to match the lines seen by the cameras. To find the extrinsic calibration, only a partial overlap between the laser lines and the field of view of the cameras is needed. Real-world experiments were conducted both with and without overlapping fields of view, resulting in rotation errors below 0.5°. We show that the accuracy is comparable to other state-of-the-art methods while offering a more practical procedure. The method can also be used in large-scale applications and can be fully automated.

scholarly journals The dimensions of streaming: toward a typology of an evolving concept

2020 ◽  
Vol 42 (7-8) ◽  
pp. 1210-1225 ◽  
Author(s):  
Hendrik Storstein Spilker ◽  
Terje Colbjørnsen
Keyword(s):  
Comparative Research ◽  
Live Streaming ◽  
General Audience ◽  
Industrial History ◽  
On Demand ◽  
History Of ◽  
Media Research ◽  
Key Dimensions

Based on research on the development of streaming solutions across media forms and industries, this article traces the dynamics and dimensions of the notion of streaming. It theorizes streaming as an evolving concept, and argues against strict, set and limited definitions such as those suggested by Lotz and Herbert et al. A short substantive and industrial history of streaming is provided, recognizing its many manifestations and variations. Five key dimensions are identified, and trends and traits within each of them discussed: (1) professional versus user-generated streaming, (2) legal versus piracy streaming, (3) on-demand versus live streaming, (4) streaming on dedicated versus multi-feature platforms, and (5) niche versus general-audience streaming. The article concludes by pointing out how streaming is a concept that metaphorically unites media research across industries, practices, and media forms, encouraging more comparative research.

Amelioration of user experiences and glycaemic outcomes with an Advanced Hybrid Closed Loop System in a real-world clinical setting

2021 ◽  
pp. 108986
Author(s):  
Pilar Isabel Beato-Víbora ◽  
Fabiola Gallego-Gamero ◽  
Ana Ambrojo-López ◽  
Estela Gil-Poch ◽  
Irene Martín-Romo ◽  
...  
Keyword(s):  
Real World ◽  
Clinical Setting ◽  
Closed Loop ◽  
Loop System ◽  
Closed Loop System ◽  
User Experiences

scholarly journals Improving Perceived Quality of Live Adaptative Video Streaming

Entropy ◽  
2021 ◽  
Vol 23 (8) ◽  
pp. 948
Author(s):  
Carlos Eduardo Maffini Santos ◽  
Carlos Alexandre Gouvea da Silva ◽  
Carlos Marcelo Pedroso
Keyword(s):  
Video Streaming ◽  
Short Term Memory ◽  
Transmission Rate ◽  
Video Quality ◽  
Video On Demand ◽  
Live Streaming ◽  
Network Congestion ◽  
Congested Networks ◽  
Over The Top

Quality of service (QoS) requirements for live streaming are most required for video-on-demand (VoD), where they are more sensitive to variations in delay, jitter, and packet loss. Dynamic Adaptive Streaming over HTTP (DASH) is the most popular technology for live streaming and VoD, where it has been massively deployed on the Internet. DASH is an over-the-top application using unmanaged networks to distribute content with the best possible quality. Widely, it uses large reception buffers in order to keep a seamless playback for VoD applications. However, the use of large buffers in live streaming services is not allowed because of the induced delay. Hence, network congestion caused by insufficient queues could decrease the user-perceived video quality. Active Queue Management (AQM) arises as an alternative to control the congestion in a router’s queue, pressing the TCP traffic sources to reduce their transmission rate when it detects incipient congestion. As a consequence, the DASH client tends to decrease the quality of the streamed video. In this article, we evaluate the performance of recent AQM strategies for real-time adaptive video streaming and propose a new AQM algorithm using Long Short-Term Memory (LSTM) neural networks to improve the user-perceived video quality. The LSTM forecast the trend of queue delay to allow earlier packet discard in order to avoid the network congestion. The results show that the proposed method outperforms the competing AQM algorithms, mainly in scenarios where there are congested networks.

scholarly journals Movement Around Real and Virtual Cluttered Environments

2005 ◽  
Vol 14 (5) ◽  
pp. 580-596 ◽  
Author(s):  
Simon Lessels ◽  
Roy A. Ruddle
Keyword(s):  
Real World ◽  
Search Task ◽  
Scale Space ◽  
Visual Scene ◽  
Field Of View ◽  
Small Scale ◽  
High Fidelity ◽  
Cluttered Environments ◽  
The Real ◽  
Desktop Virtual Environment

Two experiments investigated participants' ability to search for targets in a cluttered small-scale space. The first experiment was conducted in the real world with two field of view conditions (full vs. restricted), and participants found the task trivial to perform in both. The second experiment used the same search task but was conducted in a desktop virtual environment (VE), and investigated two movement interfaces and two visual scene conditions. Participants restricted to forward only movement performed the search task quicker and more efficiently (visiting fewer targets) than those who used an interface that allowed more flexible movement (forward, backward, left, right, and diagonal). Also, participants using a high fidelity visual scene performed the task significantly quicker and more efficiently than those who used a low fidelity scene. The performance differences among all the conditions decreased with practice, but the performance of the best VE group approached that of the real-world participants. These results indicate the importance of using high fidelity scenes in VEs, and suggest that the use of a simple control system is sufficient for maintaining one's spatial orientation during searching.

Beyond the Known: Detecting Novel Feasible Domains Over an Unbounded Design Space

2017 ◽  
Vol 139 (11) ◽  
Author(s):  
Wei Chen ◽  
Mark Fuge
Keyword(s):  
Real World ◽  
Adaptive Sampling ◽  
Design Space ◽  
Fundamental Problem ◽  
Domain Boundary ◽  
Sampling Technique ◽  
Standard Test ◽  
Worst Case ◽  
Trade Offs ◽  
Design Variables

To solve a design problem, sometimes it is necessary to identify the feasible design space. For design spaces with implicit constraints, sampling methods are usually used. These methods typically bound the design space; that is, limit the range of design variables. But bounds that are too small will fail to cover all possible designs, while bounds that are too large will waste sampling budget. This paper tries to solve the problem of efficiently discovering (possibly disconnected) feasible domains in an unbounded design space. We propose a data-driven adaptive sampling technique—ε-margin sampling, which learns the domain boundary of feasible designs and also expands our knowledge on the design space as available budget increases. This technique is data-efficient, in that it makes principled probabilistic trade-offs between refining existing domain boundaries versus expanding the design space. We demonstrate that this method can better identify feasible domains on standard test functions compared to both random and active sampling (via uncertainty sampling). However, a fundamental problem when applying adaptive sampling to real world designs is that designs often have high dimensionality and thus require (in the worst case) exponentially more samples per dimension. We show how coupling design manifolds with ε-margin sampling allows us to actively expand high-dimensional design spaces without incurring this exponential penalty. We demonstrate this on real-world examples of glassware and bottle design, where our method discovers designs that have different appearance and functionality from its initial design set.

Sign in / Sign up

Export Citation Format

Share Document