Adaptive Scheduling Framework for Real-Time Video Encoding on Heterogeneous Systems

2016 ◽  
Vol 26 (3) ◽  
pp. 597-611 ◽  
Author(s):  
Aleksandar Ilic ◽  
Svetislav Momcilovic ◽  
Nuno Roma ◽  
Leonel Sousa
Author(s):  
Gaurav Chaurasia ◽  
Arthur Nieuwoudt ◽  
Alexandru-Eugen Ichim ◽  
Richard Szeliski ◽  
Alexander Sorkine-Hornung

We present an end-to-end system for real-time environment capture, 3D reconstruction, and stereoscopic view synthesis on a mobile VR headset. Our solution allows the user to use the cameras on their VR headset as their eyes to see and interact with the real world while still wearing their headset, a feature often referred to as Passthrough. The central challenge when building such a system is the choice and implementation of algorithms under the strict compute, power, and performance constraints imposed by the target user experience and mobile platform. A key contribution of this paper is a complete description of a corresponding system that performs temporally stable passthrough rendering at 72 Hz with only 200 mW power consumption on a mobile Snapdragon 835 platform. Our algorithmic contributions for enabling this performance include the computation of a coarse 3D scene proxy on the embedded video encoding hardware, followed by a depth densification and filtering step, and finally stereoscopic texturing and spatio-temporal up-sampling. We provide a detailed discussion and evaluation of the challenges we encountered, as well as algorithm and performance trade-offs in terms of compute and resulting passthrough quality.;AB@The described system is available to users as the Passthrough+ feature on Oculus Quest. We believe that by publishing the underlying system and methods, we provide valuable insights to the community on how to design and implement real-time environment sensing and rendering on heavily resource constrained hardware.


Author(s):  
Chuyuan Wang ◽  
Linxuan Zhang ◽  
Chongdang Liu

In order to deal with the dynamic production environment with frequent fluctuation of processing time, robotic cell needs an efficient scheduling strategy which meets the real-time requirements. This paper proposes an adaptive scheduling method based on pattern classification algorithm to guide the online scheduling process. The method obtains the scheduling knowledge of manufacturing system from the production data and establishes an adaptive scheduler, which can adjust the scheduling rules according to the current production status. In the process of establishing scheduler, how to choose essential attributes is the main difficulty. In order to solve the low performance and low efficiency problem of embedded feature selection method, based on the application of Extreme Gradient Boosting model (XGBoost) to obtain the adaptive scheduler, an improved hybrid optimization algorithm which integrates Gini impurity of XGBoost model into Particle Swarm Optimization (PSO) is employed to acquire the optimal subset of features. The results based on simulated robotic cell system show that the proposed PSO-XGBoost algorithm outperforms existing pattern classification algorithms and the newly learned adaptive model can improve the basic dispatching rules. At the same time, it can meet the demand of real-time scheduling.


2011 ◽  
Vol 383-390 ◽  
pp. 5028-5033
Author(s):  
Xue Mei Xu ◽  
Qin Mo ◽  
Lan Ni ◽  
Qiao Yun Guo ◽  
An Li

In the video encoding system, motion estimation plays an important role at the front-end of encoder, which can eliminate inter redundancy efficiently and improve encoding efficiency. However, traditional motion estimation algorithm can’t be used in real-time application like video monitoring due to its computational complexity. In order to improve real-time efficiency, an improved motion estimation algorithm is proposed in this paper. The essential ideas consist of early termination rules, prediction of initial search point, and determination of motion type. Furthermore, our algorithm adopts different search patterns for certain motion activity. Experimental result shows that the improved algorithm reduces the computation time significantly while maintaining the image quality, and satisfies real time requirement in monitoring system.


2014 ◽  
Vol 1003 ◽  
pp. 249-253
Author(s):  
Hao Fang ◽  
Ai Hua Li ◽  
Yan Fei Liu

To solve the difficulty of traditional video monitoring system in system upgrade and expansion, an solution of embedded video monitoring system based on DaVinci technology was put forward in this paper. By building the monitoring platform by DM6437 and DSP/BIOS in the solution, TVP5151 was used for receiving video signal in PAL/NTSC formats, and an JPEG Baseline Profile Encoder was integrated for video encoding, and the 10/100M Ethernet transmission function was realized based on NDK. Finally, the system is tested and the result shows that the system can capture and transmit D1 format signal in 25f/s and met the real-time requirement. At the same time, the system is easy to use and expand with a bright application prospect.


2012 ◽  
Vol 50 (2) ◽  
pp. 280-286 ◽  
Author(s):  
Nikolaus Karpinsky ◽  
Song Zhang
Keyword(s):  

Sign in / Sign up

Export Citation Format

Share Document