scholarly journals A Spherical Convolution Approach for Learning Long Term Viewport Prediction in 360 Immersive Video

2020 ◽  
Vol 34 (01) ◽  
pp. 14003-14040
Author(s):  
Chenglei Wu ◽  
Ruixiao Zhang ◽  
Zhi Wang ◽  
Lifeng Sun
Keyword(s):  
Neural Network ◽  
Semantic Content ◽  
Mixture Density ◽  
Head Mounted Display ◽  
Prediction Scheme ◽  
Level Information ◽  
Plane Projection ◽  
And Performance ◽  
Convolution Approach

Viewport prediction for 360 video forecasts a viewer’s viewport when he/she watches a 360 video with a head-mounted display, which benefits many VR/AR applications such as 360 video streaming and mobile cloud VR. Existing studies based on planar convolutional neural network (CNN) suffer from the image distortion and split caused by the sphere-to-plane projection. In this paper, we start by proposing a spherical convolution based feature extraction network to distill spatial-temporal 360 information. We provide a solution for training such a network without a dedicated 360 image or video classification dataset. We differ with previous methods, which base their predictions on image pixel-level information, and propose a semantic content and preference based viewport prediction scheme. In this paper, we adopt a recurrent neural network (RNN) network to extract a user's personal preference of 360 video content from minutes of embedded viewing histories. We utilize this semantic preference as spatial attention to help network find the "interested'' regions on a future video. We further design a tailored mixture density network (MDN) based viewport prediction scheme, including viewport modeling, tailored loss function, etc, to improve efficiency and accuracy. Our extensive experiments demonstrate the rationality and performance of our method, which outperforms state-of-the-art methods, especially in long-term prediction.

LONG-TERM PREDICTION OF ETHERNET TRAFFIC USING MULTISCALE-BILINEAR RECURRENT NEURAL NETWORK WITH ADAPTIVE LEARNING

2010 ◽  
Vol 19 (01) ◽  
pp. 155-171 ◽  
Author(s):  
DONG-CHUL PARK
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Adaptive Learning ◽  
Traffic Signals ◽  
Prediction Scheme ◽  
Resolution Level ◽  
Ethernet Network ◽  
Conventional Models ◽  
Long Term Prediction

A prediction scheme for Ethernet traffic data using a Multiscale-Bilinear Recurrent Neural Network with Adaptive Learning (M-BRNN-AL) is proposed and presented in this paper. The proposed predictor integrates an M-BRNN and an AL algorithm. In M-BRNN, the wavelet transform is employed to decompose the original traffic signals into several simple traffic signals. A BRNN is then used to predict each decomposed traffic signal. An AL algorithm is also applied in order to improve the learning process at each resolution level in M-BRNN-AL. Experiments and results on a set of Ethernet network traffic predictions show that the proposed scheme converges faster and archives better prediction performance than the other conventional models such as the Multi-layer Perception Type Neural Network, BRNN, and the original M-BRNN in terms of the normalized mean square error.

Digitalization system of ancient architecture decoration art based on neural network and image features

2020 ◽  
pp. 1-12
Author(s):  
Wu Xin ◽  
Qiu Daping
Keyword(s):  
Neural Network ◽  
Construction Industry ◽  
Three Dimensional ◽  
Performance Testing ◽  
Image Features ◽  
Three Dimensional Model ◽  
Performance Effect ◽  
Data Process ◽  
And Performance ◽  
Construction Mode

The inheritance and innovation of ancient architecture decoration art is an important way for the development of the construction industry. The data process of traditional ancient architecture decoration art is relatively backward, which leads to the obvious distortion of the digitalization of ancient architecture decoration art. In order to improve the digital effect of ancient architecture decoration art, based on neural network, this paper combines the image features to construct a neural network-based ancient architecture decoration art data system model, and graphically expresses the static construction mode and dynamic construction process of the architecture group. Based on this, three-dimensional model reconstruction and scene simulation experiments of architecture groups are realized. In order to verify the performance effect of the system proposed in this paper, it is verified through simulation and performance testing, and data visualization is performed through statistical methods. The result of the study shows that the digitalization effect of the ancient architecture decoration art proposed in this paper is good.

Semi-supervised Multi-task Network For Image Aesthetic Assessment

2020 ◽  
Vol 2020 (8) ◽  
pp. 188-1-188-7
Author(s):  
Xiaoyu Xiang ◽  
Yang Cheng ◽  
Jianhang Chen ◽  
Qian Lin ◽  
Jan Allebach
Keyword(s):  
Neural Network ◽  
Semantic Content ◽  
Training Data ◽  
Score Distribution ◽  
Generalization Ability ◽  
Subjective Preference ◽  
Ranking Tasks

Image aesthetic assessment has always been regarded as a challenging task because of the variability of subjective preference. Besides, the assessment of a photo is also related to its style, semantic content, etc. Conventionally, the estimations of aesthetic score and style for an image are treated as separate problems. In this paper, we explore the inter-relatedness between the aesthetics and image style, and design a neural network that can jointly categorize image by styles and give an aesthetic score distribution. To this end, we propose a multi-task network (MTNet) with an aesthetic column serving as a score predictor and a style column serving as a style classifier. The angular-softmax loss is applied in training primary style classifiers to maximize the margin among classes in single-label training data; the semi-supervised method is applied to improve the network’s generalization ability iteratively. We combine the regression loss and classification loss in training aesthetic score. Experiments on the AVA dataset show the superiority of our network in both image attributes classification and aesthetic ranking tasks.

LONG-TERM EVALUATION OF STORM SURGES BY NEURAL NETWORK — CASE STUDY IN ISE BAY —

2018 ◽  
Vol 74 (2) ◽  
pp. I_601-I_606
Author(s):  
Yuji ARAKI ◽  
Tomohiro YASUDA ◽  
Nobuhito MORI
Keyword(s):  
Neural Network ◽  
Storm Surges ◽  
Ise Bay ◽  
Term Evaluation

Design and planning of the balanced scorecard: A case study

2007 ◽  
Vol 26 (3) ◽  
pp. 217-227
Author(s):  
Ming-Hon Hwang ◽  
Hsin Rau
Keyword(s):  
Performance Evaluation ◽  
Balanced Scorecard ◽  
Process Management ◽  
Planning Process ◽  
Company Performance ◽  
Correct Process ◽  
The Balanced Scorecard ◽  
Strategy Planning ◽  
And Performance

In the industrial economy, evaluating company performance based on financial results was good enough. However, in the current globalized and highly competitive environment, maintaining long term competitiveness requires companies to engage in overall strategic planning and performance evaluation. The balanced scorecard is a tool or method for balancing an organization's performance and can react to situations where a company's direction becomes disoriented. This approach assists in strategy planning, process management, and performance evaluation from four perspectives, including financial, customer, internal process, and learning and growth. Good strategy planning provides companies with a correct management direction, correct process management ensures the efficient execution of plans, and correct performance evaluation illustrates the execution results. This study mainly focuses on how a large rubber company in Taiwan utilizes the balanced scorecard in its organization. As the technical perspective is important in the rubber keypad industry, besides the four above perspectives, this company has added the technical perspective. By introducing this company and its progress in implementing the balanced scorecard, this study hopes to provide other companies, especially rubber companies, with a planning direction and reference for the future implementation of the balanced scorecard.

Optimal adaptive inspection and maintenance for redundant systems

2021 ◽  
pp. 1748006X2110201
Author(s):  
Chaochao Lin ◽  
Matteo Pozzi
Keyword(s):  
Engineering Systems ◽  
Discounted Cost ◽  
Markov Decision ◽  
Inspection And Maintenance ◽  
And Performance ◽  
Partially Observable ◽  
Series Systems ◽  
Selection Of ◽  
Redundant Systems

Optimal exploration of engineering systems can be guided by the principle of Value of Information (VoI), which accounts for the topological important of components, their reliability and the management costs. For series systems, in most cases higher inspection priority should be given to unreliable components. For redundant systems such as parallel systems, analysis of one-shot decision problems shows that higher inspection priority should be given to more reliable components. This paper investigates the optimal exploration of redundant systems in long-term decision making with sequential inspection and repairing. When the expected, cumulated, discounted cost is considered, it may become more efficient to give higher inspection priority to less reliable components, in order to preserve system redundancy. To investigate this problem, we develop a Partially Observable Markov Decision Process (POMDP) framework for sequential inspection and maintenance of redundant systems, where the VoI analysis is embedded in the optimal selection of exploratory actions. We investigate the use of alternative approximate POMDP solvers for parallel and more general systems, compare their computation complexities and performance, and show how the inspection priorities depend on the economic discount factor, the degradation rate, the inspection precision, and the repair cost.

scholarly journals Natural Disasters Intensity Analysis and Classification Based on Multispectral Images Using Multi-Layered Deep Convolutional Neural Network

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2648
Author(s):  
Muhammad Aamir ◽  
Tariq Ali ◽  
Muhammad Irfan ◽  
Ahmad Shaf ◽  
Muhammad Zeeshan Azam ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Natural Disasters ◽  
Deep Convolutional Neural Network ◽  
Multispectral Images ◽  
Learning Techniques ◽  
Proposed Model ◽  
Disaster Intensity ◽  
And Performance

Natural disasters not only disturb the human ecological system but also destroy the properties and critical infrastructures of human societies and even lead to permanent change in the ecosystem. Disaster can be caused by naturally occurring events such as earthquakes, cyclones, floods, and wildfires. Many deep learning techniques have been applied by various researchers to detect and classify natural disasters to overcome losses in ecosystems, but detection of natural disasters still faces issues due to the complex and imbalanced structures of images. To tackle this problem, we propose a multilayered deep convolutional neural network. The proposed model works in two blocks: Block-I convolutional neural network (B-I CNN), for detection and occurrence of disasters, and Block-II convolutional neural network (B-II CNN), for classification of natural disaster intensity types with different filters and parameters. The model is tested on 4428 natural images and performance is calculated and expressed as different statistical values: sensitivity (SE), 97.54%; specificity (SP), 98.22%; accuracy rate (AR), 99.92%; precision (PRE), 97.79%; and F1-score (F1), 97.97%. The overall accuracy for the whole model is 99.92%, which is competitive and comparable with state-of-the-art algorithms.

Sign in / Sign up

Export Citation Format

Share Document