scholarly journals Frame and Feature-Context Video Super-Resolution

2019 ◽  
Vol 33 ◽  
pp. 5597-5604
Author(s):  
Bo Yan ◽  
Chuming Lin ◽  
Weimin Tan
Keyword(s):  
Information Flow ◽  
State Of The Art ◽  
Super Resolution ◽  
Sliding Window ◽  
Visual Quality ◽  
Local Network ◽  
Context Information ◽  
High Quality ◽  
Current State ◽  
Inter Frame

For video super-resolution, current state-of-the-art approaches either process multiple low-resolution (LR) frames to produce each output high-resolution (HR) frame separately in a sliding window fashion or recurrently exploit the previously estimated HR frames to super-resolve the following frame. The main weaknesses of these approaches are: 1) separately generating each output frame may obtain high-quality HR estimates while resulting in unsatisfactory flickering artifacts, and 2) combining previously generated HR frames can produce temporally consistent results in the case of short information flow, but it will cause significant jitter and jagged artifacts because the previous super-resolving errors are constantly accumulated to the subsequent frames.In this paper, we propose a fully end-to-end trainable frame and feature-context video super-resolution (FFCVSR) network that consists of two key sub-networks: local network and context network, where the first one explicitly utilizes a sequence of consecutive LR frames to generate local feature and local SR frame, and the other combines the outputs of local network and the previously estimated HR frames and features to super-resolve the subsequent frame. Our approach takes full advantage of the inter-frame information from multiple LR frames and the context information from previously predicted HR frames, producing temporally consistent highquality results while maintaining real-time speed by directly reusing previous features and frames. Extensive evaluations and comparisons demonstrate that our approach produces state-of-the-art results on a standard benchmark dataset, with advantages in terms of accuracy, efficiency, and visual quality over the existing approaches.

scholarly journals Video Face Super-Resolution with Motion-Adaptive Feedback Cell

2020 ◽  
Vol 34 (07) ◽  
pp. 12468-12475
Author(s):  
Jingwei Xin ◽  
Nannan Wang ◽  
Jie Li ◽  
Xinbo Gao ◽  
Zhifeng Li
Keyword(s):  
Motion Estimation ◽  
State Of The Art ◽  
Super Resolution ◽  
Complex Motion ◽  
Deep Convolutional Neural Networks ◽  
Adaptive Feedback ◽  
Current State ◽  
Model Complex ◽  
Frame Motion ◽  
Inter Frame

Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN). Current state-of-the-art CNN methods usually treat the VSR problem as a large number of separate multi-frame super-resolution tasks, at which a batch of low resolution (LR) frames is utilized to generate a single high resolution (HR) frame, and running a slide window to select LR frames over the entire video would obtain a series of HR frames. However, duo to the complex temporal dependency between frames, with the number of LR input frames increase, the performance of the reconstructed HR frames become worse. The reason is in that these methods lack the ability to model complex temporal dependencies and hard to give an accurate motion estimation and compensation for VSR process. Which makes the performance degrade drastically when the motion in frames is complex. In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way. Our approach efficiently utilizes the information of the inter-frame motion, the dependence of the network on motion estimation and compensation method can be avoid. In addition, benefiting from the excellent nature of MAFC, the network can achieve better performance in the case of extremely complex motion scenarios. Extensive evaluations and comparisons validate the strengths of our approach, and the experimental results demonstrated that the proposed framework is outperform the state-of-the-art methods.

scholarly journals Evolutionary Machine Learning for Multi-Objective Class Solutions in Medical Deformable Image Registration

Algorithms ◽  
2019 ◽  
Vol 12 (5) ◽  
pp. 99 ◽  
Author(s):  
Kleopatra Pirpinia ◽  
Peter A. N. Bosman ◽  
Jan-Jakob Sonke ◽  
Marcel van Herk ◽  
Tanja Alderliesten
Keyword(s):  
Machine Learning ◽  
Image Registration ◽  
State Of The Art ◽  
Deformable Image Registration ◽  
Optimization Approach ◽  
High Quality ◽  
Trade Off ◽  
Multi Objective ◽  
Current State ◽  
Image Artefacts

Current state-of-the-art medical deformable image registration (DIR) methods optimize a weighted sum of key objectives of interest. Having a pre-determined weight combination that leads to high-quality results for any instance of a specific DIR problem (i.e., a class solution) would facilitate clinical application of DIR. However, such a combination can vary widely for each instance and is currently often manually determined. A multi-objective optimization approach for DIR removes the need for manual tuning, providing a set of high-quality trade-off solutions. Here, we investigate machine learning for a multi-objective class solution, i.e., not a single weight combination, but a set thereof, that, when used on any instance of a specific DIR problem, approximates such a set of trade-off solutions. To this end, we employed a multi-objective evolutionary algorithm to learn sets of weight combinations for three breast DIR problems of increasing difficulty: 10 prone-prone cases, 4 prone-supine cases with limited deformations and 6 prone-supine cases with larger deformations and image artefacts. Clinically-acceptable results were obtained for the first two problems. Therefore, for DIR problems with limited deformations, a multi-objective class solution can be machine learned and used to compute straightforwardly multiple high-quality DIR outcomes, potentially leading to more efficient use of DIR in clinical practice.

scholarly journals Mosaic Super-resolution via Sequential Feature Pyramid Networks

2019 ◽  
Author(s):  
Mehrdad Shoeiby ◽  
Mohammad Ali Armin ◽  
Sadegh Aliakbarian ◽  
Saeed Anwar ◽  
Lars petersson
Keyword(s):  
State Of The Art ◽  
Super Resolution ◽  
Autonomous Driving ◽  
Single Shot ◽  
Current State ◽  
Wide Range ◽  
Feature Pyramid ◽  
Novel Method ◽  
Convolutional Lstm ◽  
Mosaic Images

<div>Advances in the design of multi-spectral cameras have</div><div>led to great interests in a wide range of applications, from</div><div>astronomy to autonomous driving. However, such cameras</div><div>inherently suffer from a trade-off between the spatial and</div><div>spectral resolution. In this paper, we propose to address</div><div>this limitation by introducing a novel method to carry out</div><div>super-resolution on raw mosaic images, multi-spectral or</div><div>RGB Bayer, captured by modern real-time single-shot mo-</div><div>saic sensors. To this end, we design a deep super-resolution</div><div>architecture that benefits from a sequential feature pyramid</div><div>along the depth of the network. This, in fact, is achieved</div><div>by utilizing a convolutional LSTM (ConvLSTM) to learn the</div><div>inter-dependencies between features at different receptive</div><div>fields. Additionally, by investigating the effect of different</div><div>attention mechanisms in our framework, we show that a</div><div>ConvLSTM inspired module is able to provide superior at-</div><div>tention in our context. Our extensive experiments and anal-</div><div>yses evidence that our approach yields significant super-</div><div>resolution quality, outperforming current state-of-the-art</div><div>mosaic super-resolution methods on both Bayer and multi-</div><div>spectral images. Additionally, to the best of our knowledge,</div><div>our method is the first specialized method to super-resolve</div><div>mosaic images, whether it be multi-spectral or Bayer.</div><div><br></div>

Thermoelectric Properties of C-Axis-Oriented Ca3Co4O9+δ Films Grown on MgO-Buffered Si (100) by PLD Technique

2010 ◽  
Vol 29-32 ◽  
pp. 1913-1918
Author(s):  
Xia Zhang ◽  
Hong Chen ◽  
Qiu Hui Liao ◽  
Xia Li
Keyword(s):  
Thin Films ◽  
State Of The Art ◽  
Pulsed Laser ◽  
Buffer Layers ◽  
Laser Deposition ◽  
X Ray Diffraction ◽  
High Quality ◽  
X Ray ◽  
Current State ◽  
Good Crystallinity

High-quality c-axis-oriented Ca3Co4O9+δ thin films have been grown directly on Si (100) wafers with inserting MgO buffer layers by pulsed-laser deposition (PLD). X-ray diffraction and scan electron microscopy show good crystallinity of the Ca3Co4O9+δ films. The resistivity and Seebeck coefficient of the Ca3Co4O9+δ thin films on Si (100) substrates are 9.8 mΩcm and 189 μV/K at the temperature of 500K, respectively, comparable to the single-crystal samples. This advance demonstrates the possibility of integrating the cobaltate-based high thermoelectric materials with the current state-of-the-art silicon technology for thermoelectricity-on-a-chip applications.

scholarly journals Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

2018 ◽  
Vol 232 ◽  
pp. 01061
Author(s):  
Danhua Li ◽  
Xiaofeng Di ◽  
Xuan Qu ◽  
Yunfei Zhao ◽  
Honggang Kong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Deep Convolutional Neural Network ◽  
High Quality ◽  
Low Level ◽  
Features Fusion ◽  
Current State ◽  
High Level

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

Online Social Exergames for Seniors

2020 ◽  
pp. 1599-1631
Author(s):  
Stathis Th. Konstantinidis ◽  
Ellen Brox ◽  
Per Egil Kummervold ◽  
Josef Hallberg ◽  
Gunn Evertsen ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Physical Activity ◽  
Physical Training ◽  
State Of The Art ◽  
Social Contact ◽  
High Quality ◽  
Current State ◽  
The Future ◽  
The Impact

The population is getting older, and the resources for care will be even more limited in the future than they are now. There is thus an aim for the society that the seniors can manage themselves as long as possible, while at the same time keeping a high quality of life. Physical activity is important to stay fit, and social contact is important for the quality of life. The aim of this chapter is to provide a state-of-the-art of online social exergames for seniors, providing glimpses of senior users' opinions and games limitations. The importance of the motivational techniques is emphasized, as well as the impact that the exergames have to seniors. It contributes to the book objectives focusing on current state and practice in health games for physical training and rehabilitation and the use of gamification, exploring future opportunities and uses of gamification in eHealth and discussing the respective challenges and limitations.

scholarly journals HIGH QUALITY FACADE SEGMENTATION BASED ON STRUCTURED RANDOM FOREST, REGION PROPOSAL NETWORK AND RECTANGULAR FITTING

2018 ◽  
Vol IV-2 ◽  
pp. 223-230 ◽  
Author(s):  
K. Rahmani ◽  
H. Mayer
Keyword(s):  
Neural Network ◽  
Random Forest ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
High Quality ◽  
Current State ◽  
Forest Region ◽  
Building Facades ◽  
Art Methods

In this paper we present a pipeline for high quality semantic segmentation of building facades using Structured Random Forest (SRF), Region Proposal Network (RPN) based on a Convolutional Neural Network (CNN) as well as rectangular fitting optimization. Our main contribution is that we employ features created by the RPN as channels in the SRF.We empirically show that this is very effective especially for doors and windows. Our pipeline is evaluated on two datasets where we outperform current state-of-the-art methods. Additionally, we quantify the contribution of the RPN and the rectangular fitting optimization on the accuracy of the result.

scholarly journals Genetic Programming for Evolving a Front of Interpretable Models for Data Visualization

2020 ◽  
Author(s):  
Andrew Lensen ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Genetic Programming ◽  
Data Visualization ◽  
State Of The Art ◽  
Visual Quality ◽  
Model Complexity ◽  
High Quality ◽  
Depth Analysis ◽  
Multiobjective Approach ◽  
Interpretable Models ◽  
Insight Into

Data visualization is a key tool in data mining for understanding big datasets. Many visualization methods have been proposed, including the well-regarded state-of-the-art method t-distributed stochastic neighbor embedding. However, the most powerful visualization methods have a significant limitation: the manner in which they create their visualization from the original features of the dataset is completely opaque. Many domains require an understanding of the data in terms of the original features; there is hence a need for powerful visualization methods which use understandable models. In this article, we propose a genetic programming (GP) approach called GP-tSNE for evolving interpretable mappings from the dataset to high-quality visualizations. A multiobjective approach is designed that produces a variety of visualizations in a single run which gives different tradeoffs between visual quality and model complexity. Testing against baseline methods on a variety of datasets shows the clear potential of GP-tSNE to allow deeper insight into data than that provided by existing visualization methods. We further highlight the benefits of a multiobjective approach through an in-depth analysis of a candidate front, which shows how multiple models can be analyzed jointly to give increased insight into the dataset.

scholarly journals Genetic Programming for Evolving a Front of Interpretable Models for Data Visualization

2020 ◽  
Author(s):  
Andrew Lensen ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Genetic Programming ◽  
Data Visualization ◽  
State Of The Art ◽  
Visual Quality ◽  
Model Complexity ◽  
High Quality ◽  
Depth Analysis ◽  
Multiobjective Approach ◽  
Interpretable Models ◽  
Insight Into

Data visualization is a key tool in data mining for understanding big datasets. Many visualization methods have been proposed, including the well-regarded state-of-the-art method t-distributed stochastic neighbor embedding. However, the most powerful visualization methods have a significant limitation: the manner in which they create their visualization from the original features of the dataset is completely opaque. Many domains require an understanding of the data in terms of the original features; there is hence a need for powerful visualization methods which use understandable models. In this article, we propose a genetic programming (GP) approach called GP-tSNE for evolving interpretable mappings from the dataset to high-quality visualizations. A multiobjective approach is designed that produces a variety of visualizations in a single run which gives different tradeoffs between visual quality and model complexity. Testing against baseline methods on a variety of datasets shows the clear potential of GP-tSNE to allow deeper insight into data than that provided by existing visualization methods. We further highlight the benefits of a multiobjective approach through an in-depth analysis of a candidate front, which shows how multiple models can be analyzed jointly to give increased insight into the dataset.

scholarly journals Genetic Programming for Evolving a Front of Interpretable Models for Data Visualization

2020 ◽  
Author(s):  
Andrew Lensen ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Genetic Programming ◽  
Data Visualization ◽  
State Of The Art ◽  
Visual Quality ◽  
Model Complexity ◽  
High Quality ◽  
Depth Analysis ◽  
Multiobjective Approach ◽  
Interpretable Models ◽  
Insight Into

Data visualization is a key tool in data mining for understanding big datasets. Many visualization methods have been proposed, including the well-regarded state-of-the-art method t-distributed stochastic neighbor embedding. However, the most powerful visualization methods have a significant limitation: the manner in which they create their visualization from the original features of the dataset is completely opaque. Many domains require an understanding of the data in terms of the original features; there is hence a need for powerful visualization methods which use understandable models. In this article, we propose a genetic programming (GP) approach called GP-tSNE for evolving interpretable mappings from the dataset to high-quality visualizations. A multiobjective approach is designed that produces a variety of visualizations in a single run which gives different tradeoffs between visual quality and model complexity. Testing against baseline methods on a variety of datasets shows the clear potential of GP-tSNE to allow deeper insight into data than that provided by existing visualization methods. We further highlight the benefits of a multiobjective approach through an in-depth analysis of a candidate front, which shows how multiple models can be analyzed jointly to give increased insight into the dataset.

Sign in / Sign up

Export Citation Format

Share Document