scholarly journals Spatiotemporal Super-Resolution with Cross-Task Consistency and Its Semi-supervised Extension

2020 ◽  
Author(s):  
Han-Yi Lin ◽  
Pi-Cheng Hsiu ◽  
Tei-Wei Kuo ◽  
Yen-Yu Lin
Keyword(s):  
High Resolution ◽  
State Of The Art ◽  
Source Code ◽  
Super Resolution ◽  
The Other ◽  
Training Data ◽  
Frame Rate ◽  
Stream Network ◽  
Network Training ◽  
Temporal Dimensions

Spatiotemporal super-resolution (SR) aims to upscale both the spatial and temporal dimensions of input videos, and produces videos with higher frame resolutions and rates. It involves two essential sub-tasks: spatial SR and temporal SR. We design a two-stream network for spatiotemporal SR in this work. One stream contains a temporal SR module followed by a spatial SR module, while the other stream has the same two modules in the reverse order. Based on the interchangeability of performing the two sub-tasks, the two network streams are supposed to produce consistent spatiotemporal SR results. Thus, we present a cross-stream consistency to enforce the similarity between the outputs of the two streams. In this way, the training of the two streams is correlated, which allows the two SR modules to share their supervisory signals and improve each other. In addition, the proposed cross-stream consistency does not consume labeled training data and can guide network training in an unsupervised manner. We leverage this property to carry out semi-supervised spatiotemporal SR. It turns out that our method makes the most of training data, and can derive an effective model with few high-resolution and high-frame-rate videos, achieving the state-of-the-art performance. The source code of this work is available at https://hankweb.github.io/STSRwithCrossTask/.

scholarly journals Super-resolution segmentation network for reconstruction of packed neurites

2020 ◽  
Author(s):  
Zhou Hang ◽  
Quan Tingwei ◽  
Huang Qing ◽  
Liu Tian ◽  
Cao Tingting ◽  
...  
Keyword(s):  
High Resolution ◽  
Quantitative Data ◽  
Brain Research ◽  
State Of The Art ◽  
Super Resolution ◽  
Neuronal Morphology ◽  
The Other ◽  
Neuron Reconstruction ◽  
Art Methods ◽  
Accurate Reconstruction

AbstractNeuron reconstruction can provide the quantitative data required for measuring the neuronal morphology and is crucial in the field of brain research. However, the difficulty in reconstructing packed neuritis, wherein massive labor is required for accurate reconstruction in most cases, has not been resolved. In this work, we provide a fundamental pathway for solving this challenge by proposing the use of the super-resolution segmentation network (SRSNet) that builds the mapping of the neurites in the original neuronal images and their segmentation in a higher-resolution space. SRSNet focuses on enlarging the distances between the boundaries of the packed neurites producing the high-resolution segmentation images. Thus, in the construction of the training datasets, only the traced skeletons of neurites are required, which vastly increase the usability of SRSNet. From the results of the experiments conducted in this work, it has been observed that SRSNet achieves accurate reconstruction of packed neurites where the other state-of-the-art methods fail.

Super-Resolution Based on Generative Adversarial Network for HRTEM Images

2021 ◽  
pp. 2154027
Author(s):  
Fuqi Mao ◽  
Xiaohan Guan ◽  
Ruoyu Wang ◽  
Wen Yue
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Information Structure ◽  
Rapid Development ◽  
Super Resolution ◽  
Training Data ◽  
Learning Technology ◽  
Hrtem Image ◽  
Generative Adversarial Network ◽  
Dual Regression

As an important tool to study the microstructure and properties of materials, High Resolution Transmission Electron Microscope (HRTEM) images can obtain the lattice fringe image (reflecting the crystal plane spacing information), structure image and individual atom image (which reflects the configuration of atoms or atomic groups in crystal structure). Despite the rapid development of HTTEM devices, HRTEM images still have limited achievable resolution for human visual system. With the rapid development of deep learning technology in recent years, researchers are actively exploring the Super-resolution (SR) model based on deep learning, and the model has reached the current best level in various SR benchmarks. Using SR to reconstruct high-resolution HRTEM image is helpful to the material science research. However, there is one core issue that has not been resolved: most of these super-resolution methods require the training data to exist in pairs. In actual scenarios, especially for HRTEM images, there are no corresponding HR images. To reconstruct high quality HRTEM image, a novel Super-Resolution architecture for HRTEM images is proposed in this paper. Borrowing the idea from Dual Regression Networks (DRN), we introduce an additional dual regression structure to ESRGAN, by training the model with unpaired HRTEM images and paired nature images. Results of extensive benchmark experiments demonstrate that the proposed method achieves better performance than the most resent SISR methods with both quantitative and visual results.

scholarly journals Vehicle images reconstruction using SRCNN for improving the recognition accuracy of vehicle license plate number

2020 ◽  
Vol 8 (4) ◽  
pp. 304-310
Author(s):  
Windra Swastika ◽  
Ekky Rino Fajar Sakti ◽  
Mochamad Subianto
Keyword(s):  
High Resolution ◽  
Character Recognition ◽  
Recognition Accuracy ◽  
Super Resolution ◽  
Training Data ◽  
License Plate ◽  
Plate Number ◽  
Average Accuracy ◽  
Number Recognition ◽  
High Resolution Images

Low-resolution images can be reconstructed into high-resolution images using the Super-resolution Convolution Neural Network (SRCNN) algorithm. This study aims to improve the vehicle license plate number's recognition accuracy by generating a high-resolution vehicle image using the SRCNN. The recognition is carried out by two types of character recognition methods: Tesseract OCR and SPNet. The training data for SRCNN uses the DIV2K dataset consisting of 900 images, while the training data for character recognition uses the Chars74 dataset. The high-resolution images constructed using SRCNN can increase the average accuracy of vehicle license plate number recognition by 16.9 % using Tesseract and 13.8 % with SPNet.

scholarly journals Discriminative non-negative representation based classifier for image recognition

2021 ◽  
Vol 15 ◽  
pp. 174830262110449
Author(s):  
Kai-Jun Hu ◽  
He-Feng Yin ◽  
Jun Sun
Keyword(s):  
Pattern Classification ◽  
State Of The Art ◽  
Source Code ◽  
Classification Performance ◽  
Training Data ◽  
Practical Applications ◽  
The Past ◽  
Benchmark Datasets ◽  
Classification Tasks ◽  
Negative Representation

During the past decade, representation based classification method has received considerable attention in the community of pattern recognition. The recently proposed non-negative representation based classifier achieved superb recognition results in diverse pattern classification tasks. Unfortunately, discriminative information of training data is not fully exploited in non-negative representation based classifier, which undermines its classification performance in practical applications. To address this problem, we introduce a decorrelation regularizer into the formulation of non-negative representation based classifier and propose a discriminative non-negative representation based classifier for pattern classification. The decorrelation regularizer is able to reduce the correlation of representation results of different classes, thus promoting the competition among them. Experimental results on benchmark datasets validate the efficacy of the proposed discriminative non-negative representation based classifier, and it can outperform some state-of-the-art deep learning based methods. The source code of our proposed discriminative non-negative representation based classifier is accessible at https://github.com/yinhefeng/DNRC .

scholarly journals Low Resolution Information Also Matters: Learning Multi-Resolution Representations for Person Re-Identification

2021 ◽  
Author(s):  
Guoqing Zhang ◽  
Yuhao Chen ◽  
Weisi Lin ◽  
Arun Chandran ◽  
Xuan Jing
Keyword(s):  
Feature Extraction ◽  
High Resolution ◽  
Feature Fusion ◽  
State Of The Art ◽  
Super Resolution ◽  
Input Image ◽  
Low Resolution ◽  
Joint Learning ◽  
Novel Method ◽  
Valid Information

As a prevailing task in video surveillance and forensics field, person re-identification (re-ID) aims to match person images captured from non-overlapped cameras. In unconstrained scenarios, person images often suffer from the resolution mismatch problem, i.e., Cross-Resolution Person Re-ID. To overcome this problem, most existing methods restore low resolution (LR) images to high resolution (HR) by super-resolution (SR). However, they only focus on the HR feature extraction and ignore the valid information from original LR images. In this work, we explore the influence of resolutions on feature extraction and develop a novel method for cross-resolution person re-ID called Multi-Resolution Representations Joint Learning (MRJL). Our method consists of a Resolution Reconstruction Network (RRN) and a Dual Feature Fusion Network (DFFN). The RRN uses an input image to construct a HR version and a LR version with an encoder and two decoders, while the DFFN adopts a dual-branch structure to generate person representations from multi-resolution images. Comprehensive experiments on five benchmarks verify the superiority of the proposed MRJL over the relevent state-of-the-art methods.

scholarly journals An Example-Based Super-Resolution Algorithm for Selfie Images

2016 ◽  
Vol 2016 ◽  
pp. 1-12
Author(s):  
Jino Hans William ◽  
N. Venkateswaran ◽  
Srinath Narayanan ◽  
Sandeep Ramachandran
Keyword(s):  
High Resolution ◽  
State Of The Art ◽  
Super Resolution ◽  
Image Pair ◽  
Image Patch ◽  
Low Resolution ◽  
Pixel Resolution ◽  
Efficiency And Effectiveness ◽  
Level Information ◽  
Resolution Algorithm

A selfie is typically a self-portrait captured using the front camera of a smartphone. Most state-of-the-art smartphones are equipped with a high-resolution (HR) rear camera and a low-resolution (LR) front camera. As selfies are captured by front camera with limited pixel resolution, the fine details in it are explicitly missed. This paper aims to improve the resolution of selfies by exploiting the fine details in HR images captured by rear camera using an example-based super-resolution (SR) algorithm. HR images captured by rear camera carry significant fine details and are used as an exemplar to train an optimal matrix-value regression (MVR) operator. The MVR operator serves as an image-pair priori which learns the correspondence between the LR-HR patch-pairs and is effectively used to super-resolve LR selfie images. The proposed MVR algorithm avoids vectorization of image patch-pairs and preserves image-level information during both learning and recovering process. The proposed algorithm is evaluated for its efficiency and effectiveness both qualitatively and quantitatively with other state-of-the-art SR algorithms. The results validate that the proposed algorithm is efficient as it requires less than 3 seconds to super-resolve LR selfie and is effective as it preserves sharp details without introducing any counterfeit fine details.

A Review of Super-Resolution Restoration from Image Sequences

2011 ◽  
Vol 204-210 ◽  
pp. 1336-1341
Author(s):  
Zhi Gang Xu ◽  
Xiu Qin Su
Keyword(s):  
High Resolution ◽  
State Of The Art ◽  
Super Resolution ◽  
Image Sequences ◽  
The State ◽  
Color Images ◽  
Low Resolution ◽  
Future Directions ◽  
High Resolution Images ◽  
Multidisciplinary Studies

Super-resolution (SR) restoration produces one or a set of high resolution images from low-resolution observations. In particular, SR restoration involves many multidisciplinary studies. A review on recent SR restoration approaches was given in this paper. First, we introduced the characteristics and framework of SR restoration. The state of the art in SR restoration was surveyed by taxonomy. Then we summarized and analyzed the existing algorithms of registration and reconstruction. A comparison of performing differences between these methods would only be valid given. After that we discussed the SR problems of color images and compressed videos. At last, we concluded with some thoughts about future directions.

scholarly journals Recognizing and Imitating Programmer Style: Adversaries in Program Authorship Attribution

2018 ◽  
Vol 2018 (1) ◽  
pp. 127-144 ◽  
Author(s):  
Lucy Simko ◽  
Luke Zettlemoyer ◽  
Tadayoshi Kohno
Keyword(s):  
Qualitative Analysis ◽  
State Of The Art ◽  
Source Code ◽  
The Other ◽  
Authorship Attribution

Abstract Source code attribution classifiers have recently become powerful. We consider the possibility that an adversary could craft code with the intention of causing a misclassification, i.e., creating a forgery of another author’s programming style in order to hide the forger’s own identity or blame the other author. We find that it is possible for a non-expert adversary to defeat such a system. In order to inform the design of adversarially resistant source code attribution classifiers, we conduct two studies with C/C++ programmers to explore the potential tactics and capabilities both of such adversaries and, conversely, of human analysts doing source code authorship attribution. Through the quantitative and qualitative analysis of these studies, we (1) evaluate a state-of-the-art machine classifier against forgeries, (2) evaluate programmers as human analysts/forgery detectors, and (3) compile a set of modifications made to create forgeries. Based on our analyses, we then suggest features that future source code attribution systems might incorporate in order to be adversarially resistant.

scholarly journals DEFSI: Deep Learning Based Epidemic Forecasting with Synthetic Information

2019 ◽  
Vol 33 ◽  
pp. 9607-9612 ◽  
Author(s):  
Lijing Wang ◽  
Jiangzhuo Chen ◽  
Madhav Marathe
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
State Of The Art ◽  
Synthetic Data ◽  
State Level ◽  
The Other ◽  
County Level ◽  
Short Term ◽  
Neural Network Structure ◽  
Epidemic Forecasting

Influenza-like illness (ILI) is among the most common diseases worldwide. Producing timely, well-informed, and reliable forecasts for ILI is crucial for preparedness and optimal interventions. In this work, we focus on short-term but highresolution forecasting and propose DEFSI (Deep Learning Based Epidemic Forecasting with Synthetic Information), an epidemic forecasting framework that integrates the strengths of artificial neural networks and causal methods. In DEFSI, we build a two-branch neural network structure to take both within-season observations and between-season observations as features. The model is trained on geographically highresolution synthetic data. It enables detailed forecasting when high-resolution surveillance data is not available. Furthermore, the model is provided with better generalizability and physical consistency. Our method achieves comparable/better performance than state-of-the-art methods for short-term ILI forecasting at the state level. For high-resolution forecasting at the county level, DEFSI significantly outperforms the other methods.

scholarly journals Leveraging Human Attention in Novel Object Captioning

2021 ◽  
Author(s):  
Xianyu Chen ◽  
Ming Jiang ◽  
Qi Zhao
Keyword(s):  
State Of The Art ◽  
Source Code ◽  
Training Data ◽  
Training Method ◽  
Image Captioning ◽  
Novel Object ◽  
Text Corpora ◽  
Human Attention ◽  
Gating Mechanism ◽  
Novel Objects

Image captioning models depend on training with paired image-text corpora, which poses various challenges in describing images containing novel objects absent from the training data. While previous novel object captioning methods rely on external image taggers or object detectors to describe novel objects, we present the Attention-based Novel Object Captioner (ANOC) that complements novel object captioners with human attention features that characterize generally important information independent of tasks. It introduces a gating mechanism that adaptively incorporates human attention with self-learned machine attention, with a Constrained Self-Critical Sequence Training method to address the exposure bias while maintaining constraints of novel object descriptions. Extensive experiments conducted on the nocaps and Held-Out COCO datasets demonstrate that our method considerably outperforms the state-of-the-art novel object captioners. Our source code is available at https://github.com/chenxy99/ANOC.

Sign in / Sign up

Export Citation Format

Share Document