scholarly journals R²MRF: Defocus Blur Detection via Recurrently Refining Multi-Scale Residual Features

2020 ◽  
Vol 34 (07) ◽  
pp. 12063-12070
Author(s):  
Chang Tang ◽  
Xinwang Liu ◽  
Xinzhong Zhu ◽  
En Zhu ◽  
Kun Sun ◽  
...  
Keyword(s):  
Semantic Information ◽  
Ground Truth ◽  
Input Image ◽  
Convolutional Network ◽  
Multi Scale ◽  
Blur Detection ◽  
Defocus Blur ◽  
Potential Applications ◽  
Benchmark Datasets ◽  
Background Clutter

Defocus blur detection aims to separate the in-focus and out-of-focus regions in an image. Although attracting more and more attention due to its remarkable potential applications, there are still several challenges for accurate defocus blur detection, such as the interference of background clutter, sensitivity to scales and missing boundary details of defocus blur regions. In order to address these issues, we propose a deep neural network which Recurrently Refines Multi-scale Residual Features (R2MRF) for defocus blur detection. We firstly extract multi-scale deep features by utilizing a fully convolutional network. For each layer, we design a novel recurrent residual refinement branch embedded with multiple residual refinement modules (RRMs) to more accurately detect blur regions from the input image. Considering that the features from bottom layers are able to capture rich low-level features for details preservation while the features from top layers are capable of characterizing the semantic information for locating blur regions, we aggregate the deep features from different layers to learn the residual between the intermediate prediction and the ground truth for each recurrent step in each residual refinement branch. Since the defocus degree is sensitive to image scales, we finally fuse the side output of each branch to obtain the final blur detection map. We evaluate the proposed network on two commonly used defocus blur detection benchmark datasets by comparing it with other 11 state-of-the-art methods. Extensive experimental results with ablation studies demonstrate that R2MRF consistently and significantly outperforms the competitors in terms of both efficiency and accuracy.

scholarly journals R³Net: Recurrent Residual Refinement Network for Saliency Detection

2018 ◽  
Author(s):  
Zijun Deng ◽  
Xiaowei Hu ◽  
Lei Zhu ◽  
Xuemiao Xu ◽  
Jing Qin ◽  
...  
Keyword(s):  
Saliency Detection ◽  
Ground Truth ◽  
Input Image ◽  
Convolutional Network ◽  
Low Level ◽  
Saliency Maps ◽  
Saliency Prediction ◽  
Benchmark Datasets ◽  
Salient Regions ◽  
High Level

Saliency detection is a fundamental yet challenging task in computer vision, aiming at highlighting the most visually distinctive objects in an image. We propose a novel recurrent residual refinement network (R^3Net) equipped with residual refinement blocks (RRBs) to more accurately detect salient regions of an input image. Our RRBs learn the residual between the intermediate saliency prediction and the ground truth by alternatively leveraging the low-level integrated features and the high-level integrated features of a fully convolutional network (FCN). While the low-level integrated features are capable of capturing more saliency details, the high-level integrated features can reduce non-salient regions in the intermediate prediction. Furthermore, the RRBs can obtain complementary saliency information of the intermediate prediction, and add the residual into the intermediate prediction to refine the saliency maps. We evaluate the proposed R^3Net on five widely-used saliency detection benchmarks by comparing it with 16 state-of-the-art saliency detectors. Experimental results show that our network outperforms our competitors in all the benchmark datasets.

SGAN4AbSum: A Semantic-Enhanced Generative Adversarial Network for Abstractive Text Summarization

2021 ◽  
Author(s):  
Tham Vo
Keyword(s):  
Ground Truth ◽  
Text Summarization ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Training Strategy ◽  
Adversarial Network ◽  
Deep Recurrent Neural Network ◽  
Benchmark Datasets ◽  
Latent Representations ◽  
Abstractive Summarization

Abstract In abstractive summarization task, most of proposed models adopt the deep recurrent neural network (RNN)-based encoder-decoder architecture to learn and generate meaningful summary for a given input document. However, most of recent RNN-based models always suffer the challenges related to the involvement of much capturing high-frequency/reparative phrases in long documents during the training process which leads to the outcome of trivial and generic summaries are generated. Moreover, the lack of thorough analysis on the sequential and long-range dependency relationships between words within different contexts while learning the textual representation also make the generated summaries unnatural and incoherent. To deal with these challenges, in this paper we proposed a novel semantic-enhanced generative adversarial network (GAN)-based approach for abstractive text summarization task, called as: SGAN4AbSum. We use an adversarial training strategy for our text summarization model in which train the generator and discriminator to simultaneously handle the summary generation and distinguishing the generated summary with the ground-truth one. The input of generator is the jointed rich-semantic and global structural latent representations of training documents which are achieved by applying a combined BERT and graph convolutional network (GCN) textual embedding mechanism. Extensive experiments in benchmark datasets demonstrate the effectiveness of our proposed SGAN4AbSum which achieve the competitive ROUGE-based scores in comparing with state-of-the-art abstractive text summarization baselines.

scholarly journals Pre-Processing Filter Reflecting Human Visual Perception to Improve Saliency Detection Performance

Electronics ◽  
2021 ◽  
Vol 10 (23) ◽  
pp. 2892
Author(s):  
Kyungjun Lee ◽  
Seungwoo Wee ◽  
Jechang Jeong
Keyword(s):  
Saliency Detection ◽  
Visual Saliency ◽  
Ground Truth ◽  
Bilateral Filter ◽  
Input Image ◽  
Human Visual Perception ◽  
Difference Of Gaussians ◽  
Surrounding Environment ◽  
Benchmark Datasets ◽  
Previous State

Salient object detection is a method of finding an object within an image that a person determines to be important and is expected to focus on. Various features are used to compute the visual saliency, and in general, the color and luminance of the scene are widely used among the spatial features. However, humans perceive the same color and luminance differently depending on the influence of the surrounding environment. As the human visual system (HVS) operates through a very complex mechanism, both neurobiological and psychological aspects must be considered for the accurate detection of salient objects. To reflect this characteristic in the saliency detection process, we have proposed two pre-processing methods to apply to the input image. First, we applied a bilateral filter to improve the segmentation results by smoothing the image so that only the overall context of the image remains while preserving the important borders of the image. Second, although the amount of light is the same, it can be perceived with a difference in the brightness owing to the influence of the surrounding environment. Therefore, we applied oriented difference-of-Gaussians (ODOG) and locally normalized ODOG (LODOG) filters that adjust the input image by predicting the brightness as perceived by humans. Experiments on five public benchmark datasets for which ground truth exists show that our proposed method further improves the performance of previous state-of-the-art methods.

scholarly journals Structural Similarity Loss for Learning to Fuse Multi-Focus Images

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6647
Author(s):  
Xiang Yan ◽  
Syed Zulqarnain Gilani ◽  
Hanlin Qin ◽  
Ajmal Mian
Keyword(s):  
Ground Truth ◽  
Structural Similarity ◽  
Input Image ◽  
Test Time ◽  
Suggested Approach ◽  
Extensive Evaluation ◽  
Fused Image ◽  
Benchmark Datasets ◽  
Focus Image ◽  
Image Pairs

Convolutional neural networks have recently been used for multi-focus image fusion. However, some existing methods have resorted to adding Gaussian blur to focused images, to simulate defocus, thereby generating data (with ground-truth) for supervised learning. Moreover, they classify pixels as ‘focused’ or ‘defocused’, and use the classified results to construct the fusion weight maps. This then necessitates a series of post-processing steps. In this paper, we present an end-to-end learning approach for directly predicting the fully focused output image from multi-focus input image pairs. The suggested approach uses a CNN architecture trained to perform fusion, without the need for ground truth fused images. The CNN exploits the image structural similarity (SSIM) to calculate the loss, a metric that is widely accepted for fused image quality evaluation. What is more, we also use the standard deviation of a local window of the image to automatically estimate the importance of the source images in the final fused image when designing the loss function. Our network can accept images of variable sizes and hence, we are able to utilize real benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluation on benchmark datasets show that our method outperforms, or is comparable with, existing state-of-the-art techniques on both objective and subjective benchmarks.

scholarly journals MDAN-UNet: Multi-Scale and Dual Attention Enhanced Nested U-Net Architecture for Segmentation of Optical Coherence Tomography Images

Algorithms ◽  
2020 ◽  
Vol 13 (3) ◽  
pp. 60 ◽  
Author(s):  
Wen Liu ◽  
Yankui Sun ◽  
Qingge Ji
Keyword(s):  
Optical Coherence Tomography ◽  
State Of The Art ◽  
Imaging Technique ◽  
Optical Coherence ◽  
Convolutional Network ◽  
Multi Scale ◽  
Segmentation Methods ◽  
Benchmark Datasets ◽  
Resolution Imaging ◽  
Layer Segmentation

Optical coherence tomography (OCT) is an optical high-resolution imaging technique for ophthalmic diagnosis. In this paper, we take advantages of multi-scale input, multi-scale side output and dual attention mechanism and present an enhanced nested U-Net architecture (MDAN-UNet), a new powerful fully convolutional network for automatic end-to-end segmentation of OCT images. We have evaluated two versions of MDAN-UNet (MDAN-UNet-16 and MDAN-UNet-32) on two publicly available benchmark datasets which are the Duke Diabetic Macular Edema (DME) dataset and the RETOUCH dataset, in comparison with other state-of-the-art segmentation methods. Our experiment demonstrates that MDAN-UNet-32 achieved the best performance, followed by MDAN-UNet-16 with smaller parameter, for multi-layer segmentation and multi-fluid segmentation respectively.

scholarly journals Mapping and Discriminating Rural Settlements Using Gaofen-2 Images and a Fully Convolutional Network

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6062
Author(s):  
Ziran Ye ◽  
Bo Si ◽  
Yue Lin ◽  
Qiming Zheng ◽  
Ran Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Rural Areas ◽  
Ground Truth ◽  
Representation Learning ◽  
Rural Settlement ◽  
Spatial Characteristic ◽  
Rural Settlements ◽  
Convolutional Network ◽  
Essential Information ◽  
Multi Scale

New ongoing rural construction has resulted in an extensive mixture of new settlements with old ones in the rural areas of China. Understanding the spatial characteristic of these rural settlements is of crucial importance as it provides essential information for land management and decision-making. Despite a great advance in High Spatial Resolution (HSR) satellite images and deep learning techniques, it remains a challenging task for mapping rural settlements accurately because of their irregular morphology and distribution pattern. In this study, we proposed a novel framework to map rural settlements by leveraging the merits of Gaofen-2 HSR images and representation learning of deep learning. We combined a dilated residual convolutional network (Dilated-ResNet) and a multi-scale context subnetwork into an end-to-end architecture in order to learn high resolution feature representations from HSR images and to aggregate and refine the multi-scale features extracted by the aforementioned network. Our experiment in Tongxiang city showed that the proposed framework effectively mapped and discriminated rural settlements with an overall accuracy of 98% and Kappa coefficient of 85%, achieving comparable and improved performance compared to other existing methods. Our results bring tangible benefits to support other convolutional neural network (CNN)-based methods in accurate and timely rural settlement mapping, particularly when up-to-date ground truth is absent. The proposed method does not only offer an effective way to extract rural settlement from HSR images but open a new opportunity to obtain spatial-explicit understanding of rural settlements.

DeFusionNET: Defocus Blur Detection via Recurrently Fusing and Refining Discriminative Multi-scale Deep Features

2020 ◽  
pp. 1-1
Author(s):  
Chang Tang ◽  
Xinwang LIU ◽  
Xiao Zheng ◽  
Wanqing Li ◽  
Jian Xiong ◽  
...  
Keyword(s):  
Multi Scale ◽  
Blur Detection ◽  
Defocus Blur

scholarly journals Deterministic Routing between Layout Abstractions for Multi-Scale Classification of Visually Rich Documents

2019 ◽  
Author(s):  
Ritesh Sarkhel ◽  
Arnab Nandi
Keyword(s):  
Turnaround Time ◽  
Convolutional Network ◽  
Multi Scale ◽  
Deterministic Routing ◽  
Pyramid Model ◽  
Benchmark Datasets ◽  
Work First ◽  
Spatial Pyramid ◽  
Scale Classification

Classifying heterogeneous visually rich documents is a challenging task. Difficulty of this task increases even more if the maximum allowed inference turnaround time is constrained by a threshold. The increased overhead in inference cost, compared to the limited gain in classification capabilities make current multi-scale approaches infeasible in such scenarios. There are two major contributions of this work. First, we propose a spatial pyramid model to extract highly discriminative multi-scale feature descriptors from a visually rich document by leveraging the inherent hierarchy of its layout. Second, we propose a deterministic routing scheme for accelerating end-to-end inference by utilizing the spatial pyramid model. A depth-wise separable multi-column convolutional network is developed to enable our method. We evaluated the proposed approach on four publicly available, benchmark datasets of visually rich documents. Results suggest that our proposed approach demonstrates robust performance compared to the state-of-the-art methods in both classification accuracy and total inference turnaround.

Fast Defocus Blur Detection Network via Global Search and Local Refinements

2021 ◽  
Author(s):  
Xiaoli Sun ◽  
Yang Hai ◽  
Xiujun Zhang ◽  
Chen Xu ◽  
Min Li
Keyword(s):  
Neural Network ◽  
Image Processing ◽  
Mobile Devices ◽  
Mobile Phones ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Multi Scale ◽  
Blur Detection ◽  
Defocus Blur ◽  
Global And Local

Defocus blur detection aims at separating regions on focus from out-of-focus for image processing. With today’s popularity of mobile phones with portrait mode, accurate defocus blur detection has received more and more attention. There are many challenges that we currently confront, such as blur boundaries of defocus regions, interference of messy backgrounds and identification of large flat regions. To address these issues, in this paper, we propose a new deep neural network with both global and local pathways for defocus blur detection. In global pathway, we locate the objects on focus by semantical search. In local pathway, we refine the predicted blur regions via multi-scale supervisions. In addition, the refined results in local pathway are fused with searching results in global pathway by a simple concatenation operation. The structure of our new network is developed in a feasible way and its function appears to be quite effective and efficient, which is suitable for the deployment on mobile devices. It takes about 0.2[Formula: see text]s per image on a regular personal laptop. Experiments on both CUHK dataset and our newly proposed Defocus400 dataset show that our model outperforms existing state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document