scholarly journals Ancient Stone Inscription Image Denoising and Inpainting Methods Based on Deep Neural Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Haoming Zhang ◽  
Yue Qi ◽  
Xiaoting Xue ◽  
Yahui Nan
Keyword(s):  
Deep Learning ◽  
Image Denoising ◽  
Deep Neural Networks ◽  
Feature Fusion ◽  
Image Inpainting ◽  
Aesthetic Appreciation ◽  
Learning Methods ◽  
Protection Measures ◽  
Visual Improvement ◽  
History Of

Chinese ancient stone inscriptions contain Chinese traditional calligraphy culture and art information. However, due to the long history of the ancient stone inscriptions, natural erosion, and poor early protection measures, there are a lot of noise in the existing ancient stone inscriptions, which has adverse effects on reading these stone inscriptions and their aesthetic appreciation. At present, digital technologies have played important roles in the protection of cultural relics. For ancient stone inscriptions, we should obtain more perfect digital results without multiple types of noise, while there are few deep learning methods designed for processing stone inscription images. Therefore, we propose a basic framework for image denoising and inpainting of stone inscriptions based on deep learning methods. Firstly, we collect as many images of stone inscriptions as possible and preprocess these images to establish an inscriptions image dataset for image denoising and inpainting. In addition, an improved GAN with a denoiser is used for generating more virtual stone inscription images to expand the dataset. On the basis of these collected and generated images, we designed a stone inscription image denoising model based on multiscale feature fusion and introduced Charbonnier loss function to improve this image denoising model. To further improve the denoising results, an image inpainting model with the coherent semantic attention mechanism is introduced to recover some effective information removed by the former denoising model as much as possible. The experimental results show that our image denoising model achieves better results on PSNR, SSIM, and CEI. The final results have obvious visual improvement compared with the original stone inscription images.

scholarly journals MR-InpaintNet: Toward Deep Multi-Resolution Learning for Progressive Image Inpainting

2021 ◽  
Author(s):  
Huan Zhang ◽  
Zhao Zhang ◽  
Haijun Zhang ◽  
Yi Yang ◽  
Shuicheng Yan ◽  
...  
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Semantic Information ◽  
Feature Fusion ◽  
Image Inpainting ◽  
Feature Learning ◽  
Low Resolution ◽  
Resolution Image ◽  
Texture Information ◽  
Multiple Resolutions

<div>Deep learning based image inpainting methods have improved the performance greatly due to powerful representation ability of deep learning. However, current deep inpainting methods still tend to produce unreasonable structure and blurry texture, implying that image inpainting is still a challenging topic due to the ill-posed property of the task. To address these issues, we propose a novel deep multi-resolution learning-based progressive image inpainting method, termed MR-InpaintNet, which takes the damaged images of different resolutions as input and then fuses the multi-resolution features for repairing the damaged images. The idea is motivated by the fact that images of different resolutions can provide different levels of feature information. Specifically, the low-resolution image provides strong semantic information and the high-resolution image offers detailed texture information. The middle-resolution image can be used to reduce the gap between low-resolution and high-resolution images, which can further refine the inpainting result. To fuse and improve the multi-resolution features, a novel multi-resolution feature learning (MRFL) process is designed, which is consisted of a multi-resolution feature fusion (MRFF) module, an adaptive feature enhancement (AFE) module and a memory enhanced mechanism (MEM) module for information preservation. Then, the refined multi-resolution features contain both rich semantic information and detailed texture information from multiple resolutions. We further handle the refined multiresolution features by the decoder to obtain the recovered image. Extensive experiments on the Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRInpaintNet can effectively recover the textures and structures, and performs favorably against state-of-the-art methods.</div>

scholarly journals Deep-Learning-Based Adaptive Advertising with Augmented Reality

Sensors ◽  
2021 ◽  
Vol 22 (1) ◽  
pp. 63
Author(s):  
Marco A. Moreno-Armendáriz ◽  
Hiram Calvo ◽  
Carlos A. Duchanoy ◽  
Arturo Lara-Cázares ◽  
Enrique Ramos-Diaz ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Augmented Reality ◽  
Ambient Temperature ◽  
Big Five ◽  
Deep Neural Networks ◽  
Learning Methods

In this work we describe a system composed of deep neural networks that analyzes characteristics of customers based on their face (age, gender, and personality), as well as the ambient temperature, with the purpose of generating a personalized signal to potential buyers who pass in front of a beverage establishment; faces are automatically detected, displaying a recommendation using deep learning methods. In order to present suitable digital posters for each person, several technologies were used: Augmented reality, estimation of age, gender, and estimation of personality through the Big Five test applied to an image. The accuracy of each one of these deep neural networks is measured separately to ensure an appropriate precision over 80%. The system has been implemented into a portable solution, and is able to generate a recommendation to one or more people at the same time.

scholarly journals Tri-net for Semi-Supervised Deep Learning

2018 ◽  
Author(s):  
Dong-Dong Chen ◽  
Wei Wang ◽  
Wei Gao ◽  
Zhi-Hua Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Error Rate ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Fine Tuning ◽  
Learning Methods ◽  
Model Initialization

Deep neural networks have witnessed great successes in various real applications, but it requires a large number of labeled data for training. In this paper, we propose tri-net, a deep neural network which is able to use massive unlabeled data to help learning with limited labeled data. We consider model initialization, diversity augmentation and pseudo-label editing simultaneously. In our work, we utilize output smearing to initialize modules, use fine-tuning on labeled data to augment diversity and eliminate unstable pseudo-labels to alleviate the influence of suspicious pseudo-labeled data. Experiments show that our method achieves the best performance in comparison with state-of-the-art semi-supervised deep learning methods. In particular, it achieves 8.30% error rate on CIFAR-10 by using only 4000 labeled examples.

A Very Large-Scale Bioactivity Comparison of Deep Learning and Multiple Machine Learning Algorithms for Drug Discovery

2020 ◽  
Author(s):  
Thomas R. Lane ◽  
Daniel H. Foil ◽  
Eni Minerali ◽  
Fabio Urbina ◽  
Kimberley M. Zorn ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Drug Discovery ◽  
Deep Neural Networks ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Learning Methods ◽  
Machine Learning Methods

<p>Machine learning methods are attracting considerable attention from the pharmaceutical industry for use in drug discovery and applications beyond. In recent studies we have applied multiple machine learning algorithms, modeling metrics and in some cases compared molecular descriptors to build models for individual targets or properties on a relatively small scale. Several research groups have used large numbers of datasets from public databases such as ChEMBL in order to evaluate machine learning methods of interest to them. The largest of these types of studies used on the order of 1400 datasets. We have now extracted well over 5000 datasets from CHEMBL for use with the ECFP6 fingerprint and comparison of our proprietary software Assay Central<sup>TM</sup> with random forest, k-Nearest Neighbors, support vector classification, naïve Bayesian, AdaBoosted decision trees, and deep neural networks (3 levels). Model performance <a>was</a> assessed using an array of five-fold cross-validation metrics including area-under-the-curve, F1 score, Cohen’s kappa and Matthews correlation coefficient. <a>Based on ranked normalized scores for the metrics or datasets all methods appeared comparable while the distance from the top indicated Assay Central<sup>TM</sup> and support vector classification were comparable. </a>Unlike prior studies which have placed considerable emphasis on deep neural networks (deep learning), no advantage was seen in this case where minimal tuning was performed of any of the methods. If anything, Assay Central<sup>TM</sup> may have been at a slight advantage as the activity cutoff for each of the over 5000 datasets representing over 570,000 unique compounds was based on Assay Central<sup>TM</sup>performance, but support vector classification seems to be a strong competitor. We also apply Assay Central<sup>TM</sup> to prospective predictions for PXR and hERG to further validate these models. This work currently appears to be the largest comparison of machine learning algorithms to date. Future studies will likely evaluate additional databases, descriptors and algorithms, as well as further refining methods for evaluating and comparing models. </p><p><b> </b></p>

scholarly journals Deep Learning Techniques for Automatic Detection of Embryonic Neurodevelopmental Disorders

Diagnostics ◽  
2020 ◽  
Vol 10 (1) ◽  
pp. 27 ◽  
Author(s):  
Omneya Attallah ◽  
Maha A. Sharkas ◽  
Heba Gadelkarim
Keyword(s):  
Feature Extraction ◽  
Deep Learning ◽  
Pregnant Women ◽  
Neurodevelopmental Disorders ◽  
Feature Fusion ◽  
Feature Reduction ◽  
Learning Methods ◽  
Learning Techniques ◽  
Healthy Infants ◽  
Deep Feature Extraction

The increasing rates of neurodevelopmental disorders (NDs) are threatening pregnant women, parents, and clinicians caring for healthy infants and children. NDs can initially start through embryonic development due to several reasons. Up to three in 1000 pregnant women have embryos with brain defects; hence, the primitive detection of embryonic neurodevelopmental disorders (ENDs) is necessary. Related work done for embryonic ND classification is very limited and is based on conventional machine learning (ML) methods for feature extraction and classification processes. Feature extraction of these methods is handcrafted and has several drawbacks. Deep learning methods have the ability to deduce an optimum demonstration from the raw images without image enhancement, segmentation, and feature extraction processes, leading to an effective classification process. This article proposes a new framework based on deep learning methods for the detection of END. To the best of our knowledge, this is the first study that uses deep learning techniques for detecting END. The framework consists of four stages which are transfer learning, deep feature extraction, feature reduction, and classification. The framework depends on feature fusion. The results showed that the proposed framework was capable of identifying END from embryonic MRI images of various gestational ages. To verify the efficiency of the proposed framework, the results were compared with related work that used embryonic images. The performance of the proposed framework was competitive. This means that the proposed framework can be successively used for detecting END.

scholarly journals MR-InpaintNet: Toward Deep Multi-Resolution Learning for Progressive Image Inpainting

2021 ◽  
Author(s):  
Huan Zhang ◽  
Zhao Zhang ◽  
Haijun Zhang ◽  
Yi Yang ◽  
Shuicheng Yan ◽  
...  
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Semantic Information ◽  
Feature Fusion ◽  
Image Inpainting ◽  
Feature Learning ◽  
Low Resolution ◽  
Resolution Image ◽  
Texture Information ◽  
Multiple Resolutions

<div>Deep learning based image inpainting methods have improved the performance greatly due to powerful representation ability of deep learning. However, current deep inpainting methods still tend to produce unreasonable structure and blurry texture, implying that image inpainting is still a challenging topic due to the ill-posed property of the task. To address these issues, we propose a novel deep multi-resolution learning-based progressive image inpainting method, termed MR-InpaintNet, which takes the damaged images of different resolutions as input and then fuses the multi-resolution features for repairing the damaged images. The idea is motivated by the fact that images of different resolutions can provide different levels of feature information. Specifically, the low-resolution image provides strong semantic information and the high-resolution image offers detailed texture information. The middle-resolution image can be used to reduce the gap between low-resolution and high-resolution images, which can further refine the inpainting result. To fuse and improve the multi-resolution features, a novel multi-resolution feature learning (MRFL) process is designed, which is consisted of a multi-resolution feature fusion (MRFF) module, an adaptive feature enhancement (AFE) module and a memory enhanced mechanism (MEM) module for information preservation. Then, the refined multi-resolution features contain both rich semantic information and detailed texture information from multiple resolutions. We further handle the refined multiresolution features by the decoder to obtain the recovered image. Extensive experiments on the Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRInpaintNet can effectively recover the textures and structures, and performs favorably against state-of-the-art methods.</div>

scholarly journals Controllably Deep Supervision and Multi-Scale Feature Fusion Network for Cloud and Snow Detection Based on Medium- and High-Resolution Imagery Dataset

2021 ◽  
Vol 13 (23) ◽  
pp. 4805
Author(s):  
Guangbin Zhang ◽  
Xianjun Gao ◽  
Yuanwei Yang ◽  
Mingwei Wang ◽  
Shuhao Ran
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
High Resolution ◽  
Feature Fusion ◽  
Detection Accuracy ◽  
Learning Methods ◽  
Training Requirements ◽  
Scale Feature ◽  
Multi Scale ◽  
Hinge Loss

Clouds and snow in remote sensing imageries cover underlying surface information, reducing image availability. Moreover, they interact with each other, decreasing the cloud and snow detection accuracy. In this study, we propose a convolutional neural network for cloud and snow detection, named the cloud and snow detection network (CSD-Net). It incorporates the multi-scale feature fusion module (MFF) and the controllably deep supervision and feature fusion structure (CDSFF). MFF can capture and aggregate features at various scales, ensuring that the extracted high-level semantic features of clouds and snow are more distinctive. CDSFF can provide a deeply supervised mechanism with hinge loss and combine information from adjacent layers to gain more representative features. It ensures the gradient flow is more oriented and error-less, while retaining more effective information. Additionally, a high-resolution cloud and snow dataset based on WorldView2 (CSWV) was created and released. This dataset meets the training requirements of deep learning methods for clouds and snow in high-resolution remote sensing images. Based on the datasets with varied resolutions, CSD-Net is compared to eight state-of-the-art deep learning methods. The experiment results indicate that CSD-Net has an excellent detection accuracy and efficiency. Specifically, the mean intersection over the union (MIoU) of CSD-Net is the highest in the corresponding experiment. Furthermore, the number of parameters in our proposed network is just 7.61 million, which is the lowest of the tested methods. It only has 88.06 GFLOPs of floating point operations, which is less than the U-Net, DeepLabV3+, PSPNet, SegNet-Modified, MSCFF, and GeoInfoNet. Meanwhile, CSWV has a higher annotation quality since the same method can obtain a greater accuracy on it.

scholarly journals Research on Time Series Anomaly Detection: Based on Deep Learning Methods

2021 ◽  
Vol 2132 (1) ◽  
pp. 012012
Author(s):  
Jiaqi Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Detection Task ◽  
Detection Methods ◽  
Learning Methods

Abstract Time series anomaly detection has always been an important research direction. The early time series anomaly detection methods are mainly statistical methods and machine learning methods. With the powerful functions of deep neural network being continuously mined by researchers, the effect of deep neural network in anomaly detection task has been significantly better than the traditional methods. In view of the continuous development and application of deep neural networks such as transformer and graph neural network (GNN) in time series anomaly detection in recent years, the body of research lacks a comparative evaluation of deep learning methods in recent years. This paper studies various deep neural networks suitable for time series, which are divided into three categories according to anomaly detection methods. The evaluation is conducted on public datasets. By analyzing the evaluation criteria, this paper discusses the performance of each model, as well as the problems and development direction in the field of time series anomaly detection in the future. This study found that in the time series anomaly detection task, transformer is suitable for dealing with long-time series prediction, and studying the graph structure of time series may be the best way to deal with time series anomaly detection in the future

scholarly journals Image Denoising Using Nonlocal Regularized Deep Image Prior

Symmetry ◽  
2021 ◽  
Vol 13 (11) ◽  
pp. 2114
Author(s):  
Zhonghua Xie ◽  
Lingjun Liu ◽  
Zhongliang Luo ◽  
Jianfeng Huang
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Image Denoising ◽  
Deep Neural Networks ◽  
Optimization Problems ◽  
Training Data ◽  
Superior Performance ◽  
Convolutional Network ◽  
Image Prior ◽  
Deep Image

Deep neural networks have shown great potential in various low-level vision tasks, leading to several state-of-the-art image denoising techniques. Training a deep neural network in a supervised fashion usually requires the collection of a great number of examples and the consumption of a significant amount of time. However, the collection of training samples is very difficult for some application scenarios, such as the full-sampled data of magnetic resonance imaging and the data of satellite remote sensing imaging. In this paper, we overcome the problem of a lack of training data by using an unsupervised deep-learning-based method. Specifically, we propose a deep-learning-based method based on the deep image prior (DIP) method, which only requires a noisy image as training data, without any clean data. It infers the natural images with random inputs and the corrupted observation with the help of performing correction via a convolutional network. We improve the original DIP method as follows: Firstly, the original optimization objective function is modified by adding nonlocal regularizers, consisting of a spatial filter and a frequency domain filter, to promote the gradient sparsity of the solution. Secondly, we solve the optimization problem with the alternating direction method of multipliers (ADMM) framework, resulting in two separate optimization problems, including a symmetric U-Net training step and a plug-and-play proximal denoising step. As such, the proposed method exploits the powerful denoising ability of both deep neural networks and nonlocal regularizations. Experiments validate the effectiveness of leveraging a combination of DIP and nonlocal regularizers, and demonstrate the superior performance of the proposed method both quantitatively and visually compared with the original DIP method.

Sign in / Sign up

Export Citation Format

Share Document