scholarly journals Diversity-Generated Image Inpainting with Style Extraction

2019 ◽  
Author(s):  
Weiwei Cai ◽  
Zhanguo Wei
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Image Inpainting ◽  
Ground Truth ◽  
Generative Model ◽  
Input Noise ◽  
Latent Vector ◽  
Proposed Model ◽  
Ground Truth Image

The latest methods based on deep learning have achieved amazing results regarding the complex work of inpainting large missing areas in an image. This type of method generally attempts to generate one single "optimal" inpainting result, ignoring many other plausible results. However, considering the uncertainty of the inpainting task, one sole result can hardly be regarded as a desired regeneration of the missing area. In view of this weakness, which is related to the design of the previous algorithms, we propose a novel deep generative model equipped with a brand new style extractor which can extract the style noise (a latent vector) from the ground truth image. Once obtained, the extracted style noise and the ground truth image are both input into the generator. We also craft a consistency loss that guides the generated image to approximate the ground truth. Meanwhile, the same extractor captures the style noise from the generated image, which is forced to approach the input noise according to the consistency loss. After iterations, our generator is able to learn the styles corresponding to multiple sets of noise. The proposed model can generate a (sufficiently large) number of inpainting results consistent with the context semantics of the image. Moreover, we check the effectiveness of our model on three databases, i.e., CelebA, Agricultural Disease, and MauFlex. Compared to state-of-the-art inpainting methods, this model is able to offer desirable inpainting results with both a better quality and higher diversity. The code and model will be made available on https://github.com/vivitsai/SEGAN.

Equivariant Adversarial Network for Image-to-image Translation

2021 ◽  
Vol 17 (2s) ◽  
pp. 1-14
Author(s):  
Masoumeh Zareapoor ◽  
Jie Yang
Keyword(s):  
State Of The Art ◽  
Generative Models ◽  
Generative Model ◽  
Target Domain ◽  
Adversarial Network ◽  
Proposed Model ◽  
Image Translation ◽  
Great Performance ◽  
Representative Model ◽  
The Ideal

Image-to-Image translation aims to learn an image from a source domain to a target domain. However, there are three main challenges, such as lack of paired datasets, multimodality, and diversity, that are associated with these problems and need to be dealt with. Convolutional neural networks (CNNs), despite of having great performance in many computer vision tasks, they fail to detect the hierarchy of spatial relationships between different parts of an object and thus do not form the ideal representative model we look for. This article presents a new variation of generative models that aims to remedy this problem. We use a trainable transformer, which explicitly allows the spatial manipulation of data within training. This differentiable module can be augmented into the convolutional layers in the generative model, and it allows to freely alter the generated distributions for image-to-image translation. To reap the benefits of proposed module into generative model, our architecture incorporates a new loss function to facilitate an effective end-to-end generative learning for image-to-image translation. The proposed model is evaluated through comprehensive experiments on image synthesizing and image-to-image translation, along with comparisons with several state-of-the-art algorithms.

scholarly journals Semantic segmentation of gonio-photographs via adaptive ROI localisation and uncertainty estimation

2021 ◽  
Vol 6 (1) ◽  
pp. e000898
Author(s):  
Andrea Peroni ◽  
Anna Paviotti ◽  
Mauro Campigotto ◽  
Luis Abegão Pinto ◽  
Carlo Alberto Cutolo ◽  
...  
Keyword(s):  
Region Of Interest ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Uncertainty Estimation ◽  
Depth Of Field ◽  
Clinical Settings ◽  
Proposed Model ◽  
Validation Experiment ◽  
Segmentation Accuracy ◽  
Ground Truth Image

ObjectiveTo develop and test a deep learning (DL) model for semantic segmentation of anatomical layers of the anterior chamber angle (ACA) in digital gonio-photographs.Methods and analysisWe used a pilot dataset of 274 ACA sector images, annotated by expert ophthalmologists to delineate five anatomical layers: iris root, ciliary body band, scleral spur, trabecular meshwork and cornea. Narrow depth-of-field and peripheral vignetting prevented clinicians from annotating part of each image with sufficient confidence, introducing a degree of subjectivity and features correlation in the ground truth. To overcome these limitations, we present a DL model, designed and trained to perform two tasks simultaneously: (1) maximise the segmentation accuracy within the annotated region of each frame and (2) identify a region of interest (ROI) based on local image informativeness. Moreover, our calibrated model provides results interpretability returning pixel-wise classification uncertainty through Monte Carlo dropout.ResultsThe model was trained and validated in a 5-fold cross-validation experiment on ~90% of available data, achieving ~91% average segmentation accuracy within the annotated part of each ground truth image of the hold-out test set. An appropriate ROI was successfully identified in all test frames. The uncertainty estimation module located correctly inaccuracies and errors of segmentation outputs.ConclusionThe proposed model improves the only previously published work on gonio-photographs segmentation and may be a valid support for the automatic processing of these images to evaluate local tissue morphology. Uncertainty estimation is expected to facilitate acceptance of this system in clinical settings.

Background Noise Removal Technique using Deep Learning Segmentation Network without Segmentation Map

2021 ◽  
Vol 263 (2) ◽  
pp. 4441-4445
Author(s):  
Hyunsuk Huh ◽  
Seungchul Lee
Keyword(s):  
Deep Learning ◽  
Background Noise ◽  
Ground Truth ◽  
Noise Removal ◽  
Industrial Manufacturing ◽  
Proposed Model ◽  
Audio Data ◽  
Passive Noise ◽  
Removal Technique ◽  
Noise Canceling

Audio data acquired at industrial manufacturing sites often include unexpected background noise. Since the performance of data-driven models can be worse by background noise. Therefore, it is important to get rid of unwanted background noise. There are two main techniques for noise canceling in a traditional manner. One is Active Noise Canceling (ANC), which generates an inverted phase of the sound that we want to remove. The other is Passive Noise Canceling (PNC), which physically blocks the noise. However, these methods require large device size and expensive cost. Thus, we propose a deep learning-based noise canceling method. This technique was developed using audio imaging technique and deep learning segmentation network. However, the proposed model only needs the information on whether the audio contains noise or not. In other words, unlike the general segmentation technique, a pixel-wise ground truth segmentation map is not required for this method. We demonstrate to evaluate the separation using pump sound of MIMII dataset, which is open-source dataset.

scholarly journals Exploitation of deep learning in the automatic detection of cracks on paved roads

GEOMATICA ◽  
2019 ◽  
Vol 73 (2) ◽  
pp. 29-44
Author(s):  
Won Mo Jung ◽  
Faizaan Naveed ◽  
Baoxin Hu ◽  
Jianguo Wang ◽  
Ningyuan Li
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Ground Truth ◽  
Learning Networks ◽  
Test Image ◽  
Convolutional Network ◽  
Image Patches ◽  
Severity Levels ◽  
First Time ◽  
Different Levels

With the advance of deep learning networks, their applications in the assessment of pavement conditions are gaining more attention. A convolutional neural network (CNN) is the most commonly used network in image classification. In terms of pavement assessment, most existing CNNs are designed to only distinguish between cracks and non-cracks. Few networks classify cracks in different levels of severity. Information on the severity of pavement cracks is critical for pavement repair services. In this study, the state-of-the-art CNN used in the detection of pavement cracks was improved to localize the cracks and identify their distress levels based on three categories (low, medium, and high). In addition, a fully convolutional network (FCN) was, for the first time, utilized in the detection of pavement cracks. These designed architectures were validated using the data acquired on four highways in Ontario, Canada, and compared with the ground truth that was provided by the Ministry of Transportation of Ontario (MTO). The results showed that with the improved CNN, the prediction precision on a series of test image patches were 72.9%, 73.9%, and 73.1% for cracks with the severity levels of low, medium, and high, respectively. The precision for the FCN was tested on whole pavement images, resulting in 62.8%, 63.3%, and 66.4%, respectively, for cracks with the severity levels of low, medium, and high. It is worth mentioning that the ground truth contained some uncertainties, which partially contributed to the relatively low precision.

scholarly journals Learning to Combine Local and Global Image Information for Contactless Palmprint Recognition

Sensors ◽  
2021 ◽  
Vol 22 (1) ◽  
pp. 73
Author(s):  
Marjan Stoimchev ◽  
Marija Ivanovska ◽  
Vitomir Štruc
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Input Image ◽  
Palmprint Recognition ◽  
Learning Approaches ◽  
Elastic Deformations ◽  
Feature Representations ◽  
Palmar Surface ◽  
Proposed Model ◽  
Visual Artifacts

In the past few years, there has been a leap from traditional palmprint recognition methodologies, which use handcrafted features, to deep-learning approaches that are able to automatically learn feature representations from the input data. However, the information that is extracted from such deep-learning models typically corresponds to the global image appearance, where only the most discriminative cues from the input image are considered. This characteristic is especially problematic when data is acquired in unconstrained settings, as in the case of contactless palmprint recognition systems, where visual artifacts caused by elastic deformations of the palmar surface are typically present in spatially local parts of the captured images. In this study we address the problem of elastic deformations by introducing a new approach to contactless palmprint recognition based on a novel CNN model, designed as a two-path architecture, where one path processes the input in a holistic manner, while the second path extracts local information from smaller image patches sampled from the input image. As elastic deformations can be assumed to most significantly affect the global appearance, while having a lesser impact on spatially local image areas, the local processing path addresses the issues related to elastic deformations thereby supplementing the information from the global processing path. The model is trained with a learning objective that combines the Additive Angular Margin (ArcFace) Loss and the well-known center loss. By using the proposed model design, the discriminative power of the learned image representation is significantly enhanced compared to standard holistic models, which, as we show in the experimental section, leads to state-of-the-art performance for contactless palmprint recognition. Our approach is tested on two publicly available contactless palmprint datasets—namely, IITD and CASIA—and is demonstrated to perform favorably against state-of-the-art methods from the literature. The source code for the proposed model is made publicly available.

scholarly journals A Tweet Sentiment Classification Approach Using a Hybrid Stacked Ensemble Technique

Information ◽  
2021 ◽  
Vol 12 (9) ◽  
pp. 374
Author(s):  
Babacar Gaye ◽  
Dezheng Zhang ◽  
Aziguli Wulamu
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Short Term Memory ◽  
State Of The Art ◽  
Accuracy Score ◽  
Learning Models ◽  
Proposed Model

With the extensive availability of social media platforms, Twitter has become a significant tool for the acquisition of peoples’ views, opinions, attitudes, and emotions towards certain entities. Within this frame of reference, sentiment analysis of tweets has become one of the most fascinating research areas in the field of natural language processing. A variety of techniques have been devised for sentiment analysis, but there is still room for improvement where the accuracy and efficacy of the system are concerned. This study proposes a novel approach that exploits the advantages of the lexical dictionary, machine learning, and deep learning classifiers. We classified the tweets based on the sentiments extracted by TextBlob using a stacked ensemble of three long short-term memory (LSTM) as base classifiers and logistic regression (LR) as a meta classifier. The proposed model proved to be effective and time-saving since it does not require feature extraction, as LSTM extracts features without any human intervention. We also compared our proposed approach with conventional machine learning models such as logistic regression, AdaBoost, and random forest. We also included state-of-the-art deep learning models in comparison with the proposed model. Experiments were conducted on the sentiment140 dataset and were evaluated in terms of accuracy, precision, recall, and F1 Score. Empirical results showed that our proposed approach manifested state-of-the-art results by achieving an accuracy score of 99%.

A Deep Learning Framework to Predict Routability for FPGA Circuit Placement

2021 ◽  
Vol 14 (3) ◽  
pp. 1-28
Author(s):  
Abeer Al-Hyari ◽  
Hannah Szentimrey ◽  
Ahmed Shamli ◽  
Timothy Martin ◽  
Gary Gréwal ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Parameter Tuning ◽  
Learning Framework ◽  
Proposed Model ◽  
Field Programmable ◽  
Circuit Placement ◽  
Detailed Placement ◽  
Place And Route ◽  
Deep Learning Model

The ability to accurately and efficiently estimate the routability of a circuit based on its placement is one of the most challenging and difficult tasks in the Field Programmable Gate Array (FPGA) flow. In this article, we present a novel, deep learning framework based on a Convolutional Neural Network (CNN) model for predicting the routability of a placement. Since the performance of the CNN model is strongly dependent on the hyper-parameters selected for the model, we perform an exhaustive parameter tuning that significantly improves the model’s performance and we also avoid overfitting the model. We also incorporate the deep learning model into a state-of-the-art placement tool and show how the model can be used to (1) avoid costly, but futile, place-and-route iterations, and (2) improve the placer’s ability to produce routable placements for hard-to-route circuits using feedback based on routability estimates generated by the proposed model. The model is trained and evaluated using over 26K placement images derived from 372 benchmarks supplied by Xilinx Inc. We also explore several opportunities to further improve the reliability of the predictions made by the proposed DLRoute technique by splitting the model into two separate deep learning models for (a) global and (b) detailed placement during the optimization process. Experimental results show that the proposed framework achieves a routability prediction accuracy of 97% while exhibiting runtimes of only a few milliseconds.

scholarly journals Global and Local Attention-Based Free-Form Image Inpainting

Sensors ◽  
2020 ◽  
Vol 20 (11) ◽  
pp. 3204
Author(s):  
S. M. Nadim Uddin ◽  
Yong Ju Jung
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Image Inpainting ◽  
Experimental Results ◽  
Free Form ◽  
Local Similarity ◽  
Global And Local ◽  
Similarity Information

Deep-learning-based image inpainting methods have shown significant promise in both rectangular and irregular holes. However, the inpainting of irregular holes presents numerous challenges owing to uncertainties in their shapes and locations. When depending solely on convolutional neural network (CNN) or adversarial supervision, plausible inpainting results cannot be guaranteed because irregular holes need attention-based guidance for retrieving information for content generation. In this paper, we propose two new attention mechanisms, namely a mask pruning-based global attention module and a global and local attention module to obtain global dependency information and the local similarity information among the features for refined results. The proposed method is evaluated using state-of-the-art methods, and the experimental results show that our method outperforms the existing methods in both quantitative and qualitative measures.

scholarly journals A Deep Learning Based Approach for Localization and Recognition of Pakistani Vehicle License Plates

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7696
Author(s):  
Umair Yousaf ◽  
Ahmad Khan ◽  
Hazrat Ali ◽  
Fiaz Gul Khan ◽  
Zia ur Rehman ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
The Other ◽  
License Plate ◽  
Standard Size ◽  
Current State ◽  
Bounding Box ◽  
Proposed Model ◽  
Plate Area

License plate localization is the process of finding the license plate area and drawing a bounding box around it, while recognition is the process of identifying the text within the bounding box. The current state-of-the-art license plate localization and recognition approaches require license plates of standard size, style, fonts, and colors. Unfortunately, in Pakistan, license plates are non-standard and vary in terms of the characteristics mentioned above. This paper presents a deep-learning-based approach to localize and recognize Pakistani license plates with non-uniform and non-standardized sizes, fonts, and styles. We developed a new Pakistani license plate dataset (PLPD) to train and evaluate the proposed model. We conducted extensive experiments to compare the accuracy of the proposed approach with existing techniques. The results show that the proposed method outperformed the other methods to localize and recognize non-standard license plates.

scholarly journals DeepSide: A Deep Learning Framework for Drug Side Effect Prediction

2019 ◽  
Author(s):  
Onur Can Uner ◽  
Ramazan Gokberk Cinbis ◽  
Oznur Tastan ◽  
A. Ercument Cicek
Keyword(s):  
Gene Expression ◽  
Deep Learning ◽  
Side Effect ◽  
State Of The Art ◽  
Expression Profiles ◽  
Gene Expression Profiles ◽  
Ground Truth ◽  
The State ◽  
Specific Information ◽  
Context Specific

AbstractDrug failures due to unforeseen adverse effects at clinical trials pose health risks for the participants and lead to substantial financial losses. Side effect prediction algorithms have the potential to guide the drug design process. LINCS L1000 dataset provides a vast resource of cell line gene expression data perturbed by different drugs and creates a knowledge base for context specific features. The state-of-the-art approach that aims at using context specific information relies on only the high-quality experiments in LINCS L1000 and discards a large portion of the experiments. In this study, our goal is to boost the prediction performance by utilizing this data to its full extent. We experiment with 5 deep learning architectures. We find that a multi-modal architecture produces the best predictive performance among multi-layer perceptron-based architectures when drug chemical structure (CS), and the full set of drug perturbed gene expression profiles (GEX) are used as modalities. Overall, we observe that the CS is more informative than the GEX. A convolutional neural network-based model that uses only SMILES string representation of the drugs achieves the best results and provides 13.0% macro-AUC and 3.1% micro-AUC improvements over the state-of-the-art. We also show that the model is able to predict side effect-drug pairs that are reported in the literature but was missing in the ground truth side effect dataset. DeepSide is available at http://github.com/OnurUner/DeepSide.

Sign in / Sign up

Export Citation Format

Share Document