scholarly journals Deep Convolutional Nets Learning Classification for Artistic Style Transfer

2022 ◽  
Vol 2022 ◽  
pp. 1-9
Author(s):  
R. Dinesh Kumar ◽  
E. Golden Julie ◽  
Y. Harold Robinson ◽  
S. Vimal ◽  
Gaurav Dhiman ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Feature Space ◽  
Input Image ◽  
Reference Image ◽  
Nonphotorealistic Rendering ◽  
Style Transfer ◽  
Artistic Style ◽  
The Neural Network

Humans have mastered the skill of creativity for many decades. The process of replicating this mechanism is introduced recently by using neural networks which replicate the functioning of human brain, where each unit in the neural network represents a neuron, which transmits the messages from one neuron to other, to perform subconscious tasks. Usually, there are methods to render an input image in the style of famous art works. This issue of generating art is normally called nonphotorealistic rendering. Previous approaches rely on directly manipulating the pixel representation of the image. While using deep neural networks which are constructed using image recognition, this paper carries out implementations in feature space representing the higher levels of the content image. Previously, deep neural networks are used for object recognition and style recognition to categorize the artworks consistent with the creation time. This paper uses Visual Geometry Group (VGG16) neural network to replicate this dormant task performed by humans. Here, the images are input where one is the content image which contains the features you want to retain in the output image and the style reference image which contains patterns or images of famous paintings and the input image which needs to be style and blend them together to produce a new image where the input image is transformed to look like the content image but “sketched” to look like the style image.

OPTIMIZATION PROCESS ANALYSIS FOR HYPERPARAMETERS OF NEURAL NETWORK DATA PROCESSING STRUCTURES

2020 ◽  
pp. 3-10
Author(s):  
V. N. Gridin ◽  
I. A. Evdokimov ◽  
B. R. Salem ◽  
V. I. Solodovnikov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Process Analysis ◽  
Validation Process ◽  
The Neural Network ◽  
Source Data ◽  
The Neural Networks ◽  
Qualitative Characteristics ◽  
Setting Parameters

The analysis of key stages, implementation features and functioning principles of the neural networks, including deep neural networks, has been carried out. The problems of choosing the number of hidden elements, methods for the internal topology selection and setting parameters are considered. It is shown that in the training and validation process it is possible to control the capacity of a neural network and evaluate the qualitative characteristics of the constructed model. The issues of construction processes automation and hyperparameters optimization of the neural network structures are considered depending on the user's tasks and the available source data. A number of approaches based on the use of probabilistic programming, evolutionary algorithms, and recurrent neural networks are presented.

scholarly journals Interactive Artistic Multi-style Transfer

2021 ◽  
Vol 14 (1) ◽  
Author(s):  
Xiaohui Wang ◽  
Yiran Lyu ◽  
Junfeng Huang ◽  
Ziying Wang ◽  
Jingyan Qin
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Deep Neural Networks ◽  
Subjective Evaluation ◽  
User Interaction ◽  
Evaluation Framework ◽  
Transfer System ◽  
Style Transfer ◽  
Artistic Style ◽  
Artistic Styles

AbstractArtistic style transfer is to render an image in the style of another image, which is a challenge problem in both image processing and arts. Deep neural networks are adopted to artistic style transfer and achieve remarkable success, such as AdaIN (adaptive instance normalization), WCT (whitening and coloring transforms), MST (multimodal style transfer), and SEMST (structure-emphasized multimodal style transfer). These algorithms modify the content image as a whole using only one style and one algorithm, which is easy to cause the foreground and background to be blurred together. In this paper, an iterative artistic multi-style transfer system is built to edit the image with multiple styles by flexible user interaction. First, a subjective evaluation experiment with art professionals is conducted to build an open evaluation framework for style transfer, including the universal evaluation questions and personalized answers for ten typical artistic styles. Then, we propose the interactive artistic multi-style transfer system, in which an interactive image crop tool is designed to cut a content image into several parts. For each part, users select a style image and an algorithm from AdaIN, WCT, MST, and SEMST by referring to the characteristics of styles and algorithms summarized by the evaluation experiments. To obtain richer results, the system provides a semantic-based parameter adjustment mode and the function of preserving colors of content image. Finally, case studies show the effectiveness and flexibility of the system.

scholarly journals Bias in Deep Neural Networks in Land Use Characterization for International Development

2021 ◽  
Vol 13 (15) ◽  
pp. 2908
Author(s):  
Do-Hyung Kim ◽  
Guzmán López ◽  
Diego Kiedanski ◽  
Iyke Maduako ◽  
Braulio Ríos ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Land Use ◽  
International Development ◽  
Deep Neural Networks ◽  
Satellite Image ◽  
Fine Tuning ◽  
Training Dataset ◽  
Model Accuracy ◽  
The Neural Network

Understanding the biases in Deep Neural Networks (DNN) based algorithms is gaining paramount importance due to its increased applications on many real-world problems. A known problem of DNN penalizing the underrepresented population could undermine the efficacy of development projects dependent on data produced using DNN-based models. In spite of this, the problems of biases in DNN for Land Use and Land Cover Classification (LULCC) have not been a subject of many studies. In this study, we explore ways to quantify biases in DNN for land use with an example of identifying school buildings in Colombia from satellite imagery. We implement a DNN-based model by fine-tuning an existing, pre-trained model for school building identification. The model achieved overall 84% accuracy. Then, we used socioeconomic covariates to analyze possible biases in the learned representation. The retrained deep neural network was used to extract visual features (embeddings) from satellite image tiles. The embeddings were clustered into four subtypes of schools, and the accuracy of the neural network model was assessed for each cluster. The distributions of various socioeconomic covariates by clusters were analyzed to identify the links between the model accuracy and the aforementioned covariates. Our results indicate that the model accuracy is lowest (57%) where the characteristics of the landscape are predominantly related to poverty and remoteness, which confirms our original assumption on the heterogeneous performances of Artificial Intelligence (AI) algorithms and their biases. Based on our findings, we identify possible sources of bias and present suggestions on how to prepare a balanced training dataset that would result in less biased AI algorithms. The framework used in our study to better understand biases in DNN models would be useful when Machine Learning (ML) techniques are adopted in lieu of ground-based data collection for international development programs. Because such programs aim to solve issues of social inequality, MLs are only applicable when they are transparent and accountable.

scholarly journals Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training

2021 ◽  
Vol 118 (43) ◽  
pp. e2103091118
Author(s):  
Cong Fang ◽  
Hangfeng He ◽  
Qi Long ◽  
Weijie J. Su
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Model Minority ◽  
Tight Frame ◽  
Learning Models ◽  
The Neural Network ◽  
Long Time ◽  
Topmost Layer

In this paper, we introduce the Layer-Peeled Model, a nonconvex, yet analytically tractable, optimization program, in a quest to better understand deep neural networks that are trained for a sufficiently long time. As the name suggests, this model is derived by isolating the topmost layer from the remainder of the neural network, followed by imposing certain constraints separately on the two parts of the network. We demonstrate that the Layer-Peeled Model, albeit simple, inherits many characteristics of well-trained neural networks, thereby offering an effective tool for explaining and predicting common empirical patterns of deep-learning training. First, when working on class-balanced datasets, we prove that any solution to this model forms a simplex equiangular tight frame, which, in part, explains the recently discovered phenomenon of neural collapse [V. Papyan, X. Y. Han, D. L. Donoho, Proc. Natl. Acad. Sci. U.S.A. 117, 24652–24663 (2020)]. More importantly, when moving to the imbalanced case, our analysis of the Layer-Peeled Model reveals a hitherto-unknown phenomenon that we term Minority Collapse, which fundamentally limits the performance of deep-learning models on the minority classes. In addition, we use the Layer-Peeled Model to gain insights into how to mitigate Minority Collapse. Interestingly, this phenomenon is first predicted by the Layer-Peeled Model before being confirmed by our computational experiments.

scholarly journals Construction of a neural network energy function for protein physics

2021 ◽  
Author(s):  
Huan Yang ◽  
Zhaoping Xiong ◽  
Francesco Zonta
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Neural Networks ◽  
Energy Function ◽  
Protein Sequence ◽  
Deep Neural Networks ◽  
Protein Structures ◽  
Learning Method ◽  
Local Structures ◽  
The Neural Network

AbstractClassical potentials are widely used to describe protein physics, due to their simplicity and accuracy, but they are continuously challenged as real applications become more demanding with time. Deep neural networks could help generating alternative ways of describing protein physics. Here we propose an unsupervised learning method to derive a neural network energy function for proteins. The energy function is a probability density model learned from plenty of 3D local structures which have been extensively explored by evolution. We tested this model on a few applications (assessment of protein structures, protein dynamics and protein sequence design), showing that the neural network can correctly recognize patterns in protein structures. In other words, the neural network learned some aspects of protein physics from experimental data.

Domain Randomization Using Deep Neural Networks for Estimating Positions of Bolts

2020 ◽  
Vol 20 (5) ◽  
Author(s):  
Ezra Ameperosa ◽  
Pranav A. Bhounsule
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Single Object ◽  
Automated Method ◽  
Bounding Box ◽  
The Neural Network ◽  
Camera Position ◽  
Position And Orientation

Abstract Current manual practices of replacing bolts on structures are time-consuming and costly, especially because of numerous bolts. Thus, an automated method that can visually detect and localize bolt positions would be highly beneficial. We demonstrate the use of deep neural networks using domain randomization for detecting and localizing bolts on a workpiece. In contrast to previous approaches that require training on real images, the use of domain randomization enables all training in simulation. The key idea is to create a wide variety of computer-generated synthetic images by varying the texture, color, camera position and orientation, distractor objects, and noise, and train the neural network on these images such that the neural network is robust to scene variability and hence provides accurate results when deployed on real images. Using domain randomization, we train two neural networks, a faster regional convolutional neural network for detecting the bolt and placing a bounding box, and a regression convolutional neural network for estimating the x- and y-position of the bolts relative to the coordinates fixed to the workpiece. Our results indicate that in the best case, we can detect bolts with 85% accuracy and can predict 75% of bolts within 1.27 cm accuracy. The novelty of this work is in using domain randomization to detect and localize: (1) multiples of a single object and (2) small-sized objects (0.6 cm × 2.5 cm).

scholarly journals A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-7
Author(s):  
Xin Long ◽  
XiangRong Zeng ◽  
Zongcheng Ben ◽  
Dianle Zhou ◽  
Maojun Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Computational Cost ◽  
Network Models ◽  
Memory Consumption ◽  
Neural Network Models ◽  
The Neural Network ◽  
The Neural Networks ◽  
Novel Strategy

The increase in sophistication of neural network models in recent years has exponentially expanded memory consumption and computational cost, thereby hindering their applications on ASIC, FPGA, and other mobile devices. Therefore, compressing and accelerating the neural networks are necessary. In this study, we introduce a novel strategy to train low-bit networks with weights and activations quantized by several bits and address two corresponding fundamental issues. One is to approximate activations through low-bit discretization for decreasing network computational cost and dot-product memory. The other is to specify weight quantization and update mechanism for discrete weights to avoid gradient mismatch. With quantized low-bit weights and activations, the costly full-precision operation will be replaced by shift operation. We evaluate the proposed method on common datasets, and results show that this method can dramatically compress the neural network with slight accuracy loss.

scholarly journals Surrogate Object Detection Explainer (SODEx) with YOLOv4 and LIME

2021 ◽  
Vol 3 (3) ◽  
pp. 662-671
Author(s):  
Jonas Herskind Sejr ◽  
Peter Schneider-Kamp ◽  
Naeem Ayoub
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Deep Neural Networks ◽  
Empirical Evaluation ◽  
Network Models ◽  
Detection Algorithm ◽  
Neural Network Models ◽  
The Neural Network

Due to impressive performance, deep neural networks for object detection in images have become a prevalent choice. Given the complexity of the neural network models used, users of these algorithms are typically given no hint as to how the objects were found. It remains, for example, unclear whether an object is detected based on what it looks like or based on the context in which it is located. We have developed an algorithm, Surrogate Object Detection Explainer (SODEx), that can explain any object detection algorithm using any classification explainer. We evaluate SODEx qualitatively and quantitatively by detecting objects in the COCO dataset with YOLOv4 and explaining these detections with LIME. This empirical evaluation does not only demonstrate the value of explainable object detection, it also provides valuable insights into how YOLOv4 detects objects.

scholarly journals Bird Detection using Siamese Neural Network

2020 ◽  
Vol 9 (7) ◽  
pp. 1168-1171
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Bird Species ◽  
Siamese Network ◽  
The Neural Network ◽  
Data Points ◽  
Bird Detection ◽  
Processing Techniques ◽  
Do So

The proposed system uses deep neural networks for identifying bird species. The model will be trained on bird images that are coming in the endangered species category. The application can also handle new data points, unlike existing systems that require model re-training for accommodating new data. The system can identify bird species in a large view of the image. The model will be trained using a convolutional neural network-based architecture called Siamese Network. This network is also called one-shot learning which means that it requires only few training example for each class. Existing models use image processing techniques or vanilla convolutional neural networks for classifying bird images. These models cannot accommodate new images and have to be retrained to do so. There is no commercially available system that can detect a species of bird in high resolution / large image. While in the Siamese network we only have to add new data, there is no need to retraining the neural network.

Sign in / Sign up

Export Citation Format

Share Document