scholarly journals A Study on the Effect of DropConnect to Control Overfitting in Designing Neural Networks

2020 ◽  
Author(s):  
Hyun-il Lim
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Neural Network Model ◽  
Network Models ◽  
Training Data ◽  
Neural Network Models ◽  
Structural Problems ◽  
The Neural Network ◽  
Training Neural Network

The neural network is an approach of machine learning by training the connected nodes of a model to predict the results of specific problems. The prediction model is trained by using previously collected training data. In training neural network models, overfitting problems can occur from the excessively dependent training of data and the structural problems of the models. In this paper, we analyze the effect of DropConnect for controlling overfitting in neural networks. It is analyzed according to the DropConnect rates and the number of nodes in designing neural networks. The analysis results of this study help to understand the effect of DropConnect in neural networks. To design an effective neural network model, the DropConnect can be applied with appropriate parameters from the understanding of the effect of the DropConnect in neural network models.

scholarly journals Robotic grasp detection using a novel two-stage approach

2021 ◽  
Vol 1 (1) ◽  
pp. 19-29
Author(s):  
Zhe Chu ◽  
Mengkai Hu ◽  
Xiangyu Chen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Models ◽  
Particle Swarm Optimizer ◽  
Neural Network Models ◽  
Two Stage ◽  
The Neural Network ◽  
End To End ◽  
Small Change ◽  
Robotic Grasp

Recently, deep learning has been successfully applied to robotic grasp detection. Based on convolutional neural networks (CNNs), there have been lots of end-to-end detection approaches. But end-to-end approaches have strict requirements for the dataset used for training the neural network models and it’s hard to achieve in practical use. Therefore, we proposed a two-stage approach using particle swarm optimizer (PSO) candidate estimator and CNN to detect the most likely grasp. Our approach achieved an accuracy of 92.8% on the Cornell Grasp Dataset, which leaped into the front ranks of the existing approaches and is able to run at real-time speeds. After a small change of the approach, we can predict multiple grasps per object in the meantime so that an object can be grasped in a variety of ways.

Analysis of Fin-Tube Evaporator Performance With Limited Experimental Data Using Artificial Neural Networks

2000 ◽  
Author(s):  
Arturo Pacheco-Vega ◽  
Mihir Sen ◽  
Rodney L. McClain
Keyword(s):  
Neural Network ◽  
Heat Rate ◽  
Network Models ◽  
Activation Function ◽  
Operating Conditions ◽  
Training Data ◽  
Neural Network Models ◽  
The Neural Network ◽  
Artificial Neural ◽  
Fin Tube

Abstract In the current study we consider the problem of accuracy in heat rate estimations from artificial neural network models of heat exchangers used for refrigeration applications. The network configuration is of the feedforward type with a sigmoid activation function and a backpropagation algorithm. Limited experimental measurements from a manufacturer are used to show the capability of the neural network technique in modeling the heat transfer in these systems. Results from this exercise show that a well-trained network correlates the data with errors of the same order as the uncertainty of the measurements. It is also shown that the number and distribution of the training data are linked to the performance of the network when estimating the heat rates under different operating conditions, and that networks trained from few tests may give large errors. A methodology based on the cross-validation technique is presented to find regions where not enough data are available to construct a reliable neural network. The results from three tests show that the proposed methodology gives an upper bound of the estimated error in the heat rates.

Artificial Neural Networks

2011 ◽  
pp. 222-243
Author(s):  
Joarder Kamruzzaman ◽  
Ruhul Sarker
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Artificial Neural Networks ◽  
Network Models ◽  
Neural Network Models ◽  
Network Applications ◽  
The Neural Network ◽  
Artificial Neural ◽  
Neural Network Applications

The primary aim of this chapter is to present an overview of the artificial neural network basics and operation, architectures, and the major algorithms used for training the neural network models. As can be seen in subsequent chapters, neural networks have made many useful contributions to solve theoretical and practical problems in finance and manufacturing areas. The secondary aim here is therefore to provide a brief review of artificial neural network applications in finance and manufacturing areas.

scholarly journals Real-Time Guitar Amplifier Emulation with Deep Learning

2020 ◽  
Vol 10 (3) ◽  
pp. 766 ◽  
Author(s):  
Alec Wright ◽  
Eero-Pekka Damskägg ◽  
Lauri Juvela ◽  
Vesa Välimäki
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Reference Signal ◽  
Network Models ◽  
Neural Network Models ◽  
Desktop Computer ◽  
The Neural Network ◽  
Highly Nonlinear ◽  
The Difference

This article investigates the use of deep neural networks for black-box modelling of audio distortion circuits, such as guitar amplifiers and distortion pedals. Both a feedforward network, based on the WaveNet model, and a recurrent neural network model are compared. To determine a suitable hyperparameter configuration for the WaveNet, models of three popular audio distortion pedals were created: the Ibanez Tube Screamer, the Boss DS-1, and the Electro-Harmonix Big Muff Pi. It is also shown that three minutes of audio data is sufficient for training the neural network models. Real-time implementations of the neural networks were used to measure their computational load. To further validate the results, models of two valve amplifiers, the Blackstar HT-5 Metal and the Mesa Boogie 5:50 Plus, were created, and subjective tests were conducted. The listening test results show that the models of the first amplifier could be identified as different from the reference, but the sound quality of the best models was judged to be excellent. In the case of the second guitar amplifier, many listeners were unable to hear the difference between the reference signal and the signals produced with the two largest neural network models. This study demonstrates that the neural network models can convincingly emulate highly nonlinear audio distortion circuits, whilst running in real-time, with some models requiring only a relatively small amount of processing power to run on a modern desktop computer.

An explicit methodology for manufacturing cost–tolerance modeling and optimization using the neural network integrated with the genetic algorithm

2020 ◽  
Vol 34 (3) ◽  
pp. 430-443
Author(s):  
A. Saravanan ◽  
J. Jerald ◽  
A. Delphin Carolina Rani
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Network Model ◽  
Neural Network Model ◽  
Network Models ◽  
Manufacturing Cost ◽  
Genetic Operators ◽  
Neural Network Models ◽  
Data Set ◽  
The Neural Network

AbstractThe objective of the paper is to develop a new method to model the manufacturing cost–tolerance and to optimize the tolerance values along with its manufacturing cost. A cost–tolerance relation has a complex nonlinear correlation among them. The property of a neural network makes it possible to model the complex correlation, and the genetic algorithm (GA) is integrated with the best neural network model to optimize the tolerance values. The proposed method used three types of neural network models (multilayer perceptron, backpropagation network, and radial basis function). These network models were developed separately for prismatic and rotational parts. For the construction of network models, part size and tolerance values were used as input neurons. The reference manufacturing cost was assigned as the output neuron. The qualitative production data set was gathered in a workshop and partitioned into three files for training, testing, and validation, respectively. The architecture of the network model was identified based on the best regression coefficient and the root-mean-square-error value. The best network model was integrated into the GA, and the role of genetic operators was also studied. Finally, two case studies from the literature were demonstrated in order to validate the proposed method. A new methodology based on the neural network model enables the design and process planning engineers to propose an intelligent decision irrespective of their experience.

scholarly journals Comparison of rule-based and neural network models for negation detection in radiology reports

2020 ◽  
pp. 1-22 ◽  
Author(s):  
D. Sykes ◽  
A. Grivas ◽  
C. Grover ◽  
R. Tobin ◽  
C. Sudlow ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Language Processing ◽  
Network Models ◽  
Neural Network Models ◽  
Test Set ◽  
Rule Based ◽  
Radiology Reports ◽  
The Neural Network ◽  
Negation Detection

Abstract Using natural language processing, it is possible to extract structured information from raw text in the electronic health record (EHR) at reasonably high accuracy. However, the accurate distinction between negated and non-negated mentions of clinical terms remains a challenge. EHR text includes cases where diseases are stated not to be present or only hypothesised, meaning a disease can be mentioned in a report when it is not being reported as present. This makes tasks such as document classification and summarisation more difficult. We have developed the rule-based EdIE-R-Neg, part of an existing text mining pipeline called EdIE-R (Edinburgh Information Extraction for Radiology reports), developed to process brain imaging reports, (https://www.ltg.ed.ac.uk/software/edie-r/) and two machine learning approaches; one using a bidirectional long short-term memory network and another using a feedforward neural network. These were developed on data from the Edinburgh Stroke Study (ESS) and tested on data from routine reports from NHS Tayside (Tayside). Both datasets consist of written reports from medical scans. These models are compared with two existing rule-based models: pyConText (Harkema et al. 2009. Journal of Biomedical Informatics42(5), 839–851), a python implementation of a generalisation of NegEx, and NegBio (Peng et al. 2017. NegBio: A high-performance tool for negation and uncertainty detection in radiology reports. arXiv e-prints, p. arXiv:1712.05898), which identifies negation scopes through patterns applied to a syntactic representation of the sentence. On both the test set of the dataset from which our models were developed, as well as the largely similar Tayside test set, the neural network models and our custom-built rule-based system outperformed the existing methods. EdIE-R-Neg scored highest on F1 score, particularly on the test set of the Tayside dataset, from which no development data were used in these experiments, showing the power of custom-built rule-based systems for negation detection on datasets of this size. The performance gap of the machine learning models to EdIE-R-Neg on the Tayside test set was reduced through adding development Tayside data into the ESS training set, demonstrating the adaptability of the neural network models.

Identifying User Suitability in sEMG Based Hand Prosthesis Using Neural Networks

2019 ◽  
Vol 14 (2) ◽  
pp. 158-164 ◽  
Author(s):  
G. Emayavaramban ◽  
A. Amudha ◽  
T. Rajendran ◽  
M. Sivaramkumar ◽  
K. Balachandar ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Pattern Recognition ◽  
Feature Extraction ◽  
Classification Accuracy ◽  
Network Models ◽  
Prosthetic Hand ◽  
Neural Network Models ◽  
The Neural Network ◽  
Semg Signals

Background: Identifying user suitability plays a vital role in various modalities like neuromuscular system research, rehabilitation engineering and movement biomechanics. This paper analysis the user suitability based on neural networks (NN), subjects, age groups and gender for surface electromyogram (sEMG) pattern recognition system to control the myoelectric hand. Six parametric feature extraction algorithms are used to extract the features from sEMG signals such as AR (Autoregressive) Burg, AR Yule Walker, AR Covariance, AR Modified Covariance, Levinson Durbin Recursion and Linear Prediction Coefficient. The sEMG signals are modeled using Cascade Forward Back propagation Neural Network (CFBNN) and Pattern Recognition Neural Network. Methods: sEMG signals generated from forearm muscles of the participants are collected through an sEMG acquisition system. Based on the sEMG signals, the type of movement attempted by the user is identified in the sEMG recognition module using signal processing, feature extraction and machine learning techniques. The information about the identified movement is passed to microcontroller wherein a control is developed to command the prosthetic hand to emulate the identified movement. Results: From the six feature extraction algorithms and two neural network models used in the study, the maximum classification accuracy of 95.13% was obtained using AR Burg with Pattern Recognition Neural Network. This justifies that the Pattern Recognition Neural Network is best suited for this study as the neural network model is specially designed for pattern matching problem. Moreover, it has simple architecture and low computational complexity. AR Burg is found to be the best feature extraction technique in this study due to its high resolution for short data records and its ability to always produce a stable model. In all the neural network models, the maximum classification accuracy is obtained for subject 10 as a result of his better muscle fitness and his maximum involvement in training sessions. Subjects in the age group of 26-30 years are best suited for the study due to their better muscle contractions. Better muscle fatigue resistance has contributed for better performance of female subjects as compared to male subjects. From the single trial analysis, it can be observed that the hand close movement has achieved best recognition rate for all neural network models. Conclusion: In this paper a study was conducted to identify user suitability for designing hand prosthesis. Data were collected from ten subjects for twelve tasks related to finger movements. The suitability of the user was identified using two neural networks with six parametric features. From the result, it was concluded thatfit women doing regular physical exercises aged between 26-30 years are best suitable for developing HMI for designing a prosthetic hand. Pattern Recognition Neural Network with AR Burg extraction features using extension movements will be a better way to design the HMI. However, Signal acquisition based on wireless method is worth considering for the future.

scholarly journals Feature-Based Interpretation of the Deep Neural Network

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2687
Author(s):  
Eun-Hun Lee ◽  
Hyeoncheol Kim
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Lower Layer ◽  
Network Models ◽  
Neural Network Models ◽  
Network Pruning ◽  
The Neural Network ◽  
Single Instance ◽  
High Level

The significant advantage of deep neural networks is that the upper layer can capture the high-level features of data based on the information acquired from the lower layer by stacking layers deeply. Since it is challenging to interpret what knowledge the neural network has learned, various studies for explaining neural networks have emerged to overcome this problem. However, these studies generate the local explanation of a single instance rather than providing a generalized global interpretation of the neural network model itself. To overcome such drawbacks of the previous approaches, we propose the global interpretation method for the deep neural network through features of the model. We first analyzed the relationship between the input and hidden layers to represent the high-level features of the model, then interpreted the decision-making process of neural networks through high-level features. In addition, we applied network pruning techniques to make concise explanations and analyzed the effect of layer complexity on interpretability. We present experiments on the proposed approach using three different datasets and show that our approach could generate global explanations on deep neural network models with high accuracy and fidelity.

scholarly journals Analysis of Algorithms for Searching Objects in Images Using Convolutional Neural Network

2021 ◽  
Vol 6 (2) ◽  
pp. 128-133
Author(s):  
Ihor Koval ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Comparative Analysis ◽  
Analysis Of Algorithms ◽  
Network Models ◽  
Data Sets ◽  
Neural Network Models ◽  
Network Algorithms ◽  
Modern Computer ◽  
The Neural Network

The problem of finding objects in images using modern computer vision algorithms has been considered. The description of the main types of algorithms and methods for finding objects based on the use of convolutional neural networks has been given. A comparative analysis and modeling of neural network algorithms to solve the problem of finding objects in images has been conducted. The results of testing neural network models with different architectures on data sets VOC2012 and COCO have been presented. The results of the study of the accuracy of recognition depending on different hyperparameters of learning have been analyzed. The change in the value of the time of determining the location of the object depending on the different architectures of the neural network has been investigated.

scholarly journals Tongue fissure visualization by using deep learning – an example of the application of artificial intelligence in traditional medicine

2020 ◽  
Author(s):  
Wen-Hsien Chang ◽  
Han-Kuei Wu ◽  
Lun-chien Lo ◽  
William W. L. Hsiao ◽  
Hsueh-Ting Chu ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Transfer Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Graphics Processing Units ◽  
Network Models ◽  
Neural Network Models ◽  
The Neural Network ◽  
Objective Evidence

Abstract Background: Traditional Chinese medicine (TCM) describes physiological and pathological changes inside and outside the human body by the application of four methods of diagnosis. One of the four methods, tongue diagnosis, is widely used by TCM physicians, since it allows direct observations that prevent discrepancies in the patient’s history and, as such, provides clinically important, objective evidence. The clinical significance of tongue features has been explored in both TCM and modern medicine. However, TCM physicians may have different interpretations of the features displayed by the same tongue, and therefore intra- and inter-observer agreements are relatively low. If an automated interpretation system could be developed, more consistent results could be obtained, and learning could also be more efficient. This study will apply a recently developed deep learning method to the classification of tongue features, and indicate the regions where the features are located.Methods: A large number of tongue photographs with labeled fissures were used. Transfer learning was conducted using the ImageNet-pretrained ResNet50 model to determine whether tongue fissures were identified on a tongue photograph. Often, the neural network model lacks interpretability, and users cannot understand how the model determines the presence of tongue fissures. Therefore, Gradient-weighted Class Activation Mapping (Grad-CAM) was also applied to directly mark the tongue features on the tongue image. Results: Only 6 epochs were trained in this study and no graphics processing units (GPUs) were used. It took less than 4 minutes for each epoch to be trained. The correct rate for the test set was approximately 70%. After the model training was completed, Grad-CAM was applied to localize tongue fissures in each image. The neural network model not only determined whether tongue fissures existed, but also allowed users to learn about the tongue fissure regions.Conclusions: This study demonstrated how to apply transfer learning using the ImageNet-pretrained ResNet50 model for the identification and localization of tongue fissures and regions. The neural network model built in this study provided interpretability and intuitiveness, (often lacking in general neural network models), and improved the feasibility for clinical application.

Sign in / Sign up

Export Citation Format

Share Document