Domain Randomization Using Deep Neural Networks for Estimating Positions of Bolts

2020 ◽  
Vol 20 (5) ◽  
Author(s):  
Ezra Ameperosa ◽  
Pranav A. Bhounsule
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Single Object ◽  
Automated Method ◽  
Bounding Box ◽  
The Neural Network ◽  
Camera Position ◽  
Position And Orientation

Abstract Current manual practices of replacing bolts on structures are time-consuming and costly, especially because of numerous bolts. Thus, an automated method that can visually detect and localize bolt positions would be highly beneficial. We demonstrate the use of deep neural networks using domain randomization for detecting and localizing bolts on a workpiece. In contrast to previous approaches that require training on real images, the use of domain randomization enables all training in simulation. The key idea is to create a wide variety of computer-generated synthetic images by varying the texture, color, camera position and orientation, distractor objects, and noise, and train the neural network on these images such that the neural network is robust to scene variability and hence provides accurate results when deployed on real images. Using domain randomization, we train two neural networks, a faster regional convolutional neural network for detecting the bolt and placing a bounding box, and a regression convolutional neural network for estimating the x- and y-position of the bolts relative to the coordinates fixed to the workpiece. Our results indicate that in the best case, we can detect bolts with 85% accuracy and can predict 75% of bolts within 1.27 cm accuracy. The novelty of this work is in using domain randomization to detect and localize: (1) multiples of a single object and (2) small-sized objects (0.6 cm × 2.5 cm).

Domain Randomization for Detection and Position Estimation of Multiples of a Single Object With Applications to Localizing Bolts on Structures

2019 ◽  
Author(s):  
Ezra Ameperosa ◽  
Pranav A. Bhounsule
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Position Estimation ◽  
Learning Approaches ◽  
Single Object ◽  
Automated Method ◽  
The Neural Network ◽  
Periodic Maintenance ◽  
Camera Position ◽  
Periodic Replacement

Abstract Periodic replacement of fasteners such as bolts are an integral part of many structures (e.g., airplanes, cars, ships) and require periodic maintenance that may involve either their tightening or replacement. Current manual practices are time consuming and costly especially due to the large number of bolts. Thus, an automated method that is able to visually detect and localize bolt positions would be highly beneficial. In this paper, we demonstrate the use of deep neural network using domain randomization for detecting and localizing multiple bolts on a workpiece. In contrast to previous deep learning approaches that require training on real images, the use of domain randomization allows for all training to be done in simulation. The key idea here is to create a wide variety of computer generated synthetic images by varying the texture, color, camera position and orientation, distractor objects, and noise, and train the neural network on these images such that the neural network is robust to scene variability and hence provides accurate results when deployed on real images. Using domain randomization, we train two neural networks, a faster regional convolutional neural network for detecting the bolt and predicting a bounding box, and a regression convolutional neural network for estimating the x- and y-position of the bolt relative to the coordinates fixed to the workpiece. Our results indicate that in the best case we are able to detect bolts with 85% accuracy and are able to predict the position of 75% of bolts within 1.27 cm. The novelty of this work is in the use of domain randomization to detect and localize: (1) multiples of a single object, and (2) small sized objects (0.6 cm × 2.5 cm).

EMOTIONS RECOGNITION IN HUMAN SPEECH USING DEEP NEURAL NETWORKS

2021 ◽  
pp. 44-51
Author(s):  
E. Yu. Shchetinin
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Network ◽  
Deep Neural Networks ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Audio Recordings ◽  
Computer Studies

The recognition of human emotions is one of the most relevant and dynamically developing areas of modern speech technologies, and the recognition of emotions in speech (RER) is the most demanded part of them. In this paper, we propose a computer model of emotion recognition based on an ensemble of bidirectional recurrent neural network with LSTM memory cell and deep convolutional neural network ResNet18. In this paper, computer studies of the RAVDESS database containing emotional speech of a person are carried out. RAVDESS-a data set containing 7356 files. Entries contain the following emotions: 0 – neutral, 1 – calm, 2 – happiness, 3 – sadness, 4 – anger, 5 – fear, 6 – disgust, 7 – surprise. In total, the database contains 16 classes (8 emotions divided into male and female) for a total of 1440 samples (speech only). To train machine learning algorithms and deep neural networks to recognize emotions, existing audio recordings must be pre-processed in such a way as to extract the main characteristic features of certain emotions. This was done using Mel-frequency cepstral coefficients, chroma coefficients, as well as the characteristics of the frequency spectrum of audio recordings. In this paper, computer studies of various models of neural networks for emotion recognition are carried out on the example of the data described above. In addition, machine learning algorithms were used for comparative analysis. Thus, the following models were trained during the experiments: logistic regression (LR), classifier based on the support vector machine (SVM), decision tree (DT), random forest (RF), gradient boosting over trees – XGBoost, convolutional neural network CNN, recurrent neural network RNN (ResNet18), as well as an ensemble of convolutional and recurrent networks Stacked CNN-RNN. The results show that neural networks showed much higher accuracy in recognizing and classifying emotions than the machine learning algorithms used. Of the three neural network models presented, the CNN + BLSTM ensemble showed higher accuracy.

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

scholarly journals APPLICATION OF CONVOLUTIONAL NEURAL NETWORK FOR DETECTION OF MELANOMA USING SKIN LESION IMAGE ON MOBILE DEVICE

2021 ◽  
Vol 3 (1) ◽  
pp. 8-14
Author(s):  
D. V. Fedasyuk ◽  
◽  
T. V. Demianets ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Mobile Device ◽  
Mobile Application ◽  
High Accuracy ◽  
File Size ◽  
Melanoma Detection ◽  
The Neural Network ◽  
Classification Time

A melanoma is the deadliest skin cancer, so early diagnosis can provide a positive prognosis for treatment. Modern methods for early detecting melanoma on the image of the tumor are considered, and their advantages and disadvantages are analyzed. The article demonstrates a prototype of a mobile application for the detection of melanoma on the image of a mole based on a convolutional neural network, which is developed for the Android operating system. The mobile application contains melanoma detection functions, history of the previous examinations and a gallery with images of the previous examinations grouped by the location of the lesion. The HAM10000-based training dataset has been supplemented with the images of melanoma from the archive of The International Skin Imaging Collaboration to eliminate class imbalances and improve network accuracy. The search for existing neural networks that provide high accuracy was conducted, and VGG16, MobileNet, and NASNetMobile neural networks have been selected for research. Transfer learning and fine-tuning has been applied to the given neural networks to adapt the networks for the task of skin lesion classification. It is established that the use of these techniques allows to obtain high accuracy of the neural network for this task. The process of converting a convolutional neural network to an optimized Flatbuffer format using TensorFlow Lite for placement and use on a mobile device is described. The performance characteristics of the selected neural networks on the mobile device are evaluated according to the classification time on the CPU and GPU and the amount of memory occupied by the file of a single network is compared. The neural network file size was compared before and after conversion. It has been shown that the use of the TensorFlow Lite converter significantly reduces the file size of the neural network without affecting its accuracy by using an optimized format. The results of the study indicate a high speed of application and compactness of networks on the device, and the use of graphical acceleration can significantly decrease the image classification time of the tumor. According to the analyzed parameters, NASNetMobile was selected as the optimal neural network to be used in the mobile application of melanoma detection.

Damage Detection of Composite Materials Using Data Fusion With Deep Neural Networks

2020 ◽  
Author(s):  
Shweta Dabetwar ◽  
Stephen Ekwaro-Osire ◽  
João Paulo Dias
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Composite Materials ◽  
Data Fusion ◽  
Damage Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Mixed Data ◽  
Image Encoding ◽  
Using Data

Abstract Composite materials have enormous applications in various fields. Thus, it is important to have an efficient damage detection method to avoid catastrophic failures. Due to the existence of multiple damage modes and the availability of data in different formats, it is important to employ efficient techniques to consider all the types of damage. Deep neural networks were seen to exhibit the ability to address similar complex problems. The research question in this work is ‘Can data fusion improve damage classification using the convolutional neural network?’ The specific aims developed were to 1) assess the performance of image encoding algorithms, 2) classify the damage using data from separate experimental coupons, and 3) classify the damage using mixed data from multiple experimental coupons. Two different experimental measurements were taken from NASA Ames Prognostic Repository for Carbon Fiber Reinforced polymer. To use data fusion, the piezoelectric signals were converted into images using Gramian Angular Field (GAF) and Markov Transition Field. Using data fusion techniques, the input dataset was created for a convolutional neural network with three hidden layers to determine the damage states. The accuracies of all the image encoding algorithms were compared. The analysis showed that data fusion provided better results as it contained more information on the damages modes that occur in composite materials. Additionally, GAF was shown to perform the best. Thus, the combination of data fusion and deep neural network techniques provides an efficient method for damage detection of composite materials.

Deep Convolutional Neural Network for Object Classification

2021 ◽  
pp. 317-343
Author(s):  
Amira Ahmad Al-Sharkawy ◽  
Gehan A. Bahgat ◽  
Elsayed E. Hemayed ◽  
Samia Abdel-Razik Mashali
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Computational Models ◽  
Human Performance ◽  
Deep Neural Networks ◽  
Object Classification ◽  
Classification Problem ◽  
Deep Convolutional Neural Network ◽  
Object Appearance

Object classification problem is essential in many applications nowadays. Human can easily classify objects in unconstrained environments easily. Classical classification techniques were far away from human performance. Thus, researchers try to mimic the human visual system till they reached the deep neural networks. This chapter gives a review and analysis in the field of the deep convolutional neural network usage in object classification under constrained and unconstrained environment. The chapter gives a brief review on the classical techniques of object classification and the development of bio-inspired computational models from neuroscience till the creation of deep neural networks. A review is given on the constrained environment issues: the hardware computing resources and memory, the object appearance and background, and the training and processing time. Datasets that are used to test the performance are analyzed according to the images environmental conditions, besides the dataset biasing is discussed.

Hand gesture recognition by means of region-based convolutional neural networks

2017 ◽  
Vol 10 (27) ◽  
pp. 1329-1342 ◽  
Author(s):  
Javier O. Pinzon Arenas ◽  
Robinson Jimenez Moreno ◽  
Paula C. Useche Murillo
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Gesture Recognition ◽  
Hand Gesture ◽  
Hand Gestures ◽  
The Neural Network ◽  
Average Accuracy

This paper presents the implementation of a Region-based Convolutional Neural Network focused on the recognition and localization of hand gestures, in this case 2 types of gestures: open and closed hand, in order to achieve the recognition of such gestures in dynamic backgrounds. The neural network is trained and validated, achieving a 99.4% validation accuracy in gesture recognition and a 25% average accuracy in RoI localization, which is then tested in real time, where its operation is verified through times taken for recognition, execution behavior through trained and untrained gestures, and complex backgrounds.

OPTIMIZATION PROCESS ANALYSIS FOR HYPERPARAMETERS OF NEURAL NETWORK DATA PROCESSING STRUCTURES

2020 ◽  
pp. 3-10
Author(s):  
V. N. Gridin ◽  
I. A. Evdokimov ◽  
B. R. Salem ◽  
V. I. Solodovnikov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Process Analysis ◽  
Validation Process ◽  
The Neural Network ◽  
Source Data ◽  
The Neural Networks ◽  
Qualitative Characteristics ◽  
Setting Parameters

The analysis of key stages, implementation features and functioning principles of the neural networks, including deep neural networks, has been carried out. The problems of choosing the number of hidden elements, methods for the internal topology selection and setting parameters are considered. It is shown that in the training and validation process it is possible to control the capacity of a neural network and evaluate the qualitative characteristics of the constructed model. The issues of construction processes automation and hyperparameters optimization of the neural network structures are considered depending on the user's tasks and the available source data. A number of approaches based on the use of probabilistic programming, evolutionary algorithms, and recurrent neural networks are presented.

scholarly journals Convolutional neural network for image classification based on transfer learning technique

2021 ◽  
Author(s):  
Ghassan Mohammed Halawani
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Image Classification ◽  
Transfer Learning ◽  
Learning Networks ◽  
The Neural Network ◽  
Learning Technique ◽  
Common Architecture

The main purpose of this project is to modify a convolutional neural network for image classification, based on a deep-learning framework. A transfer learning technique is used by the MATLAB interface to Alex-Net to train and modify the parameters in the last two fully connected layers of Alex-Net with a new dataset to perform classifications of thousands of images. First, the general common architecture of most neural networks and their benefits are presented. The mathematical models and the role of each part in the neural network are explained in detail. Second, different neural networks are studied in terms of architecture, application, and the working method to highlight the strengths and weaknesses of each of neural network. The final part conducts a detailed study on one of the most powerful deep-learning networks in image classification – i.e. the convolutional neural network – and how it can be modified to suit different classification tasks by using transfer learning technique in MATLAB.

Sign in / Sign up

Export Citation Format

Share Document