scholarly journals DeepView: Visualizing Classification Boundaries of Deep Neural Networks as Scatter Plots Using Discriminative Dimensionality Reduction

2020 ◽  
Author(s):  
Alexander Schulz ◽  
Fabian Hinder ◽  
Barbara Hammer
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Dimensionality Reduction ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Decision Function ◽  
Two Dimensions ◽  
Machine Learning Algorithms ◽  
Data Set ◽  
Scatter Plots

Machine learning algorithms using deep architectures have been able to implement increasingly powerful and successful models. However, they also become increasingly more complex, more difficult to comprehend and easier to fool. So far, most methods in the literature investigate the decision of the model for a single given input datum. In this paper, we propose to visualize a part of the decision function of a deep neural network together with a part of the data set in two dimensions with discriminative dimensionality reduction. This enables us to inspect how different properties of the data are treated by the model, such as outliers, adversaries or poisoned data. Further, the presented approach is complementary to the mentioned interpretation methods from the literature and hence might be even more useful in combination with those. Code is available at https://github.com/LucaHermes/DeepView

EMOTIONS RECOGNITION IN HUMAN SPEECH USING DEEP NEURAL NETWORKS

2021 ◽  
pp. 44-51
Author(s):  
E. Yu. Shchetinin
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Recurrent Neural Network ◽  
Deep Neural Networks ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Audio Recordings ◽  
Computer Studies

The recognition of human emotions is one of the most relevant and dynamically developing areas of modern speech technologies, and the recognition of emotions in speech (RER) is the most demanded part of them. In this paper, we propose a computer model of emotion recognition based on an ensemble of bidirectional recurrent neural network with LSTM memory cell and deep convolutional neural network ResNet18. In this paper, computer studies of the RAVDESS database containing emotional speech of a person are carried out. RAVDESS-a data set containing 7356 files. Entries contain the following emotions: 0 – neutral, 1 – calm, 2 – happiness, 3 – sadness, 4 – anger, 5 – fear, 6 – disgust, 7 – surprise. In total, the database contains 16 classes (8 emotions divided into male and female) for a total of 1440 samples (speech only). To train machine learning algorithms and deep neural networks to recognize emotions, existing audio recordings must be pre-processed in such a way as to extract the main characteristic features of certain emotions. This was done using Mel-frequency cepstral coefficients, chroma coefficients, as well as the characteristics of the frequency spectrum of audio recordings. In this paper, computer studies of various models of neural networks for emotion recognition are carried out on the example of the data described above. In addition, machine learning algorithms were used for comparative analysis. Thus, the following models were trained during the experiments: logistic regression (LR), classifier based on the support vector machine (SVM), decision tree (DT), random forest (RF), gradient boosting over trees – XGBoost, convolutional neural network CNN, recurrent neural network RNN (ResNet18), as well as an ensemble of convolutional and recurrent networks Stacked CNN-RNN. The results show that neural networks showed much higher accuracy in recognizing and classifying emotions than the machine learning algorithms used. Of the three neural network models presented, the CNN + BLSTM ensemble showed higher accuracy.

A comparison of deep machine learning and Monte Carlo methods for facies classification from seismic data

Geophysics ◽  
2020 ◽  
Vol 85 (4) ◽  
pp. WA41-WA52 ◽  
Author(s):  
Dario Grana ◽  
Leonardo Azevedo ◽  
Mingliang Liu
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Monte Carlo ◽  
Monte Carlo Methods ◽  
Seismic Data ◽  
Transition Probabilities ◽  
Machine Learning Algorithms ◽  
Data Set ◽  
Facies Classification ◽  
The Neural Network

Among the large variety of mathematical and computational methods for estimating reservoir properties such as facies and petrophysical variables from geophysical data, deep machine-learning algorithms have gained significant popularity for their ability to obtain accurate solutions for geophysical inverse problems in which the physical models are partially unknown. Solutions of classification and inversion problems are generally not unique, and uncertainty quantification studies are required to quantify the uncertainty in the model predictions and determine the precision of the results. Probabilistic methods, such as Monte Carlo approaches, provide a reliable approach for capturing the variability of the set of possible models that match the measured data. Here, we focused on the classification of facies from seismic data and benchmarked the performance of three different algorithms: recurrent neural network, Monte Carlo acceptance/rejection sampling, and Markov chain Monte Carlo. We tested and validated these approaches at the well locations by comparing classification predictions to the reference facies profile. The accuracy of the classification results is defined as the mismatch between the predictions and the log facies profile. Our study found that when the training data set of the neural network is large enough and the prior information about the transition probabilities of the facies in the Monte Carlo approach is not informative, machine-learning methods lead to more accurate solutions; however, the uncertainty of the solution might be underestimated. When some prior knowledge of the facies model is available, for example, from nearby wells, Monte Carlo methods provide solutions with similar accuracy to the neural network and allow a more robust quantification of the uncertainty, of the solution.

scholarly journals PROUD-MAL: static analysis-based progressive framework for deep unsupervised malware classification of windows portable executable

2021 ◽  
Author(s):  
Syed Khurram Jah Rizvi ◽  
Warda Aslam ◽  
Muhammad Shahzad ◽  
Shahzad Saleem ◽  
Muhammad Moazam Fraz
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Static Analysis ◽  
Deep Neural Network ◽  
Early Stage ◽  
Malware Detection ◽  
Machine Learning Algorithms ◽  
Organizational Network ◽  
Depth Analysis ◽  
Feature Attention

AbstractEnterprises are striving to remain protected against malware-based cyber-attacks on their infrastructure, facilities, networks and systems. Static analysis is an effective approach to detect the malware, i.e., malicious Portable Executable (PE). It performs an in-depth analysis of PE files without executing, which is highly useful to minimize the risk of malicious PE contaminating the system. Yet, instant detection using static analysis has become very difficult due to the exponential rise in volume and variety of malware. The compelling need of early stage detection of malware-based attacks significantly motivates research inclination towards automated malware detection. The recent machine learning aided malware detection approaches using static analysis are mostly supervised. Supervised malware detection using static analysis requires manual labelling and human feedback; therefore, it is less effective in rapidly evolutionary and dynamic threat space. To this end, we propose a progressive deep unsupervised framework with feature attention block for static analysis-based malware detection (PROUD-MAL). The framework is based on cascading blocks of unsupervised clustering and features attention-based deep neural network. The proposed deep neural network embedded with feature attention block is trained on the pseudo labels. To evaluate the proposed unsupervised framework, we collected a real-time malware dataset by deploying low and high interaction honeypots on an enterprise organizational network. Moreover, endpoint security solution is also deployed on an enterprise organizational network to collect malware samples. After post processing and cleaning, the novel dataset consists of 15,457 PE samples comprising 8775 malicious and 6681 benign ones. The proposed PROUD-MAL framework achieved an accuracy of more than 98.09% with better quantitative performance in standard evaluation parameters on collected dataset and outperformed other conventional machine learning algorithms. The implementation and dataset are available at https://bit.ly/35Sne3a.

Survey of Machine Learning Algorithms to Detect Malware in Consumer Internet of Things Devices

2021 ◽  
Vol 30 (04) ◽  
pp. 2150020
Author(s):  
Luke Holbrook ◽  
Miltiadis Alamaniotis
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Internet Of Things ◽  
Deep Neural Network ◽  
Learning Algorithms ◽  
Cyber Attacks ◽  
Machine Learning Algorithms ◽  
Coefficient Of Determination ◽  
Support Vector ◽  
Iot Devices

With the increase of cyber-attacks on millions of Internet of Things (IoT) devices, the poor network security measures on those devices are the main source of the problem. This article aims to study a number of these machine learning algorithms available for their effectiveness in detecting malware in consumer internet of things devices. In particular, the Support Vector Machines (SVM), Random Forest, and Deep Neural Network (DNN) algorithms are utilized for a benchmark with a set of test data and compared as tools in safeguarding the deployment for IoT security. Test results on a set of 4 IoT devices exhibited that all three tested algorithms presented here detect the network anomalies with high accuracy. However, the deep neural network provides the highest coefficient of determination R2, and hence, it is identified as the most precise among the tested algorithms concerning the security of IoT devices based on the data sets we have undertaken.

iLearn: an integrated platform and meta-learner for feature engineering, machine-learning analysis and modeling of DNA, RNA and protein sequence data

2019 ◽  
Vol 21 (3) ◽  
pp. 1047-1057 ◽  
Author(s):  
Zhen Chen ◽  
Pei Zhao ◽  
Fuyi Li ◽  
Tatiana T Marquez-Lago ◽  
André Leier ◽  
...  
Keyword(s):  
Machine Learning ◽  
Dimensionality Reduction ◽  
Sequence Data ◽  
Machine Learning Algorithms ◽  
User Friendliness ◽  
Data Set ◽  
Protein Sequence Data ◽  
Learning Analysis ◽  
High Throughput Manner ◽  
Online Web

Abstract With the explosive growth of biological sequences generated in the post-genomic era, one of the most challenging problems in bioinformatics and computational biology is to computationally characterize sequences, structures and functions in an efficient, accurate and high-throughput manner. A number of online web servers and stand-alone tools have been developed to address this to date; however, all these tools have their limitations and drawbacks in terms of their effectiveness, user-friendliness and capacity. Here, we present iLearn, a comprehensive and versatile Python-based toolkit, integrating the functionality of feature extraction, clustering, normalization, selection, dimensionality reduction, predictor construction, best descriptor/model selection, ensemble learning and results visualization for DNA, RNA and protein sequences. iLearn was designed for users that only want to upload their data set and select the functions they need calculated from it, while all necessary procedures and optimal settings are completed automatically by the software. iLearn includes a variety of descriptors for DNA, RNA and proteins, and four feature output formats are supported so as to facilitate direct output usage or communication with other computational tools. In total, iLearn encompasses 16 different types of feature clustering, selection, normalization and dimensionality reduction algorithms, and five commonly used machine-learning algorithms, thereby greatly facilitating feature analysis and predictor construction. iLearn is made freely available via an online web server and a stand-alone toolkit.

scholarly journals Machine learning analysis to predict the need for ankle foot orthosis in patients with stroke

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yoo Jin Choo ◽  
Jeoung Kun Kim ◽  
Jang Hwan Kim ◽  
Min Cheol Chang ◽  
Donghwi Park
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Neural Network ◽  
Machine Learning Algorithms ◽  
Shoulder Abduction ◽  
Stroke Patients ◽  
Ankle Foot Orthosis ◽  
Affected Side ◽  
Foot Orthosis ◽  
Ankle Dorsiflexor

AbstractWe investigated the potential of machine learning techniques, at an early stage after stroke, to predict the need for ankle–foot orthosis (AFO) in stroke patients. We retrospectively recruited 474 consecutive stroke patients. The need for AFO during ambulation (output variable) was classified according to the Medical Research Council (MRC) score for the ankle dorsiflexor of the affected limb. Patients with an MRC score of < 3 for the ankle dorsiflexor of the affected side were considered to require AFO, while those with scores ≥ 3 were considered not to require AFO. The following demographic and clinical data collected when patients were transferred to the rehabilitation unit (16.20 ± 6.02 days) and 6 months after stroke onset were used as input data: age, sex, type of stroke (ischemic/hemorrhagic), motor evoked potential data on the tibialis anterior muscle of the affected side, modified Brunnstrom classification, functional ambulation category, MRC score for muscle strength for shoulder abduction, elbow flexion, finger flexion, finger extension, hip flexion, knee extension, and ankle dorsiflexion of the affected side. For the deep neural network model, the area under the curve (AUC) was 0.887. For the random forest and logistic regression models, the AUC was 0.855 and 0.845, respectively. Our findings demonstrate that machine learning algorithms, particularly the deep neural network, are useful for predicting the need for AFO in stroke patients during the recovery phase.

scholarly journals Deep Neural Network for Multi-Class Prediction of Student Performance in Educational Data

2019 ◽  
Vol 8 (2) ◽  
pp. 5073-5081
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Student Performance ◽  
Activation Function ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Fine Tuning ◽  
Academic Excellence ◽  
Data Set

Prediction of student performance is the significant part in processing the educational data. Machine learning algorithms are leading the role in this process. Deep learning is one of the important concepts of machine learning algorithm. In this paper, we applied the deep learning technique for prediction of the academic excellence of the students using R Programming. Keras and Tensorflow libraries utilized for making the model using neural network on the Kaggle dataset. The data is separated into testing data training data set. Plot the neural network model using neuralnet method and created the Deep Learning model using two hidden layers using ReLu activation function and one output layer using softmax activation function. After fine tuning process until the stable changes; this model produced accuracy as 85%.

scholarly journals Micro Clustering Methodology for Document Objects using Deep Learning Techniques

2019 ◽  
Vol 9 (2) ◽  
pp. 1546-1552
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Large Data ◽  
Heterogeneous Data ◽  
Machine Learning Algorithms ◽  
Volume Data ◽  
Data Set ◽  
Clustering And Classification

Large data clustering and classification is a very challenging task in data mining. Various machine learning and deep learning systems have been proposed by many researchers on a different dataset. Data volume, data size and structure of data may affect the time complexity of the system. This paper described a new document object classification approach using deep learning (DL) and proposed a recurrent neural network (RNN) for classification with a micro-clustering approach.TF-IDF and a density-based approach are used to store the best features. The plane work used supervised learning method and it extracts features set called as BK of the desired classes. once the training part completed then proceeds to figure out the particular test instances with the help of the planned classification algorithm. Recurrent Neural Network categorized the particular test object according to their weights. The system can able to work on heterogeneous data set and generate the micro-clusters according to classified results. The system also carried out experimental analysis with classical machine learning algorithms. The proposed algorithm shows higher accuracy than the existing density-based approach on different data sets.

scholarly journals Result and Performance Analysis of Rainfall Prediction System Based on Deep Neural Network

2020 ◽  
pp. 633-638
Author(s):  
Akshay Rajendra Naik ◽  
A. V. Deorankar ◽  
P. B. Ambhore
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Neural Network ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Learning Methods ◽  
Rainfall Prediction ◽  
Machine Learning Methods ◽  
Vector Machines ◽  
And Performance

Rainfall prediction is useful for all people for decision making in all fields, such as out door gamming, farming, traveling, and factory and for other activities. We studied various methods for rainfall prediction such as machine learning and neural networks. There is various machine learning algorithms are used in previous existing methods such as naïve byes, support vector machines, random forest, decision trees, and ensemble learning methods. We used deep neural network for rainfall prediction, and for optimization of deep neural network Adam optimizer is used for setting modal parameters, as a result our method gives better results as compare to other machine learning methods.

Sign in / Sign up

Export Citation Format

Share Document