Malware Detection on Byte Streams of PDF Files Using Convolutional Neural Networks

With increasing amount of data, the threat of malware keeps growing recently. The malicious actions embedded in nonexecutable documents especially (e.g., PDF files) can be more dangerous, because it is difficult to detect and most users are not aware of such type of malicious attacks. In this paper, we design a convolutional neural network to tackle the malware detection on the PDF files. We collect malicious and benign PDF files and manually label the byte sequences within the files. We intensively examine the structure of the input data and illustrate how we design the proposed network based on the characteristics of data. The proposed network is designed to interpret high-level patterns among collectable spatial clues, thereby predicting whether the given byte sequence has malicious actions or not. By experimental results, we demonstrate that the proposed network outperform several representative machine-learning models as well as other networks with different settings.

Download Full-text

Convolutional Neural Networks for Challenges in Automated Nuclide Identification

Sensors ◽

10.3390/s21155238 ◽

2021 ◽

Vol 21 (15) ◽

pp. 5238

Author(s):

Anthony N. Turner ◽

Carl Wheldon ◽

Tzany Kokalova Wheldon ◽

Mark R. Gilbert ◽

Lee W. Packer ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Convolutional Neural Network ◽

Training Data ◽

Significant Gain ◽

Learning Models ◽

Isotope Identification ◽

Close Source ◽

Machine Learning Models

Improvements in Radio-Isotope IDentification (RIID) algorithms have seen a resurgence in interest with the increased accessibility of machine learning models. Convolutional Neural Network (CNN)-based models have been developed to identify arbitrary mixtures of unstable nuclides from gamma spectra. In service of this, methods for the simulation and pre-processing of training data were also developed. The implementation of 1D multi-class, multi-label CNNs demonstrated good generalisation to real spectra with poor statistics and significant gain shifts. It is also shown that even basic CNN architectures prove reliable for RIID under the challenging conditions of heavy shielding and close source geometries, and may be extended to generalised solutions for pragmatic RIID.

Download Full-text

Building-damage detection method based on machine learning utilizing aerial photographs of the Kumamoto earthquake

Earthquake Spectra ◽

10.1177/8755293019901309 ◽

2020 ◽

Vol 36 (3) ◽

pp. 1166-1187 ◽

Cited By ~ 4

Author(s):

Shohei Naito ◽

Hiromitsu Tomozawa ◽

Yuji Mori ◽

Takeshi Nagata ◽

Naokazu Monma ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Training Data ◽

Aerial Photographs ◽

Learning Models ◽

Visual Interpretation ◽

Damage Classification ◽

Kumamoto Earthquake ◽

Machine Learning Models

This article presents a method for detecting damaged buildings in the event of an earthquake using machine learning models and aerial photographs. We initially created training data for machine learning models using aerial photographs captured around the town of Mashiki immediately after the main shock of the 2016 Kumamoto earthquake. All buildings are classified into one of the four damage levels by visual interpretation. Subsequently, two damage discrimination models are developed: a bag-of-visual-words model and a model based on a convolutional neural network. Results are compared and validated in terms of accuracy, revealing that the latter model is preferable. Moreover, for the convolutional neural network model, the target areas are expanded and the recalls of damage classification at the four levels range approximately from 66% to 81%.

Download Full-text

A Dynamic Convolutional Neural Network Based Shared-Bike Demand Forecasting Model

ACM Transactions on Intelligent Systems and Technology ◽

10.1145/3447988 ◽

2021 ◽

Vol 12 (6) ◽

pp. 1-24

Author(s):

Shaojie Qiao ◽

Nan Han ◽

Jianbin Huang ◽

Kun Yue ◽

Rui Mao ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Prediction Accuracy ◽

Demand Forecasting ◽

Forecasting Model ◽

Learning Models ◽

Bike Sharing ◽

Demand Forecasting Model ◽

Machine Learning Models

Bike-sharing systems are becoming popular and generate a large volume of trajectory data. In a bike-sharing system, users can borrow and return bikes at different stations. In particular, a bike-sharing system will be affected by weather, the time period, and other dynamic factors, which challenges the scheduling of shared bikes. In this article, a new shared-bike demand forecasting model based on dynamic convolutional neural networks, called SDF , is proposed to predict the demand of shared bikes. SDF chooses the most relevant weather features from real weather data by using the Pearson correlation coefficient and transforms them into a two-dimensional dynamic feature matrix, taking into account the states of stations from historical data. The feature information in the matrix is extracted, learned, and trained with a newly proposed dynamic convolutional neural network to predict the demand of shared bikes in a dynamical and intelligent fashion. The phase of parameter update is optimized from three aspects: the loss function, optimization algorithm, and learning rate. Then, an accurate shared-bike demand forecasting model is designed based on the basic idea of minimizing the loss value. By comparing with classical machine learning models, the weight sharing strategy employed by SDF reduces the complexity of the network. It allows a high prediction accuracy to be achieved within a relatively short period of time. Extensive experiments are conducted on real-world bike-sharing datasets to evaluate SDF. The results show that SDF significantly outperforms classical machine learning models in prediction accuracy and efficiency.

Download Full-text

Network-guided supervised learning on gene expression using a graph convolutional neural network

10.1101/2021.12.27.474240 ◽

2021 ◽

Author(s):

Chayaporn Suphavilai ◽

Hatairat Yingtaweesittikul

Keyword(s):

Neural Network ◽

Gene Expression ◽

Machine Learning ◽

Convolutional Neural Network ◽

Protein Interaction ◽

Expression Patterns ◽

Interaction Network ◽

Gene Expression Patterns ◽

Learning Models ◽

Machine Learning Models

Background: Transcriptomic profiles have become crucial information in understanding diseases and improving treatments. While dysregulated gene sets are identified via pathway analysis, various machine learning models have been proposed for predicting phenotypes such as disease type and drug response based on gene expression patterns. However, these models still lack interpretability, as well as the ability to integrate prior knowledge from a protein-protein interaction network. Results: We propose Grandline, a graph convolutional neural network that can integrate gene expression data and structure of the protein interaction network to predict a specific phenotype. Transforming the interaction network into a spectral domain enables convolution of neighbouring genes and pinpointing high-impact subnetworks, which allow better interpretability of deep learning models. Grandline achieves high phenotype prediction accuracy (67-85% in 8 use cases), comparable to state-of-the-art machine learning models while requiring a smaller number of parameters, allowing it to learn complex but interpretable gene expression patterns from biological datasets. Conclusion: To improve the interpretability of phenotype prediction based on gene expression patterns, we developed Grandline using graph convolutional neural network technique to integrate protein interaction information. We focus on improving the ability to learn nonlinear relationships between gene expression patterns and a given phenotype and incorporation of prior knowledge, which are the main challenges of machine learning models for biological datasets. The graph convolution allows us to aggregate information from relevant genes and reduces the number of trainable parameters, facilitating model training for a small-sized biological dataset.

Download Full-text

Automated Facial Recognition for Noonan Syndrome Using Novel Deep Convolutional Neural Network With Additive Angular Margin Loss

Frontiers in Genetics ◽

10.3389/fgene.2021.669841 ◽

2021 ◽

Vol 12 ◽

Author(s):

Hang Yang ◽

Xin-Rong Hu ◽

Ling Sun ◽

Dian Hong ◽

Ying-Yi Zheng ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Loss Function ◽

Noonan Syndrome ◽

Facial Recognition ◽

Deep Convolutional Neural Network ◽

Healthy Children ◽

Learning Models ◽

Machine Learning Models

BackgroundNoonan syndrome (NS), a genetically heterogeneous disorder, presents with hypertelorism, ptosis, dysplastic pulmonary valve stenosis, hypertrophic cardiomyopathy, and small stature. Early detection and assessment of NS are crucial to formulating an individualized treatment protocol. However, the diagnostic rate of pediatricians and pediatric cardiologists is limited. To overcome this challenge, we propose an automated facial recognition model to identify NS using a novel deep convolutional neural network (DCNN) with a loss function called additive angular margin loss (ArcFace).MethodsThe proposed automated facial recognition models were trained on dataset that included 127 NS patients, 163 healthy children, and 130 children with several other dysmorphic syndromes. The photo dataset contained only one frontal face image from each participant. A novel DCNN framework with ArcFace loss function (DCNN-Arcface model) was constructed. Two traditional machine learning models and a DCNN model with cross-entropy loss function (DCNN-CE model) were also constructed. Transfer learning and data augmentation were applied in the training process. The identification performance of facial recognition models was assessed by five-fold cross-validation. Comparison of the DCNN-Arcface model to two traditional machine learning models, the DCNN-CE model, and six physicians were performed.ResultsAt distinguishing NS patients from healthy children, the DCNN-Arcface model achieved an accuracy of 0.9201 ± 0.0138 and an area under the receiver operator characteristic curve (AUC) of 0.9797 ± 0.0055. At distinguishing NS patients from children with several other genetic syndromes, it achieved an accuracy of 0.8171 ± 0.0074 and an AUC of 0.9274 ± 0.0062. In both cases, the DCNN-Arcface model outperformed the two traditional machine learning models, the DCNN-CE model, and six physicians.ConclusionThis study shows that the proposed DCNN-Arcface model is a promising way to screen NS patients and can improve the NS diagnosis rate.

Download Full-text

Early Warning System for Online STEM Learning—A Slimmer Approach Using Recurrent Neural Networks

Sustainability ◽

10.3390/su132212461 ◽

2021 ◽

Vol 13 (22) ◽

pp. 12461

Author(s):

Chih-Chang Yu ◽

Yufeng (Leon) Wu

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

At Risk ◽

At Risk Students ◽

Training Data ◽

Support Vector ◽

Learning Models ◽

Conventional Machine ◽

Machine Learning Models

While the use of deep neural networks is popular for predicting students’ learning outcomes, convolutional neural network (CNN)-based methods are used more often. Such methods require numerous features, training data, or multiple models to achieve week-by-week predictions. However, many current learning management systems (LMSs) operated by colleges cannot provide adequate information. To make the system more feasible, this article proposes a recurrent neural network (RNN)-based framework to identify at-risk students who might fail the course using only a few common learning features. RNN-based methods can be more effective than CNN-based methods in identifying at-risk students due to their ability to memorize time-series features. The data used in this study were collected from an online course that teaches artificial intelligence (AI) at a university in northern Taiwan. Common features, such as the number of logins, number of posts and number of homework assignments submitted, are considered to train the model. This study compares the prediction results of the RNN model with the following conventional machine learning models: logistic regression, support vector machines, decision trees and random forests. This work also compares the performance of the RNN model with two neural network-based models: the multi-layer perceptron (MLP) and a CNN-based model. The experimental results demonstrate that the RNN model used in this study is better than conventional machine learning models and the MLP in terms of F-score, while achieving similar performance to the CNN-based model with fewer parameters. Our study shows that the designed RNN model can identify at-risk students once one-third of the semester has passed. Some future directions are also discussed.

Download Full-text

Research on the Garbage Classification Problem Based on Convolutional Neural Network

Journal of Physics Conference Series ◽

10.1088/1742-6596/2078/1/012056 ◽

2021 ◽

Vol 2078 (1) ◽

pp. 012056

Author(s):

Shuang Wu ◽

Zeyu Li ◽

Xinqiong Chen ◽

Peiwen Zhong ◽

Liangcai Mei ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Factor Analysis ◽

Convolutional Neural Network ◽

Classification Problem ◽

Field Investigation ◽

Learning Models ◽

Classification Problems ◽

Waste Classification ◽

Machine Learning Models

Abstract In order to better promote garbage classification, machine learning models are used to discover and solve garbage classification problems. First, the factor analysis is used to conduct field investigation and data analysis on residents' perception of waste classification. Second, convolutional neural network (CNN) is used to classify and recognize garbage images, which is used to assist the judgment of garbage classification. We should put forward some reasonable classification suggestions to better promote the problem of garbage classification.

Download Full-text

Privacy-Preserving Federated Neural Network Learning for Disease-Associated Cell Classification

10.1101/2022.01.10.475610 ◽

2022 ◽

Author(s):

Sinem Sav ◽

Jean-Philippe Bossuat ◽

Juan R. Troncoso-Pastoriza ◽

Manfred Claassen ◽

Jean-Pierre Hubaux

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Network Architecture ◽

Homomorphic Encryption ◽

Privacy Preserving ◽

Model Parameters ◽

Patient Privacy ◽

Learning Models ◽

Machine Learning Models

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data-silos. Sharing or centralizing the data of different healthcare institutions is, however, unfeasible or prohibitively difficult due to privacy regulations. In this work, we address this problem by using a novel privacy-preserving federated learning-based approach, PriCell, for complex machine learning models such as convolutional neural networks. PriCell relies on multiparty homomorphic encryption and enables the collaborative training of encrypted neural networks with multiple healthcare institutions. We preserve the confidentiality of each institutions' input data, of any intermediate values, and of the trained model parameters. We efficiently replicate the training of a published state-of-the-art convolutional neural network architecture in a decentralized and privacy-preserving manner. Our solution achieves an accuracy comparable to the one obtained with the centralized solution, with an improvement of at least one-order-of-magnitude in execution time with respect to prior secure solutions. Our work guarantees patient privacy and ensures data utility for efficient multi-center studies involving complex healthcare data.

Download Full-text

414 Deep Neural Networks: A Survey Tool for Obstructive Sleep Apnea Prediction

SLEEP ◽

10.1093/sleep/zsab072.413 ◽

2021 ◽

Vol 44 (Supplement_2) ◽

pp. A164-A164

Author(s):

Pahnwat Taweesedt ◽

JungYoon Kim ◽

Jaehyun Park ◽

Jangwoon Park ◽

Munish Sharma ◽

...

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Obstructive Sleep Apnea ◽

Sleep Apnea ◽

Deep Neural Networks ◽

Support Vector ◽

Learning Models ◽

Obstructive Sleep ◽

Screening Questionnaires ◽

Machine Learning Models

Abstract Introduction Obstructive sleep apnea (OSA) is a common sleep-related breathing disorder with an estimation of one billion people. Full-night polysomnography is considered the gold standard for OSA diagnosis. However, it is time-consuming, expensive and is not readily available in many parts of the world. Many screening questionnaires and scores have been proposed for OSA prediction with high sensitivity and low specificity. The present study is intended to develop models with various machine learning techniques to predict the severity of OSA by incorporating features from multiple questionnaires. Methods Subjects who underwent full-night polysomnography in Torr sleep center, Texas and completed 5 OSA screening questionnaires/scores were included. OSA was diagnosed by using Apnea-Hypopnea Index ≥ 5. We trained five different machine learning models including Deep Neural Networks with the scaled principal component analysis (DNN-PCA), Random Forest (RF), Adaptive Boosting classifier (ABC), and K-Nearest Neighbors classifier (KNC) and Support Vector Machine Classifier (SVMC). Training:Testing subject ratio of 65:35 was used. All features including demographic data, body measurement, snoring and sleepiness history were obtained from 5 OSA screening questionnaires/scores (STOP-BANG questionnaires, Berlin questionnaires, NoSAS score, NAMES score and No-Apnea score). Performance parametrics were used to compare between machine learning models. Results Of 180 subjects, 51.5 % of subjects were male with mean (SD) age of 53.6 (15.1). One hundred and nineteen subjects were diagnosed with OSA. Area Under the Receiver Operating Characteristic Curve (AUROC) of DNN-PCA, RF, ABC, KNC, SVMC, STOP-BANG questionnaire, Berlin questionnaire, NoSAS score, NAMES score, and No-Apnea score were 0.85, 0.68, 0.52, 0.74, 0.75, 0.61, 0.63, 0,61, 0.58 and 0,58 respectively. DNN-PCA showed the highest AUROC with sensitivity of 0.79, specificity of 0.67, positive-predictivity of 0.93, F1 score of 0.86, and accuracy of 0.77. Conclusion Our result showed that DNN-PCA outperforms OSA screening questionnaires, scores and other machine learning models. Support (if any):

Download Full-text

Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals

Sensors ◽

10.3390/s19010210 ◽

2019 ◽

Vol 19 (1) ◽

pp. 210 ◽

Cited By ~ 32

Author(s):

Zied Tayeb ◽

Juri Fedjaev ◽

Nejla Ghaboosi ◽

Christoph Richter ◽

Lukas Everding ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Deep Learning ◽

Convolutional Neural Network ◽

Motor Imagery ◽

Classification Performance ◽

Feature Engineering ◽

Learning Models ◽

Eeg Signals ◽

Learning Methods

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g., hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause by the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: (1) A long short-term memory (LSTM); (2) a spectrogram-based convolutional neural network model (CNN); and (3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (any manual) feature engineering. Results were evaluated on our own publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from “BCI Competition IV”. Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

Download Full-text