Convolutional Neural Networks for Challenges in Automated Nuclide Identification

Improvements in Radio-Isotope IDentification (RIID) algorithms have seen a resurgence in interest with the increased accessibility of machine learning models. Convolutional Neural Network (CNN)-based models have been developed to identify arbitrary mixtures of unstable nuclides from gamma spectra. In service of this, methods for the simulation and pre-processing of training data were also developed. The implementation of 1D multi-class, multi-label CNNs demonstrated good generalisation to real spectra with poor statistics and significant gain shifts. It is also shown that even basic CNN architectures prove reliable for RIID under the challenging conditions of heavy shielding and close source geometries, and may be extended to generalised solutions for pragmatic RIID.

Download Full-text

Building-damage detection method based on machine learning utilizing aerial photographs of the Kumamoto earthquake

Earthquake Spectra ◽

10.1177/8755293019901309 ◽

2020 ◽

Vol 36 (3) ◽

pp. 1166-1187 ◽

Cited By ~ 4

Author(s):

Shohei Naito ◽

Hiromitsu Tomozawa ◽

Yuji Mori ◽

Takeshi Nagata ◽

Naokazu Monma ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Training Data ◽

Aerial Photographs ◽

Learning Models ◽

Visual Interpretation ◽

Damage Classification ◽

Kumamoto Earthquake ◽

Machine Learning Models

This article presents a method for detecting damaged buildings in the event of an earthquake using machine learning models and aerial photographs. We initially created training data for machine learning models using aerial photographs captured around the town of Mashiki immediately after the main shock of the 2016 Kumamoto earthquake. All buildings are classified into one of the four damage levels by visual interpretation. Subsequently, two damage discrimination models are developed: a bag-of-visual-words model and a model based on a convolutional neural network. Results are compared and validated in terms of accuracy, revealing that the latter model is preferable. Moreover, for the convolutional neural network model, the target areas are expanded and the recalls of damage classification at the four levels range approximately from 66% to 81%.

Download Full-text

Early Warning System for Online STEM Learning—A Slimmer Approach Using Recurrent Neural Networks

Sustainability ◽

10.3390/su132212461 ◽

2021 ◽

Vol 13 (22) ◽

pp. 12461

Author(s):

Chih-Chang Yu ◽

Yufeng (Leon) Wu

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

At Risk ◽

At Risk Students ◽

Training Data ◽

Support Vector ◽

Learning Models ◽

Conventional Machine ◽

Machine Learning Models

While the use of deep neural networks is popular for predicting students’ learning outcomes, convolutional neural network (CNN)-based methods are used more often. Such methods require numerous features, training data, or multiple models to achieve week-by-week predictions. However, many current learning management systems (LMSs) operated by colleges cannot provide adequate information. To make the system more feasible, this article proposes a recurrent neural network (RNN)-based framework to identify at-risk students who might fail the course using only a few common learning features. RNN-based methods can be more effective than CNN-based methods in identifying at-risk students due to their ability to memorize time-series features. The data used in this study were collected from an online course that teaches artificial intelligence (AI) at a university in northern Taiwan. Common features, such as the number of logins, number of posts and number of homework assignments submitted, are considered to train the model. This study compares the prediction results of the RNN model with the following conventional machine learning models: logistic regression, support vector machines, decision trees and random forests. This work also compares the performance of the RNN model with two neural network-based models: the multi-layer perceptron (MLP) and a CNN-based model. The experimental results demonstrate that the RNN model used in this study is better than conventional machine learning models and the MLP in terms of F-score, while achieving similar performance to the CNN-based model with fewer parameters. Our study shows that the designed RNN model can identify at-risk students once one-third of the semester has passed. Some future directions are also discussed.

Download Full-text

Malware Detection on Byte Streams of PDF Files Using Convolutional Neural Networks

Security and Communication Networks ◽

10.1155/2019/8485365 ◽

2019 ◽

Vol 2019 ◽

pp. 1-9 ◽

Cited By ~ 3

Author(s):

Young-Seob Jeong ◽

Jiyoung Woo ◽

Ah Reum Kang

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Convolutional Neural Network ◽

Input Data ◽

Malware Detection ◽

Learning Models ◽

High Level ◽

The Given ◽

Machine Learning Models

With increasing amount of data, the threat of malware keeps growing recently. The malicious actions embedded in nonexecutable documents especially (e.g., PDF files) can be more dangerous, because it is difficult to detect and most users are not aware of such type of malicious attacks. In this paper, we design a convolutional neural network to tackle the malware detection on the PDF files. We collect malicious and benign PDF files and manually label the byte sequences within the files. We intensively examine the structure of the input data and illustrate how we design the proposed network based on the characteristics of data. The proposed network is designed to interpret high-level patterns among collectable spatial clues, thereby predicting whether the given byte sequence has malicious actions or not. By experimental results, we demonstrate that the proposed network outperform several representative machine-learning models as well as other networks with different settings.

Download Full-text

Convolutional Neural Network

10.4018/978-1-6684-2408-7.ch077 ◽

2022 ◽

pp. 1559-1575

Author(s):

Mário Pereira Véstias

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Artificial Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Machine Learning Algorithms ◽

Training Data ◽

Machine Learning Model ◽

Artificial Neural

Machine learning is the study of algorithms and models for computing systems to do tasks based on pattern identification and inference. When it is difficult or infeasible to develop an algorithm to do a particular task, machine learning algorithms can provide an output based on previous training data. A well-known machine learning model is deep learning. The most recent deep learning models are based on artificial neural networks (ANN). There exist several types of artificial neural networks including the feedforward neural network, the Kohonen self-organizing neural network, the recurrent neural network, the convolutional neural network, the modular neural network, among others. This article focuses on convolutional neural networks with a description of the model, the training and inference processes and its applicability. It will also give an overview of the most used CNN models and what to expect from the next generation of CNN models.

Download Full-text

Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study

Applied Sciences ◽

10.3390/app8122663 ◽

2018 ◽

Vol 8 (12) ◽

pp. 2663 ◽

Cited By ~ 11

Author(s):

Davy Preuveneers ◽

Vera Rimmer ◽

Ilias Tsingenopoulos ◽

Jan Spooren ◽

Wouter Joosen ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Intrusion Detection ◽

Anomaly Detection ◽

Training Data ◽

Learning Models ◽

Traditional System ◽

Blockchain Technology ◽

Malicious Behavior ◽

Machine Learning Models

The adoption of machine learning and deep learning is on the rise in the cybersecurity domain where these AI methods help strengthen traditional system monitoring and threat detection solutions. However, adversaries too are becoming more effective in concealing malicious behavior amongst large amounts of benign behavior data. To address the increasing time-to-detection of these stealthy attacks, interconnected and federated learning systems can improve the detection of malicious behavior by joining forces and pooling together monitoring data. The major challenge that we address in this work is that in a federated learning setup, an adversary has many more opportunities to poison one of the local machine learning models with malicious training samples, thereby influencing the outcome of the federated learning and evading detection. We present a solution where contributing parties in federated learning can be held accountable and have their model updates audited. We describe a permissioned blockchain-based federated learning method where incremental updates to an anomaly detection machine learning model are chained together on the distributed ledger. By integrating federated learning with blockchain technology, our solution supports the auditing of machine learning models without the necessity to centralize the training data. Experiments with a realistic intrusion detection use case and an autoencoder for anomaly detection illustrate that the increased complexity caused by blockchain technology has a limited performance impact on the federated learning, varying between 5 and 15%, while providing full transparency over the distributed training process of the neural network. Furthermore, our blockchain-based federated learning solution can be generalized and applied to more sophisticated neural network architectures and other use cases.

Download Full-text

A Dynamic Convolutional Neural Network Based Shared-Bike Demand Forecasting Model

ACM Transactions on Intelligent Systems and Technology ◽

10.1145/3447988 ◽

2021 ◽

Vol 12 (6) ◽

pp. 1-24

Author(s):

Shaojie Qiao ◽

Nan Han ◽

Jianbin Huang ◽

Kun Yue ◽

Rui Mao ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Prediction Accuracy ◽

Demand Forecasting ◽

Forecasting Model ◽

Learning Models ◽

Bike Sharing ◽

Demand Forecasting Model ◽

Machine Learning Models

Bike-sharing systems are becoming popular and generate a large volume of trajectory data. In a bike-sharing system, users can borrow and return bikes at different stations. In particular, a bike-sharing system will be affected by weather, the time period, and other dynamic factors, which challenges the scheduling of shared bikes. In this article, a new shared-bike demand forecasting model based on dynamic convolutional neural networks, called SDF , is proposed to predict the demand of shared bikes. SDF chooses the most relevant weather features from real weather data by using the Pearson correlation coefficient and transforms them into a two-dimensional dynamic feature matrix, taking into account the states of stations from historical data. The feature information in the matrix is extracted, learned, and trained with a newly proposed dynamic convolutional neural network to predict the demand of shared bikes in a dynamical and intelligent fashion. The phase of parameter update is optimized from three aspects: the loss function, optimization algorithm, and learning rate. Then, an accurate shared-bike demand forecasting model is designed based on the basic idea of minimizing the loss value. By comparing with classical machine learning models, the weight sharing strategy employed by SDF reduces the complexity of the network. It allows a high prediction accuracy to be achieved within a relatively short period of time. Extensive experiments are conducted on real-world bike-sharing datasets to evaluate SDF. The results show that SDF significantly outperforms classical machine learning models in prediction accuracy and efficiency.

Download Full-text

Deep learning surrogate models for spatial and visual connectivity

International Journal of Architectural Computing ◽

10.1177/1478077119894483 ◽

2019 ◽

Vol 18 (1) ◽

pp. 53-66 ◽

Cited By ~ 1

Author(s):

Sherif Tarabishy ◽

Stamatios Psarras ◽

Marcin Kosicki ◽

Martha Tsigkari

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Distributed Computation ◽

Training Data ◽

Connectivity Analysis ◽

Floor Plan ◽

Learning Models ◽

Computationally Intensive ◽

Analysis Engine ◽

Machine Learning Models

Spatial and visual connectivity are important metrics when developing workplace layouts. Calculating those metrics in real time can be difficult, depending on the size of the floor plan being analysed and the resolution of the analyses. This article investigates the possibility of considerably speeding up the outcomes of such computationally intensive simulations by using machine learning to create models capable of identifying the spatial and visual connectivity potential of a space. To that end, we present the entire process of investigating different machine learning models and a pipeline for training them on such task, from the incorporation of a bespoke spatial and visual connectivity analysis engine through a distributed computation pipeline, to the process of synthesizing training data and evaluating the performance of different neural networks.

Download Full-text

Network-guided supervised learning on gene expression using a graph convolutional neural network

10.1101/2021.12.27.474240 ◽

2021 ◽

Author(s):

Chayaporn Suphavilai ◽

Hatairat Yingtaweesittikul

Keyword(s):

Neural Network ◽

Gene Expression ◽

Machine Learning ◽

Convolutional Neural Network ◽

Protein Interaction ◽

Expression Patterns ◽

Interaction Network ◽

Gene Expression Patterns ◽

Learning Models ◽

Machine Learning Models

Background: Transcriptomic profiles have become crucial information in understanding diseases and improving treatments. While dysregulated gene sets are identified via pathway analysis, various machine learning models have been proposed for predicting phenotypes such as disease type and drug response based on gene expression patterns. However, these models still lack interpretability, as well as the ability to integrate prior knowledge from a protein-protein interaction network. Results: We propose Grandline, a graph convolutional neural network that can integrate gene expression data and structure of the protein interaction network to predict a specific phenotype. Transforming the interaction network into a spectral domain enables convolution of neighbouring genes and pinpointing high-impact subnetworks, which allow better interpretability of deep learning models. Grandline achieves high phenotype prediction accuracy (67-85% in 8 use cases), comparable to state-of-the-art machine learning models while requiring a smaller number of parameters, allowing it to learn complex but interpretable gene expression patterns from biological datasets. Conclusion: To improve the interpretability of phenotype prediction based on gene expression patterns, we developed Grandline using graph convolutional neural network technique to integrate protein interaction information. We focus on improving the ability to learn nonlinear relationships between gene expression patterns and a given phenotype and incorporation of prior knowledge, which are the main challenges of machine learning models for biological datasets. The graph convolution allows us to aggregate information from relevant genes and reduces the number of trainable parameters, facilitating model training for a small-sized biological dataset.

Download Full-text

Automated Facial Recognition for Noonan Syndrome Using Novel Deep Convolutional Neural Network With Additive Angular Margin Loss

Frontiers in Genetics ◽

10.3389/fgene.2021.669841 ◽

2021 ◽

Vol 12 ◽

Author(s):

Hang Yang ◽

Xin-Rong Hu ◽

Ling Sun ◽

Dian Hong ◽

Ying-Yi Zheng ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Convolutional Neural Network ◽

Loss Function ◽

Noonan Syndrome ◽

Facial Recognition ◽

Deep Convolutional Neural Network ◽

Healthy Children ◽

Learning Models ◽

Machine Learning Models

BackgroundNoonan syndrome (NS), a genetically heterogeneous disorder, presents with hypertelorism, ptosis, dysplastic pulmonary valve stenosis, hypertrophic cardiomyopathy, and small stature. Early detection and assessment of NS are crucial to formulating an individualized treatment protocol. However, the diagnostic rate of pediatricians and pediatric cardiologists is limited. To overcome this challenge, we propose an automated facial recognition model to identify NS using a novel deep convolutional neural network (DCNN) with a loss function called additive angular margin loss (ArcFace).MethodsThe proposed automated facial recognition models were trained on dataset that included 127 NS patients, 163 healthy children, and 130 children with several other dysmorphic syndromes. The photo dataset contained only one frontal face image from each participant. A novel DCNN framework with ArcFace loss function (DCNN-Arcface model) was constructed. Two traditional machine learning models and a DCNN model with cross-entropy loss function (DCNN-CE model) were also constructed. Transfer learning and data augmentation were applied in the training process. The identification performance of facial recognition models was assessed by five-fold cross-validation. Comparison of the DCNN-Arcface model to two traditional machine learning models, the DCNN-CE model, and six physicians were performed.ResultsAt distinguishing NS patients from healthy children, the DCNN-Arcface model achieved an accuracy of 0.9201 ± 0.0138 and an area under the receiver operator characteristic curve (AUC) of 0.9797 ± 0.0055. At distinguishing NS patients from children with several other genetic syndromes, it achieved an accuracy of 0.8171 ± 0.0074 and an AUC of 0.9274 ± 0.0062. In both cases, the DCNN-Arcface model outperformed the two traditional machine learning models, the DCNN-CE model, and six physicians.ConclusionThis study shows that the proposed DCNN-Arcface model is a promising way to screen NS patients and can improve the NS diagnosis rate.

Download Full-text

Research on the Garbage Classification Problem Based on Convolutional Neural Network

Journal of Physics Conference Series ◽

10.1088/1742-6596/2078/1/012056 ◽

2021 ◽

Vol 2078 (1) ◽

pp. 012056

Author(s):

Shuang Wu ◽

Zeyu Li ◽

Xinqiong Chen ◽

Peiwen Zhong ◽

Liangcai Mei ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Factor Analysis ◽

Convolutional Neural Network ◽

Classification Problem ◽

Field Investigation ◽

Learning Models ◽

Classification Problems ◽

Waste Classification ◽

Machine Learning Models

Abstract In order to better promote garbage classification, machine learning models are used to discover and solve garbage classification problems. First, the factor analysis is used to conduct field investigation and data analysis on residents' perception of waste classification. Second, convolutional neural network (CNN) is used to classify and recognize garbage images, which is used to assist the judgment of garbage classification. We should put forward some reasonable classification suggestions to better promote the problem of garbage classification.

Download Full-text