Spatially Disaggregated Car Ownership Prediction Using Deep Neural Networks

Predicting car ownership patterns at high spatial resolution is key to understanding pathways for decarbonisation—via electrification and demand reduction—of the private vehicle fleet. As the factors widely understood to influence car ownership are highly interdependent, linearised regression models, which dominate previous work on spatially explicit car ownership modelling in the UK, have shortcomings in accurately predicting the relationship. This paper presents predictions of spatially disaggregated car ownership—and change in car ownership over time—in Great Britain (GB) using deep neural networks (NNs) with hyperparameter tuning. The inputs to the models are demographic, socio-economic and geographic datasets compiled at the level of Census Lower Super Output Areas (LSOAs)—areas covering between 300 and 600 households. It was found that when optimal hyperparameters are selected, these neural networks can predict car ownership with a mean absolute error of up to 29% lower than when formulating the same problem as a linear regression; the results from NN regression are also shown to outperform three other artificial intelligence (AI)-based methods: random forest, stochastic gradient descent and support vector regression. The methods presented in this paper could enhance the capability of transport/energy modelling frameworks in predicting the spatial distribution of vehicle fleets, particularly as demographics, socio-economics and the built environment—such as public transport availability and the provision of local amenities—evolve over time. A particularly relevant contribution of this method is that by coupling it with a technology dissipation model, it could be used to explore the possible effects of changing policy, behaviour and socio-economics on uptake pathways for electric vehicles —cited as a vital technology for meeting Net Zero greenhouse gas emissions by 2050.

Download Full-text

414 Deep Neural Networks: A Survey Tool for Obstructive Sleep Apnea Prediction

SLEEP ◽

10.1093/sleep/zsab072.413 ◽

2021 ◽

Vol 44 (Supplement_2) ◽

pp. A164-A164

Author(s):

Pahnwat Taweesedt ◽

JungYoon Kim ◽

Jaehyun Park ◽

Jangwoon Park ◽

Munish Sharma ◽

...

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Obstructive Sleep Apnea ◽

Sleep Apnea ◽

Deep Neural Networks ◽

Support Vector ◽

Learning Models ◽

Obstructive Sleep ◽

Screening Questionnaires ◽

Machine Learning Models

Abstract Introduction Obstructive sleep apnea (OSA) is a common sleep-related breathing disorder with an estimation of one billion people. Full-night polysomnography is considered the gold standard for OSA diagnosis. However, it is time-consuming, expensive and is not readily available in many parts of the world. Many screening questionnaires and scores have been proposed for OSA prediction with high sensitivity and low specificity. The present study is intended to develop models with various machine learning techniques to predict the severity of OSA by incorporating features from multiple questionnaires. Methods Subjects who underwent full-night polysomnography in Torr sleep center, Texas and completed 5 OSA screening questionnaires/scores were included. OSA was diagnosed by using Apnea-Hypopnea Index ≥ 5. We trained five different machine learning models including Deep Neural Networks with the scaled principal component analysis (DNN-PCA), Random Forest (RF), Adaptive Boosting classifier (ABC), and K-Nearest Neighbors classifier (KNC) and Support Vector Machine Classifier (SVMC). Training:Testing subject ratio of 65:35 was used. All features including demographic data, body measurement, snoring and sleepiness history were obtained from 5 OSA screening questionnaires/scores (STOP-BANG questionnaires, Berlin questionnaires, NoSAS score, NAMES score and No-Apnea score). Performance parametrics were used to compare between machine learning models. Results Of 180 subjects, 51.5 % of subjects were male with mean (SD) age of 53.6 (15.1). One hundred and nineteen subjects were diagnosed with OSA. Area Under the Receiver Operating Characteristic Curve (AUROC) of DNN-PCA, RF, ABC, KNC, SVMC, STOP-BANG questionnaire, Berlin questionnaire, NoSAS score, NAMES score, and No-Apnea score were 0.85, 0.68, 0.52, 0.74, 0.75, 0.61, 0.63, 0,61, 0.58 and 0,58 respectively. DNN-PCA showed the highest AUROC with sensitivity of 0.79, specificity of 0.67, positive-predictivity of 0.93, F1 score of 0.86, and accuracy of 0.77. Conclusion Our result showed that DNN-PCA outperforms OSA screening questionnaires, scores and other machine learning models. Support (if any):

Download Full-text

A Diffusion Approximation Theory of Momentum Stochastic Gradient Descent in Nonconvex Optimization

Stochastic Systems ◽

10.1287/stsy.2021.0083 ◽

2021 ◽

Author(s):

Tianyi Liu ◽

Zhehui Chen ◽

Enlu Zhou ◽

Tuo Zhao

Keyword(s):

Neural Networks ◽

Nonconvex Optimization ◽

Gradient Descent ◽

Deep Neural Networks ◽

Optimization Problems ◽

Saddle Points ◽

Stochastic Gradient ◽

Stochastic Gradient Descent ◽

Nonconvex Optimization Problems ◽

Empirical Success

Momentum stochastic gradient descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning (e.g., training deep neural networks, variational Bayesian inference, etc.). Despite its empirical success, there is still a lack of theoretical understanding of convergence properties of MSGD. To fill this gap, we propose to analyze the algorithmic behavior of MSGD by diffusion approximations for nonconvex optimization problems with strict saddle points and isolated local optima. Our study shows that the momentum helps escape from saddle points but hurts the convergence within the neighborhood of optima (if without the step size annealing or momentum annealing). Our theoretical discovery partially corroborates the empirical success of MSGD in training deep neural networks.

Download Full-text

Bidirectional deep neural networks to integrate RNA and DNA data for predicting outcome for patients with hepatocellular carcinoma

Future Oncology ◽

10.2217/fon-2021-0659 ◽

2021 ◽

Author(s):

Guojun Huang ◽

Cheng Wang ◽

Xi Fu

Keyword(s):

Hepatocellular Carcinoma ◽

Neural Networks ◽

Deep Neural Networks ◽

Support Vector ◽

Machine Model ◽

Mrna Expression Data ◽

Machine Learning Approach ◽

Patient Profiling ◽

Predicting Outcome ◽

Subgroup Classification

Aims: Individualized patient profiling is instrumental for personalized management in hepatocellular carcinoma (HCC). This study built a model based on bidirectional deep neural networks (BiDNNs), an unsupervised machine-learning approach, to integrate multi-omics data and predict survival in HCC. Methods: DNA methylation and mRNA expression data for HCC samples from the TCGA database were integrated using BiDNNs. With optimal clusters as labels, a support vector machine model was developed to predict survival. Results: Using the BiDNN-based model, samples were clustered into two survival subgroups. The survival subgroup classification was an independent prognostic factor. BiDNNs were superior to multimodal autoencoders. Conclusion: This study constructed and validated a BiDNN-based model for predicting prognosis in HCC, with implications for individualized therapies in HCC.

Download Full-text

Ship-Iceberg Classification in SAR and Multispectral Satellite Images with Neural Networks

Remote Sensing ◽

10.3390/rs12152353 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2353

Author(s):

Henning Heiselberg

Keyword(s):

Neural Networks ◽

Satellite Images ◽

Deep Neural Networks ◽

Data Augmentation ◽

Neural Nets ◽

The Arctic ◽

Support Vector ◽

Sar Images ◽

Multispectral Satellite Images ◽

The Cost

Classification of ships and icebergs in the Arctic in satellite images is an important problem. We study how to train deep neural networks for improving the discrimination of ships and icebergs in multispectral satellite images. We also analyze synthetic-aperture radar (SAR) images for comparison. The annotated datasets of ships and icebergs are collected from multispectral Sentinel-2 data and taken from the C-CORE dataset of Sentinel-1 SAR images. Convolutional Neural Networks with a range of hyperparameters are tested and optimized. Classification accuracies are considerably better for deep neural networks than for support vector machines. Deeper neural nets improve the accuracy per epoch but at the cost of longer processing time. Extending the datasets with semi-supervised data from Greenland improves the accuracy considerably whereas data augmentation by rotating and flipping the images has little effect. The resulting classification accuracies for ships and icebergs are 86% for the SAR data and 96% for the MSI data due to the better resolution and more multispectral bands. The size and quality of the datasets are essential for training the deep neural networks, and methods to improve them are discussed. The reduced false alarm rates and exploitation of multisensory data are important for Arctic search and rescue services.

Download Full-text

THE USE OF CONTROL THEORY METHODS IN TRAINING NEURAL NETWORKS ON THE EXAMPLE OF TEETH RECOGNITION ON PANORAMIC X-RAY IMAGES

Automation technological and business processes ◽

10.15673/atbp.v13i2.2055 ◽

2021 ◽

Vol 13 (2) ◽

pp. 36-40

Author(s):

A. Smorodin

Keyword(s):

Neural Networks ◽

Control Theory ◽

Gradient Descent ◽

Deep Neural Networks ◽

Discrete Dynamical System ◽

Stochastic Gradient Descent ◽

Network Training ◽

Panoramic Images ◽

Important Field ◽

New Algorithms

The article investigated a modification of stochastic gradient descent (SGD), based on the previously developed stabilization theory of discrete dynamical system cycles. Relation between stabilization of cycles in discrete dynamical systems and finding extremum points allowed us to apply new control methods to accelerate gradient descent when approaching local minima. Gradient descent is often used in training deep neural networks on a par with other iterative methods. Two gradient SGD and Adam were experimented, and we conducted comparative experiments. All experiments were conducted during solving a practical problem of teeth recognition on 2-D panoramic images. Network training showed that the new method outperforms the SGD in its capabilities and as for parameters chosen it approaches the capabilities of Adam, which is a “state of the art” method. Thus, practical utility of using control theory in the training of deep neural networks and possibility of expanding its applicability in the process of creating new algorithms in this important field are shown.

Download Full-text

OPTIMIZED DEEP NEURAL NETWORKS ARCHITECTURE MODEL FOR BREAST CANCER DIAGNOSIS

YMER Digital ◽

10.37896/ymer20.11/15 ◽

2021 ◽

Vol 20 (11) ◽

pp. 161-175

Author(s):

G Kanimozhi ◽

◽

P Shanmugavadivu ◽

Keyword(s):

Breast Cancer ◽

Neural Networks ◽

Cancer Detection ◽

Deep Neural Networks ◽

Medical Image Analysis ◽

Optimization Techniques ◽

Breast Cancer Detection ◽

Automated System ◽

Stochastic Gradient Descent ◽

Architecture Model

Breast cancer has increasingly claimed the lives of women. Oncologists use digital mammograms as a viable source to detect breast cancer and classify it into benign and malignant based on the severity. The performance of the traditional methods on breast cancer detection could not be improved beyond a certain point due to the limitations and scope of computing. Moreover, the constrained scope of image processing techniques in developing automated breast cancer detection systems has motivated the researchers to shift their focus towards Artificial Intelligence based models. The Neural Networks (NN) have exhibited greater scope for the development of automated medical image analysis systems with the highest degree of accuracy. As NN model enables the automated system to understand the feature of problem-solving without being explicitly programmed. The optimization for NN offers an additional payoff on accuracy, computational complexity, and time. As the scope and suitability of optimization methods are data-dependent, the choice of selection of an appropriate optimization method itself is emerging as a prominent domain of research. In this paper, Deep Neural Networks (DNN) with different optimizers and Learning rates were designed for the prediction of breast cancer and its classification. Comparative performance analysis of five distinct first-order gradient-based optimization techniques, namely, Adaptive Gradient (Adagrad), Root Mean Square Propagation (RMSProp), Adaptive Delta (Adadelta), Adaptive Moment Estimation (Adam), and Stochastic Gradient Descent (SGD), is carried out to make predictions on the classification of breast cancer masses. For this purpose, the Mammographic Mass dataset was chosen for experimentation. The parameters determined for experiments were chosen on the number of hidden layers and learning rate along with hyperparameter tuning. The impacts of those optimizers were tested on the NN with One Hidden Layer (NN1HL), DNN with Three Hidden Layers (DNN4HL), and DNN with Eight Hidden Layers (DNN8HL). The experimental results showed that DNN8HL-Adam (DNN8HL-AM) had produced the highest accuracy of 91% among its counterparts. This research endorsed that the incorporation of optimizers in DNN contributes to an increased accuracy and optimized architecture for automated system development using neural networks.

Download Full-text

Semi-Natural and Spontaneous Speech Recognition Using Deep Neural Networks with Hybrid Features Unification

Processes ◽

10.3390/pr9122286 ◽

2021 ◽

Vol 9 (12) ◽

pp. 2286

Author(s):

Ammar Amjad ◽

Lal Khan ◽

Hsien-Tsung Chang

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Feature Fusion ◽

Spontaneous Speech ◽

Identification Accuracy ◽

Independent Experiment ◽

Support Vector ◽

Acoustic Feature ◽

Hybrid Features ◽

Feature Sets

Recently, identifying speech emotions in a spontaneous database has been a complex and demanding study area. This research presents an entirely new approach for recognizing semi-natural and spontaneous speech emotions with multiple feature fusion and deep neural networks (DNN). A proposed framework extracts the most discriminative features from hybrid acoustic feature sets. However, these feature sets may contain duplicate and irrelevant information, leading to inadequate emotional identification. Therefore, an support vector machine (SVM) algorithm is utilized to identify the most discriminative audio feature map after obtaining the relevant features learned by the fusion approach. We investigated our approach utilizing the eNTERFACE05 and BAUM-1s benchmark databases and observed a significant identification accuracy of 76% for a speaker-independent experiment with SVM and 59% accuracy with, respectively. Furthermore, experiments on the eNTERFACE05 and BAUM-1s dataset indicate that the suggested framework outperformed current state-of-the-art techniques on the semi-natural and spontaneous datasets.

Download Full-text

Modular Dynamic Neural Network: A Continual Learning Architecture

Applied Sciences ◽

10.3390/app112412078 ◽

2021 ◽

Vol 11 (24) ◽

pp. 12078

Author(s):

Daniel Turner ◽

Pedro J. S. Cardoso ◽

João M. F. Rodrigues

Keyword(s):

Neural Network ◽

Neural Networks ◽

Feature Extraction ◽

Deep Neural Networks ◽

State Of The Art ◽

Simple Task ◽

Dynamic Neural Network ◽

Main Components ◽

Over Time ◽

Continual Learning

Learning to recognize a new object after having learned to recognize other objects may be a simple task for a human, but not for machines. The present go-to approaches for teaching a machine to recognize a set of objects are based on the use of deep neural networks (DNN). So, intuitively, the solution for teaching new objects on the fly to a machine should be DNN. The problem is that the trained DNN weights used to classify the initial set of objects are extremely fragile, meaning that any change to those weights can severely damage the capacity to perform the initial recognitions; this phenomenon is known as catastrophic forgetting (CF). This paper presents a new (DNN) continual learning (CL) architecture that can deal with CF, the modular dynamic neural network (MDNN). The presented architecture consists of two main components: (a) the ResNet50-based feature extraction component as the backbone; and (b) the modular dynamic classification component, which consists of multiple sub-networks and progressively builds itself up in a tree-like structure that rearranges itself as it learns over time in such a way that each sub-network can function independently. The main contribution of the paper is a new architecture that is strongly based on its modular dynamic training feature. This modular structure allows for new classes to be added while only altering specific sub-networks in such a way that previously known classes are not forgotten. Tests on the CORe50 dataset showed results above the state of the art for CL architectures.

Download Full-text