A novel structured sparse fully connected layer in convolutional neural networks

Damage identification of composite structures is a major ongoing challenge for a secure operational life-cycle due to the complex, gradual damage behaviour of composite materials. Especially for composite rotors in aero-engines and wind-turbines, a cost-intensive maintenance service has to be performed in order to avoid critical failure. A major advantage of composite structures is that they are able to safely operate after damage initiation and under ongoing damage propagation. Therefore, a robust, efficient diagnostic damage identification method would allow monitoring the damage process with intervention occurring only when necessary. This study investigates the structural vibration response of composite rotors by applying machine learning methods and the ability to identify, localise and quantify the present damage. To this end, multiple fully connected neural networks and convolutional neural networks were trained on vibration response spectra from damaged composite rotors with barely visible damage, mostly matrix cracks and local delaminations using dimensionality reduction and data augmentation. A databank containing 720 simulated test cases with different damage states is used as a basis for the generation of multiple data sets. The trained models are tested using k-fold cross validation and they are evaluated based on the sensitivity, specificity and accuracy. Convolutional neural networks perform slightly better providing a performance accuracy of up to 99.3% for the damage localisation and quantification.

Download Full-text

Driving Stress Detection Using Multimodal Convolutional Neural Networks with Nonlinear Representation of Short-Term Physiological Signals

Sensors ◽

10.3390/s21072381 ◽

2021 ◽

Vol 21 (7) ◽

pp. 2381

Author(s):

Jaewon Lee ◽

Hyeonjeong Lee ◽

Miyoung Shin

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Traffic Accidents ◽

Physiological Signals ◽

Superior Performance ◽

Short Term ◽

Input Signals ◽

Skin Response ◽

Novel Method ◽

Fully Connected

Mental stress can lead to traffic accidents by reducing a driver’s concentration or increasing fatigue while driving. In recent years, demand for methods to detect drivers’ stress in advance to prevent dangerous situations increased. Thus, we propose a novel method for detecting driving stress using nonlinear representations of short-term (30 s or less) physiological signals for multimodal convolutional neural networks (CNNs). Specifically, from hand/foot galvanic skin response (HGSR, FGSR) and heart rate (HR) short-term input signals, first, we generate corresponding two-dimensional nonlinear representations called continuous recurrence plots (Cont-RPs). Second, from the Cont-RPs, we use multimodal CNNs to automatically extract FGSR, HGSR, and HR signal representative features that can effectively differentiate between stressed and relaxed states. Lastly, we concatenate the three extracted features into one integrated representation vector, which we feed to a fully connected layer to perform classification. For the evaluation, we use a public stress dataset collected from actual driving environments. Experimental results show that the proposed method demonstrates superior performance for 30-s signals, with an overall accuracy of 95.67%, an approximately 2.5–3% improvement compared with that of previous works. Additionally, for 10-s signals, the proposed method achieves 92.33% classification accuracy, which is similar to or better than the performance of other methods using long-term signals (over 100 s).

Download Full-text

KLASIFIKASI BATIK RIAU DENGAN MENGGUNAKAN CONVOLUTIONAL NEURAL NETWORKS (CNN)

Jurnal Ilmu Komputer ◽

10.33060/jik/2020/vol9.iss1.144 ◽

2020 ◽

Vol 9 (1) ◽

pp. 7-10

Author(s):

Hendry Fonda

Keyword(s):

Neural Network ◽

Neural Networks ◽

Artificial Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

18Th Century ◽

The Public ◽

Artificial Neural ◽

The Difference ◽

Fully Connected

ABSTRACT Riau batik is known since the 18th century and is used by royal kings. Riau Batik is made by using a stamp that is mixed with coloring and then printed on fabric. The fabric used is usually silk. As its development, comparing Javanese batik with riau batik Riau is very slowly accepted by the public. Convolutional Neural Networks (CNN) is a combination of artificial neural networks and deeplearning methods. CNN consists of one or more convolutional layers, often with a subsampling layer followed by one or more fully connected layers as a standard neural network. In the process, CNN will conduct training and testing of Riau batik so that a collection of batik models that have been classified based on the characteristics that exist in Riau batik can be determined so that images are Riau batik and non-Riau batik. Classification using CNN produces Riau batik and not Riau batik with an accuracy of 65%. Accuracy of 65% is due to basically many of the same motifs between batik and other batik with the difference lies in the color of the absorption in the batik riau. Kata kunci: Batik; Batik Riau; CNN; Image; Deep Learning ABSTRAK Batik Riau dikenal sejak abad ke 18 dan digunakan oleh bangsawan raja. Batik Riau dibuat dengan menggunakan cap yang dicampur dengan pewarna kemudian dicetak di kain. Kain yang digunakan biasanya sutra. Seiring perkembangannya, dibandingkan batik Jawa maka batik Riau sangat lambat diterima oleh masyarakat. Convolutional Neural Networks (CNN) merupakan kombinasi dari jaringan syaraf tiruan dan metode deeplearning. CNN terdiri dari satu atau lebih lapisan konvolutional, seringnya dengan suatu lapisan subsampling yang diikuti oleh satu atau lebih lapisan yang terhubung penuh sebagai standar jaringan syaraf. Dalam prosesnya CNN akan melakukan training dan testing terhadap batik Riau sehingga didapat kumpulan model batik yang telah terklasi fikasi berdasarkan ciri khas yang ada pada batik Riau sehingga dapat ditentukan gambar (image) yang merupakan batik Riau dan yang bukan merupakan batik Riau. Klasifikasi menggunakan CNN menghasilkan batik riau dan bukan batik riau dengan akurasi 65%. Akurasi 65% disebabkan pada dasarnya banyak motif yang sama antara batik riau dengan batik lainnya dengan perbedaan terletak pada warna cerap pada batik riau. Kata kunci: Batik; Batik Riau; CNN; Image; Deep Learning

Download Full-text

A primer on deep learning and convolutional neural networks for clinicians

Insights into Imaging ◽

10.1186/s13244-021-01052-z ◽

2021 ◽

Vol 12 (1) ◽

Author(s):

Lara Lloret Iglesias ◽

Pablo Sanz Bellón ◽

Amaia Pérez del Barrio ◽

Pablo Menéndez Fernández-Miranda ◽

David Rodríguez González ◽

...

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Learning Capabilities ◽

Computer Vision Applications ◽

Fully Connected ◽

Imaging Perspective ◽

Adaptive Momentum

AbstractDeep learning is nowadays at the forefront of artificial intelligence. More precisely, the use of convolutional neural networks has drastically improved the learning capabilities of computer vision applications, being able to directly consider raw data without any prior feature extraction. Advanced methods in the machine learning field, such as adaptive momentum algorithms or dropout regularization, have dramatically improved the convolutional neural networks predicting ability, outperforming that of conventional fully connected neural networks. This work summarizes, in an intended didactic way, the main aspects of these cutting-edge techniques from a medical imaging perspective.

Download Full-text

Benchmarking of Fully Connected and Convolutional Neural Networks on Face Recognition: Case of Makeup and Occlusions

10.1007/978-3-030-80129-8_20 ◽

2021 ◽

pp. 270-285

Author(s):

Stanislav Selitskiy ◽

Nikolaos Christou ◽

Natalya Selitskaya

Keyword(s):

Neural Networks ◽

Face Recognition ◽

Convolutional Neural Networks ◽

Fully Connected

Download Full-text

Regularizing Deep Neural Networks with an Ensemble-based Decorrelation Method

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/301 ◽

2018 ◽

Author(s):

Shuqin Gu ◽

Yuexian Hou ◽

Lipeng Zhang ◽

Yazhou Zhang

Keyword(s):

Neural Networks ◽

Ensemble Learning ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

Experimental Results ◽

Excellent Performance ◽

Hidden Layer ◽

Base Learner ◽

Fully Connected

Although Deep Neural Networks (DNNs) have achieved excellent performance in many tasks, improving the generalization capacity of DNNs still remains a challenge. In this work, we propose a novel regularizer named Ensemble-based Decorrelation Method (EDM), which is motivated by the idea of the ensemble learning to improve generalization capacity of DNNs. EDM can be applied to hidden layers in fully connected neural networks or convolutional neural networks. We treat each hidden layer as an ensemble of several base learners through dividing all the hidden units into several non-overlap groups, and each group will be viewed as a base learner. EDM encourages DNNs to learn more diverse representations by minimizing the covariance between all base learners during the training step. Experimental results on MNIST and CIFAR datasets demonstrate that EDM can effectively reduce the overfitting and improve the generalization capacity of DNNs

Download Full-text

Design space exploration of Convolutional Neural Networks based on Evolutionary Algorithms

Journal of Computational Vision and Imaging Systems ◽

10.15353/vsnl.v3i1.162 ◽

2017 ◽

Vol 3 (1) ◽

Cited By ~ 4

Author(s):

Abeer Al-Hyari ◽

Shawki Areibi

Keyword(s):

Neural Networks ◽

Genetic Algorithms ◽

Evolutionary Algorithms ◽

Convolutional Neural Networks ◽

Design Space Exploration ◽

Design Space ◽

Space Exploration ◽

Handwritten Digit ◽

Fully Connected ◽

The Given

This paper proposes a framework for design space exploration ofConvolutional Neural Networks (CNNs) using Genetic Algorithms(GAs). CNNs have many hyperparameters that need to be tunedcarefully in order to achieve favorable results when used for imageclassification tasks or similar vision applications. Genetic Algorithmsare adopted to efficiently traverse the huge search spaceof CNNs hyperparameters, and generate the best architecture thatfits the given task. Some of the hyperparameters that were testedinclude the number of convolutional and fully connected layers, thenumber of filters for each convolutional layer, and the number ofnodes in the fully connected layers. The proposed approach wastested using MNIST dataset for handwritten digit classification andresults obtained indicate that the proposed approach is able to generatea CNN architecture with validation accuracy up to 96.66% onaverage.

Download Full-text

Can Pre-Trained Convolutional Neural Networks be directly used as a Feature Extractor for Video-based Neonatal Sleep and Wake Classification?

10.21203/rs.3.rs-56693/v3 ◽

2020 ◽

Author(s):

Muhammad Awais ◽

Xi Long ◽

Bin Yin ◽

Chen chen ◽

Saeed Akbarzadeh ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Convolutional Neural Networks ◽

Video Recording ◽

Principal Component ◽

Classification Performance ◽

Support Vector ◽

Video Frames ◽

Feature Extractor ◽

Fully Connected

Abstract Objective: In this paper, we propose to evaluate the use of a pre-trained convolutional neural networks (CNNs) as a features extractor followed by the Principal Component Analysis (PCA) to find the best discriminant features to perform classification using support vector machine (SVM) algorithm for neonatal sleep and wake states using Fluke® facial video frames. Using pre-trained CNNs as feature extractor would hugely reduce the effort of collecting new neonatal data for training a neural network which could be computationally very expensive. The features are extracted after fully connected layers (FCL’s), where we compare several pre-trained CNNs, e.g., VGG16, VGG19, InceptionV3, GoogLeNet, ResNet, and AlexNet. Results: From around 2-h Fluke® video recording of seven neonate, we achieved a modest classification performance with an accuracy, sensitivity, and specificity of 65.3%, 69.8%, 61.0%, respectively with AlexNet using Fluke® (RGB) video frames. This indicates that using a pre-trained model as a feature extractor could not fully suffice for highly reliable sleep and wake classification in neonates. Therefore, in future a dedicated neural network trained on neonatal data or a transfer learning approach is required.

Download Full-text

Introspective analysis of convolutional neural networks for improving discrimination performance and feature visualisation

PeerJ Computer Science ◽

10.7717/peerj-cs.497 ◽

2021 ◽

Vol 7 ◽

pp. e497

Author(s):

Shakeel Shafiq ◽

Tayyaba Azim

Keyword(s):

Neural Network ◽

Neural Networks ◽

Convolutional Neural Network ◽

Convolutional Neural Networks ◽

Discrimination Performance ◽

Input Image ◽

Data Sets ◽

Discrimination Power ◽

Level Information ◽

Fully Connected

Deep neural networks have been widely explored and utilised as a useful tool for feature extraction in computer vision and machine learning. It is often observed that the last fully connected (FC) layers of convolutional neural network possess higher discrimination power as compared to the convolutional and maxpooling layers whose goal is to preserve local and low-level information of the input image and down sample it to avoid overfitting. Inspired from the functionality of local binary pattern (LBP) operator, this paper proposes to induce discrimination into the mid layers of convolutional neural network by introducing a discriminatively boosted alternative to pooling (DBAP) layer that has shown to serve as a favourable replacement of early maxpooling layer in a convolutional neural network (CNN). A thorough research of the related works show that the proposed change in the neural architecture is novel and has not been proposed before to bring enhanced discrimination and feature visualisation power achieved from the mid layer features. The empirical results reveal that the introduction of DBAP layer in popular neural architectures such as AlexNet and LeNet produces competitive classification results in comparison to their baseline models as well as other ultra-deep models on several benchmark data sets. In addition, better visualisation of intermediate features can allow one to seek understanding and interpretation of black box behaviour of convolutional neural networks, used widely by the research community.

Download Full-text

Differentiating COVID-19 from other types of pneumonia with convolutional neural networks

10.1101/2020.05.26.20113761 ◽

2020 ◽

Author(s):

Ilker Ozsahin ◽

Confidence Onyebuchi ◽

Boran Sekeroglu

Keyword(s):

Neural Networks ◽

Real Time ◽

Convolutional Neural Networks ◽

Bacterial Pneumonia ◽

Viral Pneumonia ◽

Rt Pcr ◽

Healthy Group ◽

X Ray ◽

Healthy Control ◽

Fully Connected

AbstractINTRODUCTIONA widely-used method for diagnosing COVID-19 is the nucleic acid test based on real-time reverse transcriptase-polymerase chain reaction (RT-PCR). However, the sensitivity of real time RT-PCR tests is low and it can take up to 8 hours to receive the test results. Radiologic methods can provide higher sensitivity. The aim of this study is to investigate the use of X-ray and convolutional neural networks for the diagnosis of COVID-19 and to differentiate it from viral and/or bacterial pneumonia, as 2-class (bacterial pneumonia vs COVID-19 and viral pneumonia vs COVID-19) and 3- class (bacterial pneumonia, COVID-19, and healthy group (BCH), and among viral pneumonia, COVID- 19, and healthy group (VCH)) experiments.METHODS225 COVID-19, 1,583 healthy control, 2,780 bacterial pneumonia, and 1,493 viral pneumonia chest X-ray images were used. 2-class- and 3-class-experiments were performed with different convolutional neural network (ConvNet) architectures, with different variations of convolutional layers and fully-connected layers.RESULTSThe results showed that bacterial pneumonia vs COVID-19 and viral pneumonia vs COVID- 19 reached a mean ROC AUC of 97.32% and 96.80%, respectively. In the 3-class-experiments, macro-average F1 scores of 95.79% and 94.59% were obtained in terms of detecting COVID-19 among BCH and VCH, respectively.CONCLUSIONSThe ConvNet was able to distinguish the COVID-19 images among non-COVID-19 images, namely bacterial and viral pneumonia as well as normal X-ray images.

Download Full-text