scholarly journals Comparison of Attention-based Deep LearningModels for EEG Classification

2021 ◽  
Author(s):  
Giulia Cisotto ◽  
Alessio Zanga ◽  
Joanna Chlebus ◽  
Italo Zoppis ◽  
Sara Manzoni ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Classification Performance ◽  
Classification Problems ◽  
Large Variability ◽  
Eeg Classification ◽  
Time Frequency ◽  
Promising Strategy ◽  
Complex Reasoning

Abstract Deep Learning (DL) has recently shown promising classification performance in Electroencephalography (EEG) in many different scenarios. However, the complex reasoning of such models often prevent the user to explain their classification abilities. Attention, one of the most recent and influential ideas in DL, allows the models to learn which portions of the data are relevant to the final classification output. In this work, we compared three attention-enhanced DL models, the brand-new InstaGATs , an LSTM with attention and a CNN with attention. We used these models to classify normal and abnormal, including artifactual and pathological, EEG patterns in three different datasets. We achieved the state of the art in all classification problems, regardless the large variability of the datasets and the simple architecture of the attention-enhanced models. Additionally, we proved that, depending on how the attention mechanism is applied and where the attention layer is located in the model, we can alternatively leverage the information contained in the time, frequency or space domain of the EEG dataset. Therefore, attention represents a promising strategy to evaluate the quality of the EEG information, and its relevance for classification, in different real-world scenarios.

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Ferdinand Filip ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Science ◽  
State Of The Art ◽  
Science Methods ◽  
Learning Models ◽  
Diverse Range ◽  
Hybrid Machine ◽  
Economics Research

This paper provides a state-of-the-art investigation of advances in data science in emerging economic applications. The analysis was performed on novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a wide and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, was used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which, based on the accuracy metric, outperform other learning algorithms. It is further expected that the trends will converge toward the advancements of sophisticated hybrid deep learning models.

scholarly journals Performance of deep learning technology for evaluation of positioning quality in periapical radiography of the maxillary canine

2021 ◽  
Author(s):  
Mizuho Mori ◽  
Yoshiko Ariji ◽  
Motoki Fukuda ◽  
Tomoya Kitano ◽  
Takuma Funakoshi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Characteristic Curve ◽  
Classification Performance ◽  
Learning Systems ◽  
Learning Technology ◽  
System 2 ◽  
Potential Benefits ◽  
System 1 ◽  
Periapical Radiography

Abstract Objectives The aim of the present study was to create and test an automatic system for assessing the technical quality of positioning in periapical radiography of the maxillary canines using deep learning classification and segmentation techniques. Methods We created and tested two deep learning systems using 500 periapical radiographs (250 each of good- and bad-quality images). We assigned 350, 70, and 80 images as the training, validation, and test datasets, respectively. The learning model of system 1 was created with only the classification process, whereas system 2 consisted of both the segmentation and classification models. In each model, 500 epochs of training were performed using AlexNet and U-net for classification and segmentation, respectively. The segmentation results were evaluated by the intersection over union method, with values of 0.6 or more considered as success. The classification results were compared between the two systems. Results The segmentation performance of system 2 was recall, precision, and F measure of 0.937, 0.961, and 0.949, respectively. System 2 showed better classification performance values than those obtained by system 1. The area under the receiver operating characteristic curve values differed significantly between system 1 (0.649) and system 2 (0.927). Conclusions The deep learning systems we created appeared to have potential benefits in evaluation of the technical positioning quality of periapical radiographs through the use of segmentation and classification functions.

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

2020 ◽  
Author(s):  
Saeed Nosratabadi ◽  
Amir Mosavi ◽  
Puhong Duan ◽  
Pedram Ghamisi ◽  
Filip Ferdinand ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Prediction Accuracy ◽  
Data Science ◽  
State Of The Art ◽  
Hybrid Models ◽  
The Other ◽  
Learning Models ◽  
Comprehensive Review

Abstract This paper provides the state of the art of data science in economics. Through a novel taxonomy of applications and methods advances in data science are investigated. The data science advances are investigated in three individual classes of deep learning models, ensemble models, and hybrid models. Application domains include stock market, marketing, E-commerce, corporate banking, and cryptocurrency. Prisma method, a systematic literature review methodology is used to ensure the quality of the survey. The findings revealed that the trends are on advancement of hybrid models as more than 51% of the reviewed articles applied hybrid model. On the other hand, it is found that based on the RMSE accuracy metric, hybrid models had higher prediction accuracy than other algorithms. While it is expected the trends go toward the advancements of deep learning models.

Enhanced deep residual network with multilevel correlation information for fault diagnosis of rotating machinery

2020 ◽  
pp. 107754632094971 ◽  
Author(s):  
Shoucong Xiong ◽  
Shuai He ◽  
Jianping Xuan ◽  
Qi Xia ◽  
Tielin Shi
Keyword(s):  
Deep Learning ◽  
Fault Diagnosis ◽  
Wavelet Packet ◽  
Learning Algorithms ◽  
Vibration Signal ◽  
Rolling Bearing ◽  
Classification Performance ◽  
Residual Network ◽  
Time Frequency ◽  
Correlation Information

Modern machinery becomes more precious with the advance of science, and fault diagnosis is vital for avoiding economical losses or casualties. Among massive diagnosis methods, deep learning algorithms stand out to open an era of intelligent fault diagnosis. Deep residual networks are the state-of-the-art deep learning models which can continuously improve performance by deepening the network structures. However, in vibration-based fault diagnosis, the transient property instability of vibration signal usually calls for time–frequency analysis methods, and the characters of time–frequency matrices are distinct from standard images, which brings some natural limitations for the diagnosis performance of deep learning algorithms. To handle this issue, an enhanced deep residual network named the multilevel correlation stack-deep residual network is proposed in this article. Wavelet packet transform is used to preprocess the sensor signal, and then the proposed multilevel correlation stack-deep residual network uses kernels with different shapes to fully dig various kinds of useful information from any local regions of the processed input. Experiments on two rolling bearing datasets are carried out. Test results show that the multilevel correlation stack-deep residual network exhibits a more satisfactory classification performance than original deep residual networks and other similar methods, revealing significant potentials for realistic fault diagnosis applications.

scholarly journals Deep learning for brain disorders: from data processing to disease treatment

2020 ◽  
Author(s):  
Ninon Burgos ◽  
Simona Bottani ◽  
Johann Faouzi ◽  
Elina Thibeau-Sutre ◽  
Olivier Colliot
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Language Processing ◽  
State Of The Art ◽  
Imaging Genetics ◽  
Environmental Data ◽  
Brain Disorders ◽  
Disease Treatment ◽  
Clinical Routine

Abstract In order to reach precision medicine and improve patients’ quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

scholarly journals DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6457
Author(s):  
Hayat Ullah ◽  
Muhammad Irfan ◽  
Kyungjin Han ◽  
Jong Weon Lee
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Quality Assessment ◽  
Image Quality Assessment ◽  
State Of The Art ◽  
Reference Image ◽  
Primary Concern ◽  
Panoramic Images ◽  
Stitching Errors

Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

scholarly journals HYBRID NSGA-II OPTIMIZATION FOR IMPROVING THE THREE-TERM BP NETWORK FOR MULTICLASS CLASSIFICATION PROBLEMS

2015 ◽  
Author(s):  
Ashraf Osman Ibrahim ◽  
Siti Mariyam Shamsuddin ◽  
Sultan Noman Qasem
Keyword(s):  
Genetic Algorithm ◽  
Evolutionary Algorithm ◽  
Hybrid Algorithm ◽  
Search Algorithm ◽  
Classification Performance ◽  
Classification Problems ◽  
Nsga Ii ◽  
Bp Network ◽  
Multiobjective Evolutionary Algorithm

Recently, hybrid algorithms have received considerable attention from a number of researchers. This paper presents a hybrid of the multiobjective evolutionary algorithm to gain a better accuracy of the fi nal solutions. The aim of using the hybrid algorithm is to improve the multiobjective evolutionary algorithm performance in terms of the enhancement of all the individuals in the population and increase the quality of the Pareto optimal solutions. The multiobjective evolutionary algorithm used in this study is a nondominated sorting genetic algorithm-II (NSGA-II) together with its hybrid, the backpropagation algorithm (BP), which is used as a local search algorithm to optimize the accuracy and complexity of the three-term backpropagation (TBP) network. The outcome positively demonstrates that the hybrid algorithm is able to improve the classification performance with a smaller number of hidden nodes and is effective in multiclass classifi cation problems. Furthermore, the results indicate that the proposed hybrid method is a potentially useful classifi er for enhancing the classification process ability when compared with the multiobjective genetic algorithm based on the TBP network (MOGATBP) and certain other methods found in the literature.  

scholarly journals Classification and Representation Joint Learning via Deep Networks

2017 ◽  
Author(s):  
Ya Li ◽  
Xinmei Tian ◽  
Xu Shen ◽  
Dacheng Tao
Keyword(s):  
Spatial Distribution ◽  
Deep Learning ◽  
Feature Learning ◽  
Classification Performance ◽  
Local Distribution ◽  
Classification Problems ◽  
Class Label ◽  
Channel Network ◽  
Training Samples ◽  
Label Information

Deep learning has been proven to be effective for classification problems. However, the majority of previous works trained classifiers by considering only class label information and ignoring the local information from the spatial distribution of training samples. In this paper, we propose a deep learning framework that considers both class label information and local spatial distribution information between training samples. A two-channel network with shared weights is used to measure the local distribution. The classification performance can be improved with more detailed information provided by the local distribution, particularly when the training samples are insufficient. Additionally, the class label information can help to learn better feature representations compared with other feature learning methods that use only local distribution information between samples. The local distribution constraint between sample pairs can also be viewed as a regularization of the network, which can efficiently prevent the overfitting problem. Extensive experiments are conducted on several benchmark image classification datasets, and the results demonstrate the effectiveness of our proposed method.

scholarly journals HAVAs: Alzheimer’s Disease Detection using Normative and Pathological Lifespan Models

2021 ◽  
Author(s):  
Pierrick Coupé ◽  
José V. Manjón ◽  
Boris Mansencal ◽  
Thomas Tourdias ◽  
Gwenaëlle Catheline ◽  
...  
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Deep Learning ◽  
Cognitive Impairment ◽  
Clinical Practice ◽  
State Of The Art ◽  
Brain Structures ◽  
Volumetric Analysis ◽  
Classification Performance

AbstractIn this paper, we present an innovative MRI-based method for Alzheimer’s Disease (AD) detection and mild cognitive impairment (MCI) prognostic, using lifespan trajectories of brain structures. After a full screening of the most discriminant structures between AD and normal aging based on MRI volumetric analysis of 3032 subjects, we propose a novel Hippocampal-Amygdalo-Ventricular Alzheimer score (HAVAs) based on normative lifespan models and AD lifespan models. During a validation on three external datasets on 1039 subjects, our approach showed very accurate detection (AUC ≥ 94%) of patients with AD compared to control subjects and accurate discrimination (AUC=78%) between progressive MCI and stable MCI (during a 3 years follow-up). Compared to normative modelling and recent state-of-the-art deep learning methods, our method demonstrated better classification performance. Moreover, HAVAs simplicity makes it fully understandable and thus well-suited for clinical practice or future pharmaceutical trials.

Sign in / Sign up

Export Citation Format

Share Document