Deep Learning Strategies for ProtoDUNE Raw Data Denoising

AbstractIn this work, we investigate different machine learning-based strategies for denoising raw simulation data from the ProtoDUNE experiment. The ProtoDUNE detector is hosted by CERN and it aims to test and calibrate the technologies for DUNE, a forthcoming experiment in neutrino physics. The reconstruction workchain consists of converting digital detector signals into physical high-level quantities. We address the first step in reconstruction, namely raw data denoising, leveraging deep learning algorithms. We design two architectures based on graph neural networks, aiming to enhance the receptive field of basic convolutional neural networks. We benchmark this approach against traditional algorithms implemented by the DUNE collaboration. We test the capabilities of graph neural network hardware accelerator setups to speed up training and inference processes.

Download Full-text

A Framework Based on Nesting of Convolutional Neural Networks to Classify Secondary Roads in High Resolution Aerial Orthoimages

Remote Sensing ◽

10.3390/rs12050765 ◽

2020 ◽

Vol 12 (5) ◽

pp. 765 ◽

Cited By ~ 4

Author(s):

Calimanut-Ionut Cira ◽

Ramon Alcarria ◽

Miguel-Ángel Manso-Callejo ◽

Francisco Serradilla

Keyword(s):

Remote Sensing ◽

Neural Networks ◽

Deep Learning ◽

High Resolution ◽

Learning Strategies ◽

Convolutional Neural Networks ◽

Test Data ◽

Road Network ◽

Ideal Solution ◽

The Road

Remote sensing imagery combined with deep learning strategies is often regarded as an ideal solution for interpreting scenes and monitoring infrastructures with remarkable performance levels. In addition, the road network plays an important part in transportation, and currently one of the main related challenges is detecting and monitoring the occurring changes in order to update the existent cartography. This task is challenging due to the nature of the object (continuous and often with no clearly defined borders) and the nature of remotely sensed images (noise, obstructions). In this paper, we propose a novel framework based on convolutional neural networks (CNNs) to classify secondary roads in high-resolution aerial orthoimages divided in tiles of 256 × 256 pixels. We will evaluate the framework’s performance on unseen test data and compare the results with those obtained by other popular CNNs trained from scratch.

Download Full-text

Online Planner Selection with Graph Neural Networks and Adaptive Scheduling

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.5949 ◽

2020 ◽

Vol 34 (04) ◽

pp. 5077-5084

Author(s):

Tengfei Ma ◽

Patrick Ferber ◽

Siyu Huo ◽

Jie Chen ◽

Michael Katz

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Automated Planning ◽

Adaptive Scheduling ◽

Optimal Planning ◽

Graph Representations ◽

Structural Graph ◽

Scheduling Method ◽

Graph Neural Networks ◽

Node Labels

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference.Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based.The code is available at https://github.com/matenure/GNN_planner.

Download Full-text

DNNBrain: A Unifying Toolbox for Mapping Deep Neural Networks and Brains

Frontiers in Computational Neuroscience ◽

10.3389/fncom.2020.580632 ◽

2020 ◽

Vol 14 ◽

Cited By ~ 1

Author(s):

Xiayu Chen ◽

Ming Zhou ◽

Zhengxin Gong ◽

Wei Xu ◽

Xingyu Liu ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Cognitive Neuroscience ◽

Deep Neural Networks ◽

Learning Strategy ◽

Neural Systems ◽

Levels Of Abstraction ◽

Internal Representations ◽

Internal Operations ◽

High Level

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Deep learning's success is appealing to neuroscientists not only as a method for applying DNNs to model biological neural systems but also as a means of adopting concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks, such as PyTorch and TensorFlow, could be used to allow such cross-disciplinary investigations, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed as a mechanism for cognitive neuroscientists to map both DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring the internal representations of DNNs as well as brains. Through the integration of DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios. These include extracting DNN activation, probing and visualizing DNN representations, and mapping DNN representations onto the brain. We expect that our toolbox will accelerate scientific research by both applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.

Download Full-text

Biological network analysis with deep learning

Briefings in Bioinformatics ◽

10.1093/bib/bbaa257 ◽

2020 ◽

Author(s):

Giulia Muzio ◽

Leslie O’Bray ◽

Karsten Borgwardt

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Biological Networks ◽

Protein Function ◽

Regulatory Networks ◽

Biological Network ◽

Gene Interaction ◽

Disease Diagnosis ◽

Interaction Prediction ◽

Graph Neural Networks

Abstract Recent advancements in experimental high-throughput technologies have expanded the availability and quantity of molecular data in biology. Given the importance of interactions in biological processes, such as the interactions between proteins or the bonds within a chemical compound, this data is often represented in the form of a biological network. The rise of this data has created a need for new computational tools to analyze networks. One major trend in the field is to use deep learning for this goal and, more specifically, to use methods that work with networks, the so-called graph neural networks (GNNs). In this article, we describe biological networks and review the principles and underlying algorithms of GNNs. We then discuss domains in bioinformatics in which graph neural networks are frequently being applied at the moment, such as protein function prediction, protein–protein interaction prediction and in silico drug discovery and development. Finally, we highlight application areas such as gene regulatory networks and disease diagnosis where deep learning is emerging as a new tool to answer classic questions like gene interaction prediction and automatic disease prediction from data.

Download Full-text

An Interpretable Deep Learning Approach for Biomarker Detection in LC-MS Proteomics Data

10.1101/2021.02.19.431935 ◽

2021 ◽

Author(s):

Sahar Iravani ◽

Tim O.F. Conrad

Keyword(s):

Mass Spectrometry ◽

Neural Networks ◽

Deep Learning ◽

Clinical State ◽

Biomarker Detection ◽

Proteomics Data ◽

Comparable Amount ◽

Complex Protein ◽

Propagation Technique ◽

High Level

AbstractAnalyzing mass spectrometry-based proteomics data with deep learning (DL) approaches poses several challenges due to the high dimensionality, low sample size, and high level of noise. Besides, DL-based workflows are often hindered to be integrated into medical settings due to the lack of interpretable explanation. We present DLearnMS, a DL biomarker detection framework, to address these challenges on proteomics instances of liquid chromatography-mass spectrometry (LC-MS) - a well-established tool for quantifying complex protein mixtures. Our DLearnMS framework learns the clinical state of LC-MS data instances using convolutional neural networks. Next, based on the trained neural networks, biomarkers can be identified using the layer-wise relevance propagation technique. This enables detecting discriminating regions of the data and the design of more robust networks. We show that DLearnMS outperforms conventional LC-MS biomarker detection approaches in detecting fewer false positive peaks while maintaining a comparable amount of true positives peaks. Unlike other methods, no explicit preprocessing step is needed in DLearnMS.

Download Full-text

Deep learning in bioinformatics: introduction, application, and perspective in big data era

10.1101/563601 ◽

2019 ◽

Cited By ~ 9

Author(s):

Yu Li ◽

Chao Huang ◽

Lizhong Ding ◽

Zhongxiao Li ◽

Yijie Pan ◽

...

Keyword(s):

Neural Networks ◽

Big Data ◽

Deep Learning ◽

Generative Adversarial Networks ◽

Great Success ◽

Research Directions ◽

Adversarial Networks ◽

Variational Autoencoder ◽

The Common ◽

Graph Neural Networks

AbstractDeep learning, which is especially formidable in handling big data, has achieved great success in various fields, including bioinformatics. With the advances of the big data era in biology, it is foreseeable that deep learning will become increasingly important in the field and will be incorporated in vast majorities of analysis pipelines. In this review, we provide both the exoteric introduction of deep learning, and concrete examples and implementations of its representative applications in bioinformatics. We start from the recent achievements of deep learning in the bioinformatics field, pointing out the problems which are suitable to use deep learning. After that, we introduce deep learning in an easy-to-understand fashion, from shallow neural networks to legendary convolutional neural networks, legendary recurrent neural networks, graph neural networks, generative adversarial networks, variational autoencoder, and the most recent state-of-the-art architectures. After that, we provide eight examples, covering five bioinformatics research directions and all the four kinds of data type, with the implementation written in Tensorflow and Keras. Finally, we discuss the common issues, such as overfitting and interpretability, that users will encounter when adopting deep learning methods and provide corresponding suggestions. The implementations are freely available at https://github.com/lykaust15/Deep_learning_examples.

Download Full-text

Inference of Brain States under Anesthesia with Meta Learning Based Deep Learning Models

10.1101/2021.12.05.471326 ◽

2021 ◽

Author(s):

Qihang Wang ◽

Feng Liu ◽

Guihong Wan ◽

Ying Chen

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Short Term Memory ◽

Signal To Noise Ratio ◽

General Anesthetics ◽

Learning Models ◽

Multi Stage ◽

Meta Learning ◽

Brain States ◽

High Level

AbstractMonitoring the depth of unconsciousness during anesthesia is useful in both clinical settings and neuroscience investigations to understand brain mechanisms. Electroencephalogram (EEG) has been used as an objective means of characterizing brain altered arousal and/or cognition states induced by anesthetics in real-time. Different general anesthetics affect cerebral electrical activities in different ways. However, the performance of conventional machine learning models on EEG data is unsatisfactory due to the low Signal to Noise Ratio (SNR) in the EEG signals, especially in the office-based anesthesia EEG setting. Deep learning models have been used widely in the field of Brain Computer Interface (BCI) to perform classification and pattern recognition tasks due to their capability of good generalization and handling noises. Compared to other BCI applications, where deep learning has demonstrated encouraging results, the deep learning approach for classifying different brain consciousness states under anesthesia has been much less investigated. In this paper, we propose a new framework based on meta-learning using deep neural networks, named Anes-MetaNet, to classify brain states under anesthetics. The Anes-MetaNet is composed of Convolutional Neural Networks (CNN) to extract power spectrum features, and a time consequence model based on Long Short-Term Memory (LSTM) Networks to capture the temporal dependencies, and a meta-learning framework to handle large cross-subject variability. We used a multi-stage training paradigm to improve the performance, which is justified by visualizing the high-level feature mapping. Experiments on the office-based anesthesia EEG dataset demonstrate the effectiveness of our proposed Anes-MetaNet by comparison of existing methods.

Download Full-text

Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

Frontiers in Artificial Intelligence ◽

10.3389/frai.2021.618372 ◽

2021 ◽

Vol 4 ◽

Author(s):

Paul Y. Wang ◽

Sandalika Sapra ◽

Vivek Kurien George ◽

Gabriel A. Silva

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Deep Learning ◽

Neural Systems ◽

Graph Structure ◽

Imaging Data ◽

C Elegans ◽

State Classification ◽

Graph Neural Networks ◽

Emergent Behaviors

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

Download Full-text

Deep Learning

International Journal of Semantic Computing ◽

10.1142/s1793351x16500045 ◽

2016 ◽

Vol 10 (03) ◽

pp. 417-439 ◽

Cited By ~ 28

Author(s):

Xing Hao ◽

Guigang Zhang ◽

Shang Ma

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Deep Learning ◽

Complex Structures ◽

Training Algorithms ◽

High Level ◽

And Training

Deep learning is a branch of machine learning that tries to model high-level abstractions of data using multiple layers of neurons consisting of complex structures or non-liner transformations. With the increase of the amount of data and the power of computation, neural networks with more complex structures have attracted widespread attention and been applied to various fields. This paper provides an overview of deep learning in neural networks including popular architecture models and training algorithms.

Download Full-text

Two-Stage Approach to Image Classification by Deep Neural Networks

EPJ Web of Conferences ◽

10.1051/epjconf/201817301009 ◽

2018 ◽

Vol 173 ◽

pp. 01009 ◽

Cited By ~ 3

Author(s):

Gennady Ososkov ◽

Pavel Goncharov

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Deep Neural Networks ◽

Cloud Services ◽

Activation Functions ◽

Learning Networks ◽

Actual Problem ◽

Learning Time ◽

Autoassociative Neural Network ◽

Speed Up

The paper demonstrates the advantages of the deep learning networks over the ordinary neural networks on their comparative applications to image classifying. An autoassociative neural network is used as a standalone autoencoder for prior extraction of the most informative features of the input data for neural networks to be compared further as classifiers. The main efforts to deal with deep learning networks are spent for a quite painstaking work of optimizing the structures of those networks and their components, as activation functions, weights, as well as the procedures of minimizing their loss function to improve their performances and speed up their learning time. It is also shown that the deep autoencoders develop the remarkable ability for denoising images after being specially trained. Convolutional Neural Networks are also used to solve a quite actual problem of protein genetics on the example of the durum wheat classification. Results of our comparative study demonstrate the undoubted advantage of the deep networks, as well as the denoising power of the autoencoders. In our work we use both GPU and cloud services to speed up the calculations.

Download Full-text