Casting a graph net to catch dark showers

Strongly interacting dark sectors predict novel LHC signatures such as semi-visible jets resulting from dark showers that contain both stable and unstable dark mesons. Distinguishing such semi-visible jets from large QCD backgrounds is difficult and constitutes an exciting challenge for jet classification. In this article we explore the potential of supervised deep neural networks to identify semi-visible jets. We show that dynamic graph convolutional neural networks operating on so-called particle clouds outperform convolutional neural networks analysing jet images as well as other neural networks based on Lorentz vectors. We investigate how the performance depends on the properties of the dark shower and discuss training on mixed samples as a strategy to reduce model dependence. By modifying an existing mono-jet analysis we show that LHC sensitivity to dark sectors can be enhanced by more than an order of magnitude by using the dynamic graph network as a dark shower tagger.

Download Full-text

Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning

Complexity ◽

10.1155/2019/2952304 ◽

2019 ◽

Vol 2019 ◽

pp. 1-16 ◽

Cited By ~ 7

Author(s):

Alejandro Baldominos ◽

Yago Saez ◽

Pedro Isasi

Keyword(s):

Neural Networks ◽

Evolutionary Computation ◽

Transfer Learning ◽

Convolutional Neural Networks ◽

Character Recognition ◽

Deep Neural Networks ◽

Handwriting Recognition ◽

Computational Time ◽

Learning Models ◽

Expensive Process

Neuroevolution is the field of study that uses evolutionary computation in order to optimize certain aspect of the design of neural networks, most often its topology and hyperparameters. The field was introduced in the late-1980s, but only in the latest years the field has become mature enough to enable the optimization of deep learning models, such as convolutional neural networks. In this paper, we rely on previous work to apply neuroevolution in order to optimize the topology of deep neural networks that can be used to solve the problem of handwritten character recognition. Moreover, we take advantage of the fact that evolutionary algorithms optimize a population of candidate solutions, by combining a set of the best evolved models resulting in a committee of convolutional neural networks. This process is enhanced by using specific mechanisms to preserve the diversity of the population. Additionally, in this paper, we address one of the disadvantages of neuroevolution: the process is very expensive in terms of computational time. To lessen this issue, we explore the performance of topology transfer learning: whether the best topology obtained using neuroevolution for a certain domain can be successfully applied to a different domain. By doing so, the expensive process of neuroevolution can be reused to tackle different problems, turning it into a more appealing approach for optimizing the design of neural networks topologies. After evaluating our proposal, results show that both the use of neuroevolved committees and the application of topology transfer learning are successful: committees of convolutional neural networks are able to improve classification results when compared to single models, and topologies learned for one problem can be reused for a different problem and data with a good performance. Additionally, both approaches can be combined by building committees of transferred topologies, and this combination attains results that combine the best of both approaches.

Download Full-text

Boosted Jet Tagging with Jet-Images and Deep Neural Networks

EPJ Web of Conferences ◽

10.1051/epjconf/201612700009 ◽

2016 ◽

Vol 127 ◽

pp. 00009 ◽

Cited By ~ 1

Author(s):

Michael Kagan ◽

Luke de Oliveira ◽

Lester Mackey ◽

Benjamin Nachman ◽

Ariel Schwartzman

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Jet Images

Download Full-text

Inter-Class Angular Loss for Convolutional Neural Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33013894 ◽

2019 ◽

Vol 33 ◽

pp. 3894-3901 ◽

Cited By ~ 1

Author(s):

Le Hui ◽

Xiang Li ◽

Chen Gong ◽

Meng Fang ◽

Joey Tianyi Zhou ◽

...

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

Learning Difficulties ◽

Feature Space ◽

Superior Performance ◽

Strongly Correlated ◽

Discriminative Ability ◽

Practical Applications ◽

Classification Tasks

Convolutional Neural Networks (CNNs) have shown great power in various classification tasks and have achieved remarkable results in practical applications. However, the distinct learning difficulties in discriminating different pairs of classes are largely ignored by the existing networks. For instance, in CIFAR-10 dataset, distinguishing cats from dogs is usually harder than distinguishing horses from ships. By carefully studying the behavior of CNN models in the training process, we observe that the confusion level of two classes is strongly correlated with their angular separability in the feature space. That is, the larger the inter-class angle is, the lower the confusion will be. Based on this observation, we propose a novel loss function dubbed “Inter-Class Angular Loss” (ICAL), which explicitly models the class correlation and can be directly applied to many existing deep networks. By minimizing the proposed ICAL, the networks can effectively discriminate the examples in similar classes by enlarging the angle between their corresponding class vectors. Thorough experimental results on a series of vision and nonvision datasets confirm that ICAL critically improves the discriminative ability of various representative deep neural networks and generates superior performance to the original networks with conventional softmax loss.

Download Full-text

Regularizing Deep Neural Networks with an Ensemble-based Decorrelation Method

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/301 ◽

2018 ◽

Author(s):

Shuqin Gu ◽

Yuexian Hou ◽

Lipeng Zhang ◽

Yazhou Zhang

Keyword(s):

Neural Networks ◽

Ensemble Learning ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

Experimental Results ◽

Excellent Performance ◽

Hidden Layer ◽

Base Learner ◽

Fully Connected

Although Deep Neural Networks (DNNs) have achieved excellent performance in many tasks, improving the generalization capacity of DNNs still remains a challenge. In this work, we propose a novel regularizer named Ensemble-based Decorrelation Method (EDM), which is motivated by the idea of the ensemble learning to improve generalization capacity of DNNs. EDM can be applied to hidden layers in fully connected neural networks or convolutional neural networks. We treat each hidden layer as an ensemble of several base learners through dividing all the hidden units into several non-overlap groups, and each group will be viewed as a base learner. EDM encourages DNNs to learn more diverse representations by minimizing the covariance between all base learners during the training step. Experimental results on MNIST and CIFAR datasets demonstrate that EDM can effectively reduce the overfitting and improve the generalization capacity of DNNs

Download Full-text

Vegetation Coverage in Marsh Grass Photography Using Convolutional Neural Networks

The International FLAIRS Conference Proceedings ◽

10.32473/flairs.v34i1.128498 ◽

2021 ◽

Vol 34 (1) ◽

Author(s):

Lucas Wayne Welch ◽

Xudong Liu ◽

Indika Kahanda ◽

Sandeep Reddivari ◽

Karthikeyan Umapathy

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

Vegetation Coverage ◽

Vegetation Monitoring ◽

Test Set ◽

Human Labor ◽

National Estuarine Research Reserve ◽

Multiple Species ◽

Marsh Grass

Vegetation monitoring is one of the major cornerstones of environmental protection today, giving scientists a look into changing ecosystems. One important task in vegetation monitoring is to estimate the coverage of vegetation in an area of marsh. This task often calls for extensive human labor carefully examining pixels in photos of marsh sites, a very time-consuming process. In this paper, aiming to automate this process, we propose a novel framework for such automation using deep neural networks. Then, we focus on the utmost component to build convolutional neural networks (CNNs) to identify the presence or absence of vegetation. To this end, we collect a new dataset with the help of Guana Tolomato Matanzas National Estuarine Research Reserve (GTMNERR) to be used to train and test the effectiveness of our selected CNN models, including LeNet-5 and two variants of AlexNet. Our experiments show that the AlexNet variants achieves higher accuracy scores on the test set than LeNet-5, with 92.41\% for a AlexNet variant ondistinguishing between vegetation and the lack thereof. These promising results suggest us to confidently move forward with not only expanding our dataset, but also developing models to determine multiple species in addition to the presence of live vegetation.

Download Full-text

Implementation of Convolutional Neural Networks for Warp Detection in 3D Printed Components manufactured via Fused Filament Fabrication: A Bayesian-Based Automated Approach

10.32920/ryerson.14635695 ◽

2021 ◽

Author(s):

Aditya Saluja

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

Closed Loop ◽

Structural Integrity ◽

Classification Models ◽

Fused Filament Fabrication ◽

3D Printed ◽

Monitoring Architecture ◽

Warp Deformation

Fused Filament Fabrication (FFF) is an additive manufacturing technique commonly used in industry to produce complicated structures sustainably. Although promising, the technology frequently suffers from defects, including warp deformation compromising the structural integrity of the component and, in extreme cases, the printer itself. To avoid the adverse effects of warp deformation, this thesis explores the implementation of deep neural networks to form a closed-loop in-process monitoring architecture using Convolutional Neural Networks (CNN) capable of pausing a printer once a warp is detected. Any neural network, including CNNs, depend on their hyperparameters. Hyperparameters can either be optimized using a manual or an automated approach. A manual approach, although easier to program, is often time-consuming, inaccurate and computationally inefficient, necessitating an automated approach. To evaluate this statement, classification models were optimized through both approaches and tested in a laboratory scaled manufacturing environment. The automated approach utilized a Bayesianbased optimizer yielding a mean accuracy of 100% significantly higher than 36% achieved by the other approach.

Download Full-text

Deep learning applied to seismic attribute computation

Interpretation ◽

10.1190/int-2018-0227.1 ◽

2019 ◽

Vol 7 (3) ◽

pp. SE141-SE150

Author(s):

Donald P. Griffith ◽

S. Ahmad Zamanian ◽

Jeremy Vila ◽

Antoine Vial-Aussavy ◽

John Solum ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Seismic Attributes ◽

Deep Convolutional Neural Networks ◽

Seismic Attribute ◽

Seismic Surveys ◽

Order Of Magnitude ◽

User Guidance ◽

Memory Efficient

We have trained deep convolutional neural networks (DCNs) to accelerate the computation of seismic attributes by an order of magnitude. These results are enabled by overcoming the prohibitive memory requirements typical of 3D DCNs for segmentation and regression by implementing a novel, memory-efficient 3D-to-2D convolutional architecture and by including tens of thousands of synthetically generated labeled examples to enhance DCN training. Including diverse synthetic labeled seismic in training helps the network generalize enabling it to accurately predict seismic attribute values on field-acquired seismic surveys. Once trained, our DCN tool generates attributes with no input parameters and no additional user guidance. The DCN attribute computations are virtually indistinguishable from conventionally computed attributes while computing up to 100 times faster.

Download Full-text

Geometric Attentional Dynamic Graph Convolutional Neural Networks for Point Cloud Analysis

Neurocomputing ◽

10.1016/j.neucom.2020.12.067 ◽

2020 ◽

Author(s):

Yiming Cui ◽

Xin Liu ◽

Hongmin Liu ◽

Jiyong Zhang ◽

Alina Zare ◽

...

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Point Cloud ◽

Dynamic Graph ◽

Point Cloud Analysis ◽

Cloud Analysis

Download Full-text

Microfluidic Droplet Detection via Region-Based and Single-Pass Convolutional Neural Networks with Comparison to Conventional Image Analysis Methodologies

10.26434/chemrxiv.14709477 ◽

2021 ◽

Author(s):

Gregory Rutkowski ◽

Ilgar Azizov ◽

Evan Unmann ◽

Marcin Dudek ◽

Brian Arthur Grimes

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

Data Gathering ◽

Image Data ◽

Validation Dataset ◽

Detection Accuracy ◽

Microfluidic Systems ◽

Training Set ◽

Analysis Pipeline

As the complexity of microfluidic experiments and the associated image data volumes scale, traditional feature extraction approaches begin to struggle at both detection and analysis pipeline throughput. Deep-neural networks trained to detect certain objects are rapidly emerging as data gathering tools that can either match or outperform the analysis capabilities of the conventional methods used in microfluidic emulsion science. We demonstrate that various convolutional neural networks can be trained and used as droplet detectors in a wide variety of microfluidic systems. A generalized microfluidic droplet training and validation dataset was developed and used to tune two versions of the You Only Look Once (YOLOv3/YOLOv5) model as well as Faster R-CNN. Each model was used to detect droplets in mono- and polydisperse flow cell systems. The detection accuracy of each model shows excellent statistical symmetry with an implementation of the Hough transform as well as relevant ImageJ plugins. The models were successfully used as droplet detectors in non-microfluidic micrograph observations, where these data were not included in the training set. The models outperformed the traditional methods in more complex, porous-media simulating chip architectures with a significant speedup to per-frame analysis times. Implementing these neural networks as the primary detectors in these microfluidic systems not only makes the data pipelining more efficient, but opens the door for live detection and development of autonomous microfluidic experimental platforms. <br>

Download Full-text

End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks

The International Journal of Robotics Research ◽

10.1177/0278364917734298 ◽

2017 ◽

Vol 37 (4-5) ◽

pp. 513-542 ◽

Cited By ~ 41

Author(s):

Sen Wang ◽

Ronald Clark ◽

Hongkai Wen ◽

Niki Trigoni

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Deep Neural Networks ◽

State Of The Art ◽

Visual Odometry ◽

Feature Representation ◽

Geometric Information ◽

Geometric Problem ◽

End To End

This paper studies visual odometry (VO) from the perspective of deep learning. After tremendous efforts in the robotics and computer vision communities over the past few decades, state-of-the-art VO algorithms have demonstrated incredible performance. However, since the VO problem is typically formulated as a pure geometric problem, one of the key features still missing from current VO systems is the capability to automatically gain knowledge and improve performance through learning. In this paper, we investigate whether deep neural networks can be effective and beneficial to the VO problem. An end-to-end, sequence-to-sequence probabilistic visual odometry (ESP-VO) framework is proposed for the monocular VO based on deep recurrent convolutional neural networks. It is trained and deployed in an end-to-end manner, that is, directly inferring poses and uncertainties from a sequence of raw images (video) without adopting any modules from the conventional VO pipeline. It can not only automatically learn effective feature representation encapsulating geometric information through convolutional neural networks, but also implicitly model sequential dynamics and relation for VO using deep recurrent neural networks. Uncertainty is also derived along with the VO estimation without introducing much extra computation. Extensive experiments on several datasets representing driving, flying and walking scenarios show competitive performance of the proposed ESP-VO to the state-of-the-art methods, demonstrating a promising potential of the deep learning technique for VO and verifying that it can be a viable complement to current VO systems.

Download Full-text