Deep-learning jets with uncertainties and more

Bayesian neural networks allow us to keep track of uncertainties, for example in top tagging, by learning a tagger output together with an error band. We illustrate the main features of Bayesian versions of established deep-learning taggers. We show how they capture statistical uncertainties from finite training samples, systematics related to the jet energy scale, and stability issues through pile-up. Altogether, Bayesian networks offer many new handles to understand and control deep learning at the LHC without introducing a visible prior effect and without compromising the network performance.

Download Full-text

Application of convolutional neural networks for stellar spectral classification

Monthly Notices of the Royal Astronomical Society ◽

10.1093/mnras/stz3100 ◽

2019 ◽

Vol 491 (2) ◽

pp. 2280-2300 ◽

Cited By ~ 6

Author(s):

Kaushal Sharma ◽

Ajit Kembhavi ◽

Aniruddha Kembhavi ◽

T Sivarani ◽

Sheelu Abraham ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Spectral Classification ◽

Stellar Spectra ◽

The Past ◽

Training Samples ◽

Optical Region ◽

Spectral Libraries ◽

Improved Accuracy

ABSTRACT Due to the ever-expanding volume of observed spectroscopic data from surveys such as SDSS and LAMOST, it has become important to apply artificial intelligence (AI) techniques for analysing stellar spectra to solve spectral classification and regression problems like the determination of stellar atmospheric parameters Teff, $\rm {\log g}$, and [Fe/H]. We propose an automated approach for the classification of stellar spectra in the optical region using convolutional neural networks (CNNs). Traditional machine learning (ML) methods with ‘shallow’ architecture (usually up to two hidden layers) have been trained for these purposes in the past. However, deep learning methods with a larger number of hidden layers allow the use of finer details in the spectrum which results in improved accuracy and better generalization. Studying finer spectral signatures also enables us to determine accurate differential stellar parameters and find rare objects. We examine various machine and deep learning algorithms like artificial neural networks, Random Forest, and CNN to classify stellar spectra using the Jacoby Atlas, ELODIE, and MILES spectral libraries as training samples. We test the performance of the trained networks on the Indo-U.S. Library of Coudé Feed Stellar Spectra (CFLIB). We show that using CNNs, we are able to lower the error up to 1.23 spectral subclasses as compared to that of two subclasses achieved in the past studies with ML approach. We further apply the trained model to classify stellar spectra retrieved from the SDSS data base with SNR > 20.

Download Full-text

Combining Spectral Unmixing and 3D/2D Dense Networks with Early-Exiting Strategy for Hyperspectral Image Classification

Remote Sensing ◽

10.3390/rs12050779 ◽

2020 ◽

Vol 12 (5) ◽

pp. 779 ◽

Cited By ~ 1

Author(s):

Bei Fang ◽

Yunpeng Bai ◽

Ying Li

Keyword(s):

Deep Learning ◽

Network Performance ◽

Hyperspectral Image ◽

Spectral Unmixing ◽

Classification Methods ◽

Dense Network ◽

Training Samples ◽

Mixed Pixels ◽

Benchmark Datasets ◽

Hard Samples

Recently, Hyperspectral Image (HSI) classification methods based on deep learning models have shown encouraging performance. However, the limited numbers of training samples, as well as the mixed pixels due to low spatial resolution, have become major obstacles for HSI classification. To tackle these problems, we propose a resource-efficient HSI classification framework which introduces adaptive spectral unmixing into a 3D/2D dense network with early-exiting strategy. More specifically, on one hand, our framework uses a cascade of intermediate classifiers throughout the 3D/2D dense network that is trained end-to-end. The proposed 3D/2D dense network that integrates 3D convolutions with 2D convolutions is more capable of handling spectral-spatial features, while containing fewer parameters compared with the conventional 3D convolutions, and further boosts the network performance with limited training samples. On another hand, considering the existence of mixed pixels in HSI data, the pixels in HSI classification are divided into hard samples and easy samples. With the early-exiting strategy in these intermediate classifiers, the average accuracy can be improved by reducing the amount of computation cost for easy samples, thus focusing on classifying hard samples. Furthermore, for hard samples, an adaptive spectral unmixing method is proposed as a complementary source of information for classification, which brings considerable benefits to the final performance. Experimental results on four HSI benchmark datasets demonstrate that the proposed method can achieve better performance than state-of-the-art deep learning-based methods and other traditional HSI classification methods.

Download Full-text

Remote Insects Trap Monitoring System Using Deep Learning Framework and IoT

Sensors ◽

10.3390/s20185280 ◽

2020 ◽

Vol 20 (18) ◽

pp. 5280

Author(s):

Balakrishnan Ramalingam ◽

Rajesh Elara Mohan ◽

Sathian Pookkuttath ◽

Braulio Félix Gómez ◽

Charan Satya Chandra Sairam Borusu ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Built Environment ◽

Object Detection ◽

Real Time ◽

Monitoring System ◽

Early Stage ◽

Control Measures ◽

Dependent Tasks ◽

And Control

Insect detection and control at an early stage are essential to the built environment (human-made physical spaces such as homes, hotels, camps, hospitals, parks, pavement, food industries, etc.) and agriculture fields. Currently, such insect control measures are manual, tedious, unsafe, and time-consuming labor dependent tasks. With the recent advancements in Artificial Intelligence (AI) and the Internet of things (IoT), several maintenance tasks can be automated, which significantly improves productivity and safety. This work proposes a real-time remote insect trap monitoring system and insect detection method using IoT and Deep Learning (DL) frameworks. The remote trap monitoring system framework is constructed using IoT and the Faster RCNN (Region-based Convolutional Neural Networks) Residual neural Networks 50 (ResNet50) unified object detection framework. The Faster RCNN ResNet 50 object detection framework was trained with built environment insects and farm field insect images and deployed in IoT. The proposed system was tested in real-time using four-layer IoT with built environment insects image captured through sticky trap sheets. Further, farm field insects were tested through a separate insect image database. The experimental results proved that the proposed system could automatically identify the built environment insects and farm field insects with an average of 94% accuracy.

Download Full-text

Deep Learning Network Intensification for Preventing Noisy-Labeled Samples for Remote Sensing Classification

Remote Sensing ◽

10.3390/rs13091689 ◽

2021 ◽

Vol 13 (9) ◽

pp. 1689

Author(s):

Chuang Lin ◽

Shanxin Guo ◽

Jinsong Chen ◽

Luyi Sun ◽

Xiaorou Zheng ◽

...

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

Network Performance ◽

Aerial Image ◽

Label Noise ◽

Remote Sensing Classification ◽

Learning Network ◽

Training Samples ◽

Deep Learning Network ◽

The Impact

The deep-learning-network performance depends on the accuracy of the training samples. The training samples are commonly labeled by human visual investigation or inherited from historical land-cover or land-use maps, which usually contain label noise, depending on subjective knowledge and the time of the historical map. Helping the network to distinguish noisy labels during the training process is a prerequisite for applying the model for training across time and locations. This study proposes an antinoise framework, the Weight Loss Network (WLN), to achieve this goal. The WLN contains three main parts: (1) the segmentation subnetwork, which any state-of-the-art segmentation network can replace; (2) the attention subnetwork (λ); and (3) the class-balance coefficient (α). Four types of label noise (an insufficient label, redundant label, missing label and incorrect label) were simulated by dilate and erode processing to test the network’s antinoise ability. The segmentation task was set to extract buildings from the Inria Aerial Image Labeling Dataset, which includes Austin, Chicago, Kitsap County, Western Tyrol and Vienna. The network’s performance was evaluated by comparing it with the original U-Net model by adding noisy training samples with different noise rates and noise levels. The result shows that the proposed antinoise framework (WLN) can maintain high accuracy, while the accuracy of the U-Net model dropped. Specifically, after adding 50% of dilated-label samples at noise level 3, the U-Net model’s accuracy dropped by 12.7% for OA, 20.7% for the Mean Intersection over Union (MIOU) and 13.8% for Kappa scores. By contrast, the accuracy of the WLN dropped by 0.2% for OA, 0.3% for the MIOU and 0.8% for Kappa scores. For eroded-label samples at the same level, the accuracy of the U-Net model dropped by 8.4% for OA, 24.2% for the MIOU and 43.3% for Kappa scores, while the accuracy of the WLN dropped by 4.5% for OA, 4.7% for the MIOU and 0.5% for Kappa scores. This result shows that the antinoise framework proposed in this paper can help current segmentation models to avoid the impact of noisy training labels and has the potential to be trained by a larger remote sensing image set regardless of the inner label error.

Download Full-text

Image Visual Sensor Used in Health-Care Navigation in Indoor Scenes Using Deep Reinforcement Learning (DRL) and Control Sensor Robot for Patients Data Health Information

Journal of Medical Imaging and Health Informatics ◽

10.1166/jmihi.2021.3283 ◽

2021 ◽

Vol 11 (1) ◽

pp. 104-113

Author(s):

Walead Kaled Seaman ◽

Sırma Yavuz

Keyword(s):

Deep Learning ◽

Reinforcement Learning ◽

Active Agent ◽

Visual Navigation ◽

Sequential Decision ◽

Visual Sensor ◽

Reward Function ◽

Training Samples ◽

Indoor Scenes ◽

And Control

Compared with traditional motion planners and deep reinforcement learning DRL has been applied more and more widely to achieving sequential behaviors control of movement robots in internal environment. There are two addressed issues of deep learning. The inability to generalize to achieve set of goals. The data inefficiency, that is, the model requires, many trial and error loops (often costly). Applied can impact a few key areas of medicine and explore how to build end-to-end systems. Our discussion of computer vision focuses largely on medical imaging. In this paper, we address these two issues and apply the proposed model to visual navigation in conformity with generalizing in conformity with obtaining new goals (target-driven). To tackle the first issue, we advise an actor-critic mannequin whose coverage is a feature of the intention as much properly namely the present day state, which approves higher generalization. To tackle the second issue, we advocate the 3D scenes in environment indoor simulation is AI2-THOR framework, who provides a surrounding including tremendous with high-quality 3D scenes and a physics engine. Our framework allows agents according to receive actions and have interaction with objects. Hence, we are able to accumulate an enormous number of training samples successfully with sequential decision making based totally on the RL framework. Particularly, Healthcare and medicine stand to benefit immensely from deep learning because of the sheer volume of data being generated we used the behavioral cloning approach, who enables the active agent to storeroom an expert (or mentor) policy except for the utilization of reward function stability or generalizes across targets.

Download Full-text

Hyperspectral Image Classification with Capsule Network Using Limited Training Samples

Sensors ◽

10.3390/s18093153 ◽

2018 ◽

Vol 18 (9) ◽

pp. 3153 ◽

Cited By ~ 32

Author(s):

Fei Deng ◽

Shengliang Pu ◽

Xuehong Chen ◽

Yusheng Shi ◽

Ting Yuan ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Network Architecture ◽

Hyperspectral Image ◽

Machine Learning Algorithms ◽

Superior Performance ◽

Support Vector ◽

Complex Data ◽

Training Samples ◽

Limited Training Samples

Deep learning techniques have boosted the performance of hyperspectral image (HSI) classification. In particular, convolutional neural networks (CNNs) have shown superior performance to that of the conventional machine learning algorithms. Recently, a novel type of neural networks called capsule networks (CapsNets) was presented to improve the most advanced CNNs. In this paper, we present a modified two-layer CapsNet with limited training samples for HSI classification, which is inspired by the comparability and simplicity of the shallower deep learning models. The presented CapsNet is trained using two real HSI datasets, i.e., the PaviaU (PU) and SalinasA datasets, representing complex and simple datasets, respectively, and which are used to investigate the robustness or representation of every model or classifier. In addition, a comparable paradigm of network architecture design has been proposed for the comparison of CNN and CapsNet. Experiments demonstrate that CapsNet shows better accuracy and convergence behavior for the complex data than the state-of-the-art CNN. For CapsNet using the PU dataset, the Kappa coefficient, overall accuracy, and average accuracy are 0.9456, 95.90%, and 96.27%, respectively, compared to the corresponding values yielded by CNN of 0.9345, 95.11%, and 95.63%. Moreover, we observed that CapsNet has much higher confidence for the predicted probabilities. Subsequently, this finding was analyzed and discussed with probability maps and uncertainty analysis. In terms of the existing literature, CapsNet provides promising results and explicit merits in comparison with CNN and two baseline classifiers, i.e., random forests (RFs) and support vector machines (SVMs).

Download Full-text

DGA CapsNet: 1D Application of Capsule Networks to DGA Detection

Information ◽

10.3390/info10050157 ◽

2019 ◽

Vol 10 (5) ◽

pp. 157 ◽

Cited By ~ 5

Author(s):

Daniel S. Berman

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Real Time ◽

Convolutional Neural Networks ◽

Recurrent Neural Networks ◽

State Of The Art ◽

One Dimensional ◽

Domain Names ◽

Large Numbers ◽

And Control

Domain generation algorithms (DGAs) represent a class of malware used to generate large numbers of new domain names to achieve command-and-control (C2) communication between the malware program and its C2 server to avoid detection by cybersecurity measures. Deep learning has proven successful in serving as a mechanism to implement real-time DGA detection, specifically through the use of recurrent neural networks (RNNs) and convolutional neural networks (CNNs). This paper compares several state-of-the-art deep-learning implementations of DGA detection found in the literature with two novel models: a deeper CNN model and a one-dimensional (1D) Capsule Networks (CapsNet) model. The comparison shows that the 1D CapsNet model performs as well as the best-performing model from the literature.

Download Full-text

A Strategic Analytics Using Convolutional Neural Networks for Weed Identification in Sugar Beet Fields

Agris on-line Papers in Economics and Informatics ◽

10.7160/aol.2021.130104 ◽

2021 ◽

Vol 13 (1) ◽

pp. 49-57

Author(s):

Brahim Jabir ◽

Noureddine Falih ◽

Asmaa Sarih ◽

Adil Tannouche

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Sugar Beet ◽

Convolutional Neural Networks ◽

Precision Agriculture ◽

Large Scale ◽

Aerial Images ◽

Data Set ◽

High Level ◽

And Control

Researchers in precision agriculture regularly use deep learning that will help growers and farmers control and monitor crops during the growing season; these tools help to extract meaningful information from large-scale aerial images received from the field using several techniques in order to create a strategic analytics for making a decision. The information result of the operation could be exploited for many reasons, such as sub-plot specific weed control. Our focus in this paper is on weed identification and control in sugar beet fields, particularly the creation and optimization of a Convolutional Neural Networks model and train it according to our data set to predict and identify the most popular weed strains in the region of Beni Mellal, Morocco. All that could help select herbicides that work on the identified weeds, we explore the way of transfer learning approach to design the networks, and the famous library Tensorflow for deep learning models, and Keras which is a high-level API built on Tensorflow.

Download Full-text

Deep learning for strong lensing search: tests of the convolutional neural networks and new candidates from KiDS DR3

Monthly Notices of the Royal Astronomical Society ◽

10.1093/mnras/staa1917 ◽

2020 ◽

Vol 497 (1) ◽

pp. 556-571

Author(s):

Zizhao He ◽

Xinzhong Er ◽

Qian Long ◽

Dezi Liu ◽

Xiangkun Liu ◽

...

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Network Performance ◽

Training Sample ◽

The Other ◽

Training Set ◽

Luminous Red Galaxies ◽

Training Samples ◽

Strong Lensing ◽

Red Galaxies

ABSTRACT Convolutional neural networks have been successfully applied in searching for strong lensing systems, leading to discoveries of new candidates from large surveys. On the other hand, systematic investigations about their robustness are still lacking. In this paper, we first construct a neutral network, and apply it to r-band images of luminous red galaxies (LRGs) of the Kilo Degree Survey (KiDS) Data Release 3 to search for strong lensing systems. We build two sets of training samples, one fully from simulations, and the other one using the LRG stamps from KiDS observations as the foreground lens images. With the former training sample, we find 48 high probability candidates after human inspection, and among them, 27 are newly identified. Using the latter training set, about 67 per cent of the aforementioned 48 candidates are also found, and there are 11 more new strong lensing candidates identified. We then carry out tests on the robustness of the network performance with respect to the variation of PSF. With the testing samples constructed using PSF in the range of 0.4–2 times of the median PSF of the training sample, we find that our network performs rather stable, and the degradation is small. We also investigate how the volume of the training set can affect our network performance by varying it from 0.1 to 0.8 million. The output results are rather stable showing that within the considered range, our network performance is not very sensitive to the volume size.

Download Full-text

Counterfactual Autoencoder for Unsupervised Semantic Learning

Deep Learning and Neural Networks ◽

10.4018/978-1-7998-0414-7.ch040 ◽

2020 ◽

pp. 720-736

Author(s):

Saad Sadiq ◽

Mei-Ling Shyu ◽

Daniel J. Feaster

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Computer Vision ◽

Deep Learning ◽

Language Processing ◽

Speech Processing ◽

Semantic Learning ◽

Novel Approach ◽

Training Samples ◽

Latent Space

Deep Neural Networks (DNNs) are best known for being the state-of-the-art in artificial intelligence (AI) applications including natural language processing (NLP), speech processing, computer vision, etc. In spite of all recent achievements of deep learning, it has yet to achieve semantic learning required to reason about the data. This lack of reasoning is partially imputed to the boorish memorization of patterns and curves from millions of training samples and ignoring the spatiotemporal relationships. The proposed framework puts forward a novel approach based on variational autoencoders (VAEs) by using the potential outcomes model and developing the counterfactual autoencoders. The proposed framework transforms any sort of multimedia input distributions to a meaningful latent space while giving more control over how the latent space is created. This allows us to model data that is better suited to answer inference-based queries, which is very valuable in reasoning-based AI applications.

Download Full-text