scholarly journals Early-fusion based pulsar identification with smart under-sampling

2021 ◽  
Vol 21 (10) ◽  
pp. 257
Author(s):  
Shi-Chuan Zhang ◽  
Xiang-Cong Kong ◽  
Yue-Ying Zhou ◽  
Ling-Yao Chen ◽  
Xiao-Ying Zheng ◽  
...  
Keyword(s):  
State Of The Art ◽  
Training Data ◽  
High Time Resolution ◽  
Negative Sample ◽  
Fusion Strategy ◽  
Training Time ◽  
Learning Framework ◽  
Early Fusion ◽  
Under Sampling ◽  
Speed Up

Abstract The discovery of pulsars is of great significance in the field of physics and astronomy. As the astronomical equipment produces a large number of pulsar data, an algorithm for automatically identifying pulsars becomes urgent. We propose a deep learning framework for pulsar recognition. In response to the extreme imbalance between positive and negative examples and the hard negative sample issue presented in the High Time Resolution Universe Medlat Training Data, there are two coping strategies in our framework: the smart under-sampling and the improved loss function. We also apply the early-fusion strategy to integrate features obtained from different attributes before classification to improve the performance. To our best knowledge, this is the first study that integrates these strategies and techniques in pulsar recognition. The experiment results show that our framework outperforms previous works with respect to either the training time or F1 score. We can not only speed up the training time by 10 × compared with the state-of-the-art work, but also get a competitive result in terms of F1 score.

scholarly journals DRHNet: A Deep Residual Network Based on Heterogeneous Kernel for Steganalysis

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Yang Xu ◽  
Zixi Fu ◽  
Guiyong Xu ◽  
Sicong Zhang ◽  
Xiaoyao Xie
Keyword(s):  
Neural Networks ◽  
Heterogeneous Network ◽  
State Of The Art ◽  
Training Phase ◽  
Image Size ◽  
Residual Network ◽  
Training Time ◽  
Learning Framework ◽  
Residual Learning ◽  
The Rich

Convolutional neural networks as steganalysis have problems such as poor versatility, long training time, and limited image size. For these problems, we present a heterogeneous kernel residual learning framework called DRHNet—Dual Residual Heterogeneous Network—to save time on the networks during the training phase. Instead of using the image as an input of the network, we extract and merge the images into a feature matrix using the rich model and use the generated feature matrix as the real input of the network. The architecture we proposed has good versatility and can reduce the computation and the number of parameters while still getting higher accuracy. On BOSSbase 1.01, we evaluate the performance of DRHNet in the setting of the spatial domain and frequency domain. The preliminary experimental results show that DRHNet shows excellent steganalysis performance against the state-of-the-art steganographic algorithms.

scholarly journals A Two-Stream Mutual Attention Network for Semi-Supervised Biomedical Segmentation with Noisy Labels

2019 ◽  
Vol 33 ◽  
pp. 4578-4585 ◽  
Author(s):  
Shaobo Min ◽  
Xuejin Chen ◽  
Zheng-Jun Zha ◽  
Feng Wu ◽  
Yongdong Zhang
Keyword(s):  
Supervised Learning ◽  
State Of The Art ◽  
Training Data ◽  
Attention Network ◽  
Attention Model ◽  
Learning Framework ◽  
Propagation Analysis ◽  
Supervised Methods ◽  
Multi Level ◽  
Noisy Labels

Learning-based methods suffer from a deficiency of clean annotations, especially in biomedical segmentation. Although many semi-supervised methods have been proposed to provide extra training data, automatically generated labels are usually too noisy to retrain models effectively. In this paper, we propose a Two-Stream Mutual Attention Network (TSMAN) that weakens the influence of back-propagated gradients caused by incorrect labels, thereby rendering the network robust to unclean data. The proposed TSMAN consists of two sub-networks that are connected by three types of attention models in different layers. The target of each attention model is to indicate potentially incorrect gradients in a certain layer for both sub-networks by analyzing their inferred features using the same input. In order to achieve this purpose, the attention models are designed based on the propagation analysis of noisy gradients at different layers. This allows the attention models to effectively discover incorrect labels and weaken their influence during parameter updating process. By exchanging multi-level features within two-stream architecture, the effects of noisy labels in each sub-network are reduced by decreasing the noisy gradients. Furthermore, a hierarchical distillation is developed to provide reliable pseudo labels for unlabelded data, which further boosts the performance of TSMAN. The experiments using both HVSMR 2016 and BRATS 2015 benchmarks demonstrate that our semi-supervised learning framework surpasses the state-of-the-art fully-supervised results.

scholarly journals Learning from Web Data Using Adversarial Discriminative Neural Networks for Fine-Grained Classification

2019 ◽  
Vol 33 ◽  
pp. 273-280 ◽  
Author(s):  
Xiaoxiao Sun ◽  
Liyi Chen ◽  
Jufeng Yang
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
State Of The Art ◽  
Training Data ◽  
Web Data ◽  
Fine Grained ◽  
Learning Framework ◽  
Attractive Option ◽  
Public Datasets ◽  
Noisy Labels

Fine-grained classification is absorbed in recognizing the subordinate categories of one field, which need a large number of labeled images, while it is expensive to label these images. Utilizing web data has been an attractive option to meet the demands of training data for convolutional neural networks (CNNs), especially when the well-labeled data is not enough. However, directly training on such easily obtained images often leads to unsatisfactory performance due to factors such as noisy labels. This has been conventionally addressed by reducing the noise level of web data. In this paper, we take a fundamentally different view and propose an adversarial discriminative loss to advocate representation coherence between standard and web data. This is further encapsulated in a simple, scalable and end-to-end trainable multi-task learning framework. We experiment on three public datasets using large-scale web data to evaluate the effectiveness and generalizability of the proposed approach. Extensive experiments demonstrate that our approach performs favorably against the state-of-the-art methods.

scholarly journals CAWA: An Attention-Network for Credit Attribution

2020 ◽  
Vol 34 (05) ◽  
pp. 8472-8479
Author(s):  
Saurav Manchanda ◽  
George Karypis
Keyword(s):  
State Of The Art ◽  
Text Summarization ◽  
Training Data ◽  
Learning Framework ◽  
Distant Supervision ◽  
Sentence Level ◽  
Class Labels ◽  
The Individual ◽  
Traditional Approaches ◽  
Better Than

Credit attribution is the task of associating individual parts in a document with their most appropriate class labels. It is an important task with applications to information retrieval and text summarization. When labeled training data is available, traditional approaches for sequence tagging can be used for credit attribution. However, generating such labeled datasets is expensive and time-consuming. In this paper, we present Credit Attribution With Attention (CAWA), a neural-network-based approach, that instead of using sentence-level labeled data, uses the set of class labels that are associated with an entire document as a source of distant-supervision. CAWA combines an attention mechanism with a multilabel classifier into an end-to-end learning framework to perform credit attribution. CAWA labels the individual sentences from the input document using the resultant attention-weights. CAWA improves upon the state-of-the-art credit attribution approach by not constraining a sentence to belong to just one class, but modeling each sentence as a distribution over all classes, leading to better modeling of semantically-similar classes. Experiments on the credit attribution task on a variety of datasets show that the sentence class labels generated by CAWA outperform the competing approaches. Additionally, on the multilabel text classification task, CAWA performs better than the competing credit attribution approaches1.

scholarly journals Fast SVM Trained by Divide-and-Conquer Anchors

2017 ◽  
Author(s):  
Meng Liu ◽  
Chang Xu ◽  
Chao Xu ◽  
Dacheng Tao
Keyword(s):  
Matrix Factorization ◽  
State Of The Art ◽  
Training Data ◽  
Divide And Conquer ◽  
Training Time ◽  
Learning Tasks ◽  
Multiple Datasets ◽  
Original Dataset ◽  
Representative Points ◽  
Non Negative Matrix Factorization

Supporting vector machine (SVM) is the most frequently used classifier for machine learning tasks. However, its training time could become cumbersome when the size of training data is very large. Thus, many kinds of representative subsets are chosen from the original dataset to reduce the training complexity. In this paper, we propose to choose the representative points which are noted as anchors obtained from non-negative matrix factorization (NMF) in a divide-and-conquer framework, and then use the anchors to train an approximate SVM. Our theoretical analysis shows that the solving the DCA-SVM can yield an approximate solution close to the primal SVM. Experimental results on multiple datasets demonstrate that our DCA-SVM is faster than the state-of-the-art algorithms without notably decreasing the accuracy of classification results.

scholarly journals An Optimization-Based Meta-Learning Model for MRI Reconstruction with Diverse Dataset

2021 ◽  
Author(s):  
Wanyu Bian ◽  
Yunmei Chen ◽  
Xiaojing Ye ◽  
Qingchao Zhang
Keyword(s):  
Network Architecture ◽  
Bilevel Optimization ◽  
Training Data ◽  
Training Dataset ◽  
Variational Model ◽  
Validation Dataset ◽  
Training Time ◽  
Learning Framework ◽  
Meta Learning ◽  
Mri Reconstruction

(1) Purpose: This work aims at developing a generalizable MRI reconstruction model in the meta-learning framework. The standard benchmarks in meta-learning are challenged by learning on diverse task distributions. The proposed network learns the regularization function in a variational model and reconstructs MR images with various under-sampling ratios or patterns that may or may not be seen in the training data by leveraging a heterogeneous dataset. (2) Methods: We propose an unrolling network induced by learnable optimization algorithms (LOA) for solving our nonconvex nonsmooth variational model for MRI reconstruction. In this model, the learnable regularization function contains a task-invariant common feature encoder and task-specific learner represented by a shallow network. To train the network we split the training data into two parts: training and validation, and introduce a bilevel optimization algorithm. The lower-level optimization trains task-invariant parameters for the feature encoder with fixed parameters of the task-specific learner on the training dataset, and the upper-level optimizes the parameters of the task-specific learner on the validation dataset. (3) Results: The PSNR increases 1.5 dB on average compared to the network trained through conventional supervised learning on the seen CS ratios. We test the result of quick adaption on the unseen tasks after meta-training, the average PSNR arises 1.22 dB compared to the conventional learning procedure that is directly trained on the unseen CS ratios in the meanwhile saving half of the training time. The average PSNR arises 1.87 dB for unseen sampling patterns comparing to conventional learning; (4) Conclusion: We proposed a meta-learning framework consisting of the base network architecture, design of regularization, and bi-level optimization-based training. The network inherits the convergence property of the LOA and interpretation of the variational model. The generalization ability is improved by the designated regularization and bilevel optimization-based training algorithm.

scholarly journals RS-SSKD: Self-Supervision Equipped with Knowledge Distillation for Few-Shot Remote Sensing Scene Classification

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1566
Author(s):  
Pei Zhang ◽  
Ying Li ◽  
Dong Wang ◽  
Jiyue Wang
Keyword(s):  
Remote Sensing ◽  
State Of The Art ◽  
Ground Truth ◽  
Training Data ◽  
Scene Classification ◽  
Training Time ◽  
Shot Classification ◽  
Meta Learning ◽  
Knowledge Distillation ◽  
Few Data

While growing instruments generate more and more airborne or satellite images, the bottleneck in remote sensing (RS) scene classification has shifted from data limits toward a lack of ground truth samples. There are still many challenges when we are facing unknown environments, especially those with insufficient training data. Few-shot classification offers a different picture under the umbrella of meta-learning: digging rich knowledge from a few data are possible. In this work, we propose a method named RS-SSKD for few-shot RS scene classification from a perspective of generating powerful representation for the downstream meta-learner. Firstly, we propose a novel two-branch network that takes three pairs of original-transformed images as inputs and incorporates Class Activation Maps (CAMs) to drive the network mining, the most relevant category-specific region. This strategy ensures that the network generates discriminative embeddings. Secondly, we set a round of self-knowledge distillation to prevent overfitting and boost the performance. Our experiments show that the proposed method surpasses current state-of-the-art approaches on two challenging RS scene datasets: NWPU-RESISC45 and RSD46-WHU. Finally, we conduct various ablation experiments to investigate the effect of each component of the proposed method and analyze the training time of state-of-the-art methods and ours.

Review on biomass feedstocks, pyrolysis mechanism and physicochemical properties of biochar: State-of-the-art framework to speed up vision of circular bioeconomy

2021 ◽  
Vol 297 ◽  
pp. 126645
Author(s):  
Gajanan Sampatrao Ghodake ◽  
Surendra Krushna Shinde ◽  
Avinash Ashok Kadam ◽  
Rijuta Ganesh Saratale ◽  
Ganesh Dattatraya Saratale ◽  
...  
Keyword(s):  
Physicochemical Properties ◽  
State Of The Art ◽  
Pyrolysis Mechanism ◽  
Biomass Feedstocks ◽  
Speed Up

scholarly journals SCOUR: a stepwise machine learning framework for predicting metabolite-dependent regulatory interactions

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Justin Y. Lee ◽  
Britney Nguyen ◽  
Carlos Orosco ◽  
Mark P. Styczynski
Keyword(s):  
Machine Learning ◽  
Metabolic Networks ◽  
Sampling Frequency ◽  
Low Noise ◽  
Training Data ◽  
High Noise ◽  
Regulatory Interactions ◽  
Learning Framework ◽  
Metabolic Systems ◽  
Noise Data

Abstract Background The topology of metabolic networks is both well-studied and remarkably well-conserved across many species. The regulation of these networks, however, is much more poorly characterized, though it is known to be divergent across organisms—two characteristics that make it difficult to model metabolic networks accurately. While many computational methods have been built to unravel transcriptional regulation, there have been few approaches developed for systems-scale analysis and study of metabolic regulation. Here, we present a stepwise machine learning framework that applies established algorithms to identify regulatory interactions in metabolic systems based on metabolic data: stepwise classification of unknown regulation, or SCOUR. Results We evaluated our framework on both noiseless and noisy data, using several models of varying sizes and topologies to show that our approach is generalizable. We found that, when testing on data under the most realistic conditions (low sampling frequency and high noise), SCOUR could identify reaction fluxes controlled only by the concentration of a single metabolite (its primary substrate) with high accuracy. The positive predictive value (PPV) for identifying reactions controlled by the concentration of two metabolites ranged from 32 to 88% for noiseless data, 9.2 to 49% for either low sampling frequency/low noise or high sampling frequency/high noise data, and 6.6–27% for low sampling frequency/high noise data, with results typically sufficiently high for lab validation to be a practical endeavor. While the PPVs for reactions controlled by three metabolites were lower, they were still in most cases significantly better than random classification. Conclusions SCOUR uses a novel approach to synthetically generate the training data needed to identify regulators of reaction fluxes in a given metabolic system, enabling metabolomics and fluxomics data to be leveraged for regulatory structure inference. By identifying and triaging the most likely candidate regulatory interactions, SCOUR can drastically reduce the amount of time needed to identify and experimentally validate metabolic regulatory interactions. As high-throughput experimental methods for testing these interactions are further developed, SCOUR will provide critical impact in the development of predictive metabolic models in new organisms and pathways.

scholarly journals Nodule Detection with Convolutional Neural Network Using Apache Spark and GPU Frameworks

2021 ◽  
Vol 11 (6) ◽  
pp. 2838
Author(s):  
Nikitha Johnsirani Venkatesan ◽  
Dong Ryeol Shin ◽  
Choon Sung Nam
Keyword(s):  
Neural Network ◽  
Radiation Dose ◽  
Convolutional Neural Network ◽  
Model Performance ◽  
Performance Comparison ◽  
Apache Spark ◽  
Training Time ◽  
Learning Framework ◽  
Proposed Model

In the pharmaceutical field, early detection of lung nodules is indispensable for increasing patient survival. We can enhance the quality of the medical images by intensifying the radiation dose. High radiation dose provokes cancer, which forces experts to use limited radiation. Using abrupt radiation generates noise in CT scans. We propose an optimal Convolutional Neural Network model in which Gaussian noise is removed for better classification and increased training accuracy. Experimental demonstration on the LUNA16 dataset of size 160 GB shows that our proposed method exhibit superior results. Classification accuracy, specificity, sensitivity, Precision, Recall, F1 measurement, and area under the ROC curve (AUC) of the model performance are taken as evaluation metrics. We conducted a performance comparison of our proposed model on numerous platforms, like Apache Spark, GPU, and CPU, to depreciate the training time without compromising the accuracy percentage. Our results show that Apache Spark, integrated with a deep learning framework, is suitable for parallel training computation with high accuracy.

Sign in / Sign up

Export Citation Format

Share Document