scholarly journals Acoustic Scene Classification Using Multichannel Observation with Partially Missing Channels

2021 ◽  
Author(s):  
Keisuke Imoto
Keyword(s):  
Data Augmentation ◽  
Spatial Information ◽  
Microphone Array ◽  
Classification Performance ◽  
Scene Classification ◽  
Wind Noise ◽  
Sound Sources ◽  
Time Frequency ◽  
Multichannel Audio ◽  
The Impact

Sounds recorded with smartphones or IoT devices often have partially unreliable observations caused by clipping, wind noise, and completely missing parts due to microphone failure and packet loss in data transmission over the network. In this paper, we investigate the impact of the partially missing channels on the performance of acoustic scene classification using multichannel audio recordings, especially for a distributed microphone array. Missing observations cause not only losses of time-frequency and spatial information on sound sources but also a mismatch between a trained model and evaluation data. We thus investigate how a missing channel affects the performance of acoustic scene classification in detail. We also propose simple data augmentation methods for scene classification using multichannel observations with partially missing channels and evaluate the scene classification performance using the data augmentation methods.

scholarly journals Data Augmentation with Suboptimal Warping for Time-Series Classification

Sensors ◽  
2019 ◽  
Vol 20 (1) ◽  
pp. 98 ◽  
Author(s):  
Krzysztof Kamycki ◽  
Tomasz Kapuscinski ◽  
Mariusz Oszust
Keyword(s):  
Time Series ◽  
Data Augmentation ◽  
Nearest Neighbor ◽  
Multivariate Time Series ◽  
Metric Learning ◽  
Classification Performance ◽  
Training Dataset ◽  
Time Series Classification ◽  
Extensive Evaluation ◽  
The Impact

In this paper, a novel data augmentation method for time-series classification is proposed. In the introduced method, a new time-series is obtained in warped space between suboptimally aligned input examples of different lengths. Specifically, the alignment is carried out constraining the warping path and reducing its flexibility. It is shown that the resultant synthetic time-series can form new class boundaries and enrich the training dataset. In this work, the comparative evaluation of the proposed augmentation method against related techniques on representative multivariate time-series datasets is presented. The performance of methods is examined using the nearest neighbor classifier with the dynamic time warping (NN-DTW), LogDet divergence-based metric learning with triplet constraints (LDMLT), and the recently introduced time-series cluster kernel (NN-TCK). The impact of the augmentation on the classification performance is investigated, taking into account entire datasets and cases with a small number of training examples. The extensive evaluation reveals that the introduced method outperforms related augmentation algorithms in terms of the obtained classification accuracy.

Scene Classification Based on Improved Spatial Partition Model

2014 ◽  
Vol 527 ◽  
pp. 339-342
Author(s):  
Zhi Yuan Liu ◽  
Jin He ◽  
Jin Long Wang ◽  
Fei Zhao
Keyword(s):  
Spatial Information ◽  
Classification Performance ◽  
Image Features ◽  
Natural Scene ◽  
Scene Classification ◽  
Partition Model ◽  
Spatial Partition ◽  
Improved Model ◽  
Local Image

In order to make full use of the spatial information of images in the classification of natural scene, we use the spatial partition model. But mechanically space division caused the abuse of spatial information. So spatial partition model must be properly improved to make the different categories of images were more diversity, so that the classification performance is improved. In addition, to further improve the performance, we use FAN-SIFT as local image features. Experiments made on 8 scenes image dataset and Caltech101 dataset show that the improved model can obtain better classification performance.

scholarly journals Vision Transformers for Remote Sensing Image Classification

2021 ◽  
Vol 13 (3) ◽  
pp. 516
Author(s):  
Yakoub Bazi ◽  
Laila Bashmal ◽  
Mohamad M. Al Rahhal ◽  
Reham Al Dayil ◽  
Naif Al Ajlan
Keyword(s):  
Remote Sensing ◽  
Language Processing ◽  
Additional Data ◽  
Data Augmentation ◽  
State Of The Art ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Scene Classification ◽  
Remote Sensing Image Classification ◽  
Augmentation Strategies

In this paper, we propose a remote-sensing scene-classification method based on vision transformers. These types of networks, which are now recognized as state-of-the-art models in natural language processing, do not rely on convolution layers as in standard convolutional neural networks (CNNs). Instead, they use multihead attention mechanisms as the main building block to derive long-range contextual relation between pixels in images. In a first step, the images under analysis are divided into patches, then converted to sequence by flattening and embedding. To keep information about the position, embedding position is added to these patches. Then, the resulting sequence is fed to several multihead attention layers for generating the final representation. At the classification stage, the first token sequence is fed to a softmax classification layer. To boost the classification performance, we explore several data augmentation strategies to generate additional data for training. Moreover, we show experimentally that we can compress the network by pruning half of the layers while keeping competing classification accuracies. Experimental results conducted on different remote-sensing image datasets demonstrate the promising capability of the model compared to state-of-the-art methods. Specifically, Vision Transformer obtains an average classification accuracy of 98.49%, 95.86%, 95.56% and 93.83% on Merced, AID, Optimal31 and NWPU datasets, respectively. While the compressed version obtained by removing half of the multihead attention layers yields 97.90%, 94.27%, 95.30% and 93.05%, respectively.

scholarly journals Gastrointestinal Disease Classification in Endoscopic Images Using Attention-Guided Convolutional Neural Networks

2021 ◽  
Vol 11 (23) ◽  
pp. 11136
Author(s):  
Zenebe Markos Lonseko ◽  
Prince Ebenezer Adjei ◽  
Wenju Du ◽  
Chengsi Luo ◽  
Dingcan Hu ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Spatial Information ◽  
Gastrointestinal Disease ◽  
Confusion Matrix ◽  
Automatic Classification ◽  
Classification Performance ◽  
Attention Mechanism ◽  
Disease Classification ◽  
Matrix Analysis

Gastrointestinal (GI) diseases constitute a leading problem in the human digestive system. Consequently, several studies have explored automatic classification of GI diseases as a means of minimizing the burden on clinicians and improving patient outcomes, for both diagnostic and treatment purposes. The challenge in using deep learning-based (DL) approaches, specifically a convolutional neural network (CNN), is that spatial information is not fully utilized due to the inherent mechanism of CNNs. This paper proposes the application of spatial factors in improving classification performance. Specifically, we propose a deep CNN-based spatial attention mechanism for the classification of GI diseases, implemented with encoder–decoder layers. To overcome the data imbalance problem, we adapt data-augmentation techniques. A total of 12,147 multi-sited, multi-diseased GI images, drawn from publicly available and private sources, were used to validate the proposed approach. Furthermore, a five-fold cross-validation approach was adopted to minimize inconsistencies in intra- and inter-class variability and to ensure that results were robustly assessed. Our results, compared with other state-of-the-art models in terms of mean accuracy (ResNet50 = 90.28, GoogLeNet = 91.38, DenseNets = 91.60, and baseline = 92.84), demonstrated better outcomes (Precision = 92.8, Recall = 92.7, F1-score = 92.8, and Accuracy = 93.19). We also implemented t-distributed stochastic neighbor embedding (t–SNE) and confusion matrix analysis techniques for better visualization and performance validation. Overall, the results showed that the attention mechanism improved the automatic classification of multi-sited GI disease images. We validated clinical tests based on the proposed method by overcoming previous limitations, with the goal of improving automatic classification accuracy in future work.

scholarly journals An Underwater Acoustic Target Recognition Method Based on Spectrograms with Different Resolutions

2021 ◽  
Vol 9 (11) ◽  
pp. 1246
Author(s):  
Xinwei Luo ◽  
Minghong Zhang ◽  
Ting Liu ◽  
Ming Huang ◽  
Xiaogang Xu
Keyword(s):  
Data Augmentation ◽  
Target Recognition ◽  
Automatic Target Recognition ◽  
Classification Performance ◽  
Quality Data ◽  
Generative Adversarial Network ◽  
Underwater Acoustic ◽  
Time Frequency ◽  
Radiated Noise ◽  
Acoustic Target

This paper focuses on the automatic target recognition (ATR) method based on ship-radiated noise and proposes an underwater acoustic target recognition (UATR) method based on ResNet. In the proposed method, a multi-window spectral analysis (MWSA) method is used to solve the difficulty that the traditional time–frequency (T–F) analysis method has in extracting multiple signal characteristics simultaneously. MWSA generates spectrograms with different T–F resolutions through multiple window processing to provide input for the classifier. Because of the insufficient number of ship-radiated noise samples, a conditional deep convolutional generative adversarial network (cDCGAN) model was designed for high-quality data augmentation. Experimental results on real ship-radiated noise show that the proposed UATR method has good classification performance.

scholarly journals Remote Sensing Image Scene Classification Using CNN-CapsNet

2019 ◽  
Vol 11 (5) ◽  
pp. 494 ◽  
Author(s):  
Wei Zhang ◽  
Ping Tang ◽  
Lijun Zhao
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Network Architecture ◽  
Spatial Information ◽  
Feature Learning ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Scene Classification ◽  
Feature Maps ◽  
Fully Connected

Remote sensing image scene classification is one of the most challenging problems in understanding high-resolution remote sensing images. Deep learning techniques, especially the convolutional neural network (CNN), have improved the performance of remote sensing image scene classification due to the powerful perspective of feature learning and reasoning. However, several fully connected layers are always added to the end of CNN models, which is not efficient in capturing the hierarchical structure of the entities in the images and does not fully consider the spatial information that is important to classification. Fortunately, capsule network (CapsNet), which is a novel network architecture that uses a group of neurons as a capsule or vector to replace the neuron in the traditional neural network and can encode the properties and spatial information of features in an image to achieve equivariance, has become an active area in the classification field in the past two years. Motivated by this idea, this paper proposes an effective remote sensing image scene classification architecture named CNN-CapsNet to make full use of the merits of these two models: CNN and CapsNet. First, a CNN without fully connected layers is used as an initial feature maps extractor. In detail, a pretrained deep CNN model that was fully trained on the ImageNet dataset is selected as a feature extractor in this paper. Then, the initial feature maps are fed into a newly designed CapsNet to obtain the final classification result. The proposed architecture is extensively evaluated on three public challenging benchmark remote sensing image datasets: the UC Merced Land-Use dataset with 21 scene categories, AID dataset with 30 scene categories, and the NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that the proposed method can lead to a competitive classification performance compared with the state-of-the-art methods.

scholarly journals SpecMix : A Mixed Sample Data Augmentation method for Training with Time-Frequency Domain Features

2021 ◽  
Author(s):  
Gwantae Kim ◽  
David K. Han ◽  
Hanseok Ko
Keyword(s):  
Frequency Domain ◽  
Speech Enhancement ◽  
Data Augmentation ◽  
Scene Classification ◽  
Spectral Correlation ◽  
Event Classification ◽  
Time Frequency ◽  
Mixed Sample ◽  
Sound Event ◽  
Sample Data

A mixed sample data augmentation strategy is proposed to enhance the performance of models on audio scene classification, sound event classification, and speech enhancement tasks. While there have been several augmentation methods shown to be effective in improving image classification performance, their efficacy toward time-frequency domain features of audio is not assured. We propose a novel audio data augmentation approach named "Specmix" specifically designed for dealing with time-frequency domain features. The augmentation method consists of mixing two different data samples by applying time-frequency masks effective in preserving the spectral correlation of each audio sample. Our experiments on acoustic scene classification, sound event classification, and speech enhancement tasks show that the proposed Specmix improves the performance of various neural network architectures by a maximum of 2.7\%.

scholarly journals A Discriminative Neighborhood-Based Collaborative Learning for Remote Sensing Scene Classification

2021 ◽  
Author(s):  
Usman Muhammad ◽  
Md. Ziaul Hoque ◽  
Weiqiang Wang ◽  
Mourad Oussalah
Keyword(s):  
Remote Sensing ◽  
Collaborative Learning ◽  
Visual Information ◽  
Spatial Information ◽  
Short Term Memory ◽  
Classification Performance ◽  
Bag Of Words ◽  
Scene Classification ◽  
Image Region ◽  
Local Descriptor

The bag-of-words (BoW) model is one of the most popular representation methods for image classification. However, the lack of spatial information, change of illumination, and inter-class similarity among scene categories impair its performance in the remote-sensing domain. To alleviate these issues, this paper proposes to explore the spatial dependencies between different image regions and introduce a neighborhood-based collaborative learning (NBCL) for remote-sensing scene classification. Particularly, our proposed method employs multilevel features learning based on small, medium, and large neighborhood regions to enhance the discriminative power of image representation. To achieve this, image patches are selected through a fixed-size sliding window where each image is represented by four independent image region sequences. Apart from multilevel learning, we explicitly impose Gaussian pyramids to magnify the visual information of the scene images and optimize their position and scale parameters locally. Motivated by this, a local descriptor is exploited to extract multilevel and multiscale features that we represent in terms of codewords histogram by performing k-means clustering. Finally, a simple fusion strategy is proposed to balance the contribution of these features, and the fused features are incorporated into a Bidirectional Long Short-Term Memory (BiLSTM) network for constructing the final representation for classification. Experimental results on NWPU-RESISC45, AID, UC-Merced, and WHU-RS datasets demonstrate that the proposed approach not only surpasses the conventional bag-of-words approaches but also yields significantly higher classification performance than the existing state-of-the-art deep learning methods used nowadays.

scholarly journals A Discriminative Neighborhood-Based Collaborative Learning for Remote Sensing Scene Classification

2021 ◽  
Author(s):  
Usman Muhammad ◽  
Md. Ziaul Hoque ◽  
Weiqiang Wang ◽  
Mourad Oussalah
Keyword(s):  
Remote Sensing ◽  
Collaborative Learning ◽  
Visual Information ◽  
Spatial Information ◽  
Short Term Memory ◽  
Classification Performance ◽  
Bag Of Words ◽  
Scene Classification ◽  
Image Region ◽  
Local Descriptor

The bag-of-words (BoW) model is one of the most popular representation methods for image classification. However, the lack of spatial information, change of illumination, and inter-class similarity among scene categories impair its performance in the remote-sensing domain. To alleviate these issues, this paper proposes to explore the spatial dependencies between different image regions and introduce a neighborhood-based collaborative learning (NBCL) for remote-sensing scene classification. Particularly, our proposed method employs multilevel features learning based on small, medium, and large neighborhood regions to enhance the discriminative power of image representation. To achieve this, image patches are selected through a fixed-size sliding window where each image is represented by four independent image region sequences. Apart from multilevel learning, we explicitly impose Gaussian pyramids to magnify the visual information of the scene images and optimize their position and scale parameters locally. Motivated by this, a local descriptor is exploited to extract multilevel and multiscale features that we represent in terms of codewords histogram by performing k-means clustering. Finally, a simple fusion strategy is proposed to balance the contribution of these features, and the fused features are incorporated into a Bidirectional Long Short-Term Memory (BiLSTM) network for constructing the final representation for classification. Experimental results on NWPU-RESISC45, AID, UC-Merced, and WHU-RS datasets demonstrate that the proposed approach not only surpasses the conventional bag-of-words approaches but also yields significantly higher classification performance than the existing state-of-the-art deep learning methods used nowadays.

scholarly journals Rotation Invariance Regularization for Remote Sensing Image Scene Classification with Convolutional Neural Networks

2021 ◽  
Vol 13 (4) ◽  
pp. 569
Author(s):  
Kunlun Qi ◽  
Chao Yang ◽  
Chuli Hu ◽  
Yonglin Shen ◽  
Shengyu Shen ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Data Augmentation ◽  
Remote Sensing Image ◽  
Classification Performance ◽  
Rotation Invariance ◽  
Scene Classification ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Network

Deep convolutional neural networks (DCNNs) have shown significant improvements in remote sensing image scene classification for powerful feature representations. However, because of the high variance and volume limitations of the available remote sensing datasets, DCNNs are prone to overfit the data used for their training. To address this problem, this paper proposes a novel scene classification framework based on a deep Siamese convolutional network with rotation invariance regularization. Specifically, we design a data augmentation strategy for the Siamese model to learn a rotation invariance DCNN model that is achieved by directly enforcing the labels of the training samples before and after rotating to be mapped close to each other. In addition to the cross-entropy cost function for the traditional CNN models, we impose a rotation invariance regularization constraint on the objective function of our proposed model. The experimental results obtained using three publicly-available scene classification datasets show that the proposed method can generally improve the classification performance by 2~3% and achieves satisfactory classification performance compared with some state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document