scholarly journals Hyperspectral and LiDAR Fusion Using Deep Three-Stream Convolutional Neural Networks

2018 ◽  
Vol 10 (10) ◽  
pp. 1649 ◽  
Author(s):  
Hao Li ◽  
Pedram Ghamisi ◽  
Uwe Soergel ◽  
Xiao Zhu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Rural Areas ◽  
Remote Sensing Data ◽  
Classification Performance ◽  
Sensor Data ◽  
Support Vector ◽  
Data Sets ◽  
Data Set ◽  
Multi Sensor Data Fusion

Recently, convolutional neural networks (CNN) have been intensively investigated for the classification of remote sensing data by extracting invariant and abstract features suitable for classification. In this paper, a novel framework is proposed for the fusion of hyperspectral images and LiDAR-derived elevation data based on CNN and composite kernels. First, extinction profiles are applied to both data sources in order to extract spatial and elevation features from hyperspectral and LiDAR-derived data, respectively. Second, a three-stream CNN is designed to extract informative spectral, spatial, and elevation features individually from both available sources. The combination of extinction profiles and CNN features enables us to jointly benefit from low-level and high-level features to improve classification performance. To fuse the heterogeneous spectral, spatial, and elevation features extracted by CNN, instead of a simple stacking strategy, a multi-sensor composite kernels (MCK) scheme is designed. This scheme helps us to achieve higher spectral, spatial, and elevation separability of the extracted features and effectively perform multi-sensor data fusion in kernel space. In this context, a support vector machine and extreme learning machine with their composite kernels version are employed to produce the final classification result. The proposed framework is carried out on two widely used data sets with different characteristics: an urban data set captured over Houston, USA, and a rural data set captured over Trento, Italy. The proposed framework yields the highest OA of 92 . 57 % and 97 . 91 % for Houston and Trento data sets. Experimental results confirm that the proposed fusion framework can produce competitive results in both urban and rural areas in terms of classification accuracy, and significantly mitigate the salt and pepper noise in classification maps.

scholarly journals Deep Convolutional Neural Networks for Hyperspectral Image Classification

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Wei Hu ◽  
Yangyu Huang ◽  
Li Wei ◽  
Fan Zhang ◽  
Hengchao Li
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Hyperspectral Image ◽  
Image Data ◽  
Classification Performance ◽  
Spectral Signature ◽  
Support Vector ◽  
Data Sets ◽  
Deep Convolutional Neural Networks ◽  
Visual Tasks

Recently, convolutional neural networks have demonstrated excellent performance on various visual tasks, including the classification of common two-dimensional images. In this paper, deep convolutional neural networks are employed to classify hyperspectral images directly in spectral domain. More specifically, the architecture of the proposed classifier contains five layers with weights which are the input layer, the convolutional layer, the max pooling layer, the full connection layer, and the output layer. These five layers are implemented on each spectral signature to discriminate against others. Experimental results based on several hyperspectral image data sets demonstrate that the proposed method can achieve better classification performance than some traditional methods, such as support vector machines and the conventional deep learning-based methods.

scholarly journals Can Pre-Trained Convolutional Neural Networks be directly used as a Feature Extractor for Video-based Neonatal Sleep and Wake Classification?

2020 ◽  
Author(s):  
Muhammad Awais ◽  
Xi Long ◽  
Bin Yin ◽  
Chen chen ◽  
Saeed Akbarzadeh ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Video Recording ◽  
Principal Component ◽  
Classification Performance ◽  
Support Vector ◽  
Video Frames ◽  
Feature Extractor ◽  
Fully Connected

Abstract Objective: In this paper, we propose to evaluate the use of a pre-trained convolutional neural networks (CNNs) as a features extractor followed by the Principal Component Analysis (PCA) to find the best discriminant features to perform classification using support vector machine (SVM) algorithm for neonatal sleep and wake states using Fluke® facial video frames. Using pre-trained CNNs as feature extractor would hugely reduce the effort of collecting new neonatal data for training a neural network which could be computationally very expensive. The features are extracted after fully connected layers (FCL’s), where we compare several pre-trained CNNs, e.g., VGG16, VGG19, InceptionV3, GoogLeNet, ResNet, and AlexNet. Results: From around 2-h Fluke® video recording of seven neonate, we achieved a modest classification performance with an accuracy, sensitivity, and specificity of 65.3%, 69.8%, 61.0%, respectively with AlexNet using Fluke® (RGB) video frames. This indicates that using a pre-trained model as a feature extractor could not fully suffice for highly reliable sleep and wake classification in neonates. Therefore, in future a dedicated neural network trained on neonatal data or a transfer learning approach is required.

scholarly journals Evaluation of machine learning algorithms for classification of primary biological aerosol using a new UV-LIF spectrometer

2017 ◽  
Vol 10 (2) ◽  
pp. 695-708 ◽  
Author(s):  
Simon Ruske ◽  
David O. Topping ◽  
Virginia E. Foot ◽  
Paul H. Kaye ◽  
Warren R. Stanley ◽  
...  
Keyword(s):  
Neural Networks ◽  
Decision Trees ◽  
Supervised Learning ◽  
Ensemble Methods ◽  
Gradient Boosting ◽  
Support Vector ◽  
Data Sets ◽  
Data Set ◽  
Shape Information ◽  
Accuracy Of Measurements

Abstract. Characterisation of bioaerosols has important implications within environment and public health sectors. Recent developments in ultraviolet light-induced fluorescence (UV-LIF) detectors such as the Wideband Integrated Bioaerosol Spectrometer (WIBS) and the newly introduced Multiparameter Bioaerosol Spectrometer (MBS) have allowed for the real-time collection of fluorescence, size and morphology measurements for the purpose of discriminating between bacteria, fungal spores and pollen.This new generation of instruments has enabled ever larger data sets to be compiled with the aim of studying more complex environments. In real world data sets, particularly those from an urban environment, the population may be dominated by non-biological fluorescent interferents, bringing into question the accuracy of measurements of quantities such as concentrations. It is therefore imperative that we validate the performance of different algorithms which can be used for the task of classification.For unsupervised learning we tested hierarchical agglomerative clustering with various different linkages. For supervised learning, 11 methods were tested, including decision trees, ensemble methods (random forests, gradient boosting and AdaBoost), two implementations for support vector machines (libsvm and liblinear) and Gaussian methods (Gaussian naïve Bayesian, quadratic and linear discriminant analysis, the k-nearest neighbours algorithm and artificial neural networks).The methods were applied to two different data sets produced using the new MBS, which provides multichannel UV-LIF fluorescence signatures for single airborne biological particles. The first data set contained mixed PSLs and the second contained a variety of laboratory-generated aerosol.Clustering in general performs slightly worse than the supervised learning methods, correctly classifying, at best, only 67. 6 and 91. 1 % for the two data sets respectively. For supervised learning the gradient boosting algorithm was found to be the most effective, on average correctly classifying 82. 8 and 98. 27 % of the testing data, respectively, across the two data sets.A possible alternative to gradient boosting is neural networks. We do however note that this method requires much more user input than the other methods, and we suggest that further research should be conducted using this method, especially using parallelised hardware such as the GPU, which would allow for larger networks to be trained, which could possibly yield better results.We also saw that some methods, such as clustering, failed to utilise the additional shape information provided by the instrument, whilst for others, such as the decision trees, ensemble methods and neural networks, improved performance could be attained with the inclusion of such information.

scholarly journals Can pre-trained convolutional neural networks be directly used as a feature extractor for video-based neonatal sleep and wake classification?

2020 ◽  
Vol 13 (1) ◽  
Author(s):  
Muhammad Awais ◽  
Xi Long ◽  
Bin Yin ◽  
Chen Chen ◽  
Saeed Akbarzadeh ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Video Recording ◽  
Principal Component ◽  
Classification Performance ◽  
Support Vector ◽  
Video Frames ◽  
Feature Extractor ◽  
Future Work

Abstract Objective In this paper, we propose to evaluate the use of pre-trained convolutional neural networks (CNNs) as a features extractor followed by the Principal Component Analysis (PCA) to find the best discriminant features to perform classification using support vector machine (SVM) algorithm for neonatal sleep and wake states using Fluke® facial video frames. Using pre-trained CNNs as a feature extractor would hugely reduce the effort of collecting new neonatal data for training a neural network which could be computationally expensive. The features are extracted after fully connected layers (FCL’s), where we compare several pre-trained CNNs, e.g., VGG16, VGG19, InceptionV3, GoogLeNet, ResNet, and AlexNet. Results From around 2-h Fluke® video recording of seven neonates, we achieved a modest classification performance with an accuracy, sensitivity, and specificity of 65.3%, 69.8%, 61.0%, respectively with AlexNet using Fluke® (RGB) video frames. This indicates that using a pre-trained model as a feature extractor could not fully suffice for highly reliable sleep and wake classification in neonates. Therefore, in future work a dedicated neural network trained on neonatal data or a transfer learning approach is required.

scholarly journals Improving Classification with CNNs using Wavelet Pooling with Nesterov-Accelerated Adam

10.29007/9c5j ◽  
2019 ◽  
Author(s):  
Allison Rossetto ◽  
Wenjin Zhou
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Calculation Method ◽  
Classification Accuracy ◽  
Lung Tumor ◽  
Haar Wavelet ◽  
Data Sets ◽  
Data Set ◽  
Gradient Calculation ◽  
Shannon Wavelet

Wavelet pooling methods can improve the classification accuracy of Convolutional Neural Networks (CNNs). Combining wavelet pooling with the Nesterov-accelerated Adam (NAdam) gradient calculation method can improve both the accuracy of the CNN. We have implemented wavelet pooling with NAdam in this work using both a Haar wavelet (WavPool-NH) and a Shannon wavelet (WavPool-NS). The WavPool-NH and WavPool- NS methods are most accurate of the methods we considered for the MNIST and LIDC- IDRI lung tumor data-sets. The WavPool-NH and WavPool-NS implementations have an accuracy of 95.92% and 95.52%, respectively, on the LIDC-IDRI data-set. This is an improvement from the 92.93% accuracy obtained on this data-set with the max pooling method. The WavPool methods also avoid overfitting which is a concern with max pool- ing. We also found WavPool performed fairly well on the CIFAR-10 data-set, however, overfitting was an issue with all the methods we considered. Wavelet pooling, especially when combined with an adaptive gradient and wavelets chosen specifically for the data, has the potential to outperform current methods.

Convolutional Neural Networks for Water Body Extraction from Landsat Imagery

2017 ◽  
Vol 16 (01) ◽  
pp. 1750001 ◽  
Author(s):  
Long Yu ◽  
Zhiyin Wang ◽  
Shengwei Tian ◽  
Feiyue Ye ◽  
Jianli Ding ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Water Body ◽  
Spatial Information ◽  
Landsat Imagery ◽  
Classification Performance ◽  
Support Vector ◽  
Learning Methods ◽  
Wide Range

Traditional machine learning methods for water body extraction need complex spectral analysis and feature selection which rely on wealth of prior knowledge. They are time-consuming and hard to satisfy our request for accuracy, automation level and a wide range of application. We present a novel deep learning framework for water body extraction from Landsat imagery considering both its spectral and spatial information. The framework is a hybrid of convolutional neural networks (CNN) and logistic regression (LR) classifier. CNN, one of the deep learning methods, has acquired great achievements on various visual-related tasks. CNN can hierarchically extract deep features from raw images directly, and distill the spectral–spatial regularities of input data, thus improving the classification performance. Experimental results based on three Landsat imagery datasets show that our proposed model achieves better performance than support vector machine (SVM) and artificial neural network (ANN).

scholarly journals GeoBoost: An Incremental Deep Learning Approach toward Global Mapping of Buildings from VHR Remote Sensing Images

2020 ◽  
Vol 12 (11) ◽  
pp. 1794
Author(s):  
Naisen Yang ◽  
Hong Tang
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Large Scale ◽  
Semantic Segmentation ◽  
Data Sets ◽  
Data Set ◽  
Large Scale Data ◽  
Learning Tasks ◽  
Global Mapping ◽  
Scale Data

Modern convolutional neural networks (CNNs) are often trained on pre-set data sets with a fixed size. As for the large-scale applications of satellite images, for example, global or regional mappings, these images are collected incrementally by multiple stages in general. In other words, the sizes of training datasets might be increased for the tasks of mapping rather than be fixed beforehand. In this paper, we present a novel algorithm, called GeoBoost, for the incremental-learning tasks of semantic segmentation via convolutional neural networks. Specifically, the GeoBoost algorithm is trained in an end-to-end manner on the newly available data, and it does not decrease the performance of previously trained models. The effectiveness of the GeoBoost algorithm is verified on the large-scale data set of DREAM-B. This method avoids the need for training on the enlarged data set from scratch and would become more effective along with more available data.

scholarly journals Method for Training Convolutional Neural Networks for In Situ Plankton Image Recognition and Classification Based on the Mechanisms of the Human Eye

Sensors ◽  
2020 ◽  
Vol 20 (9) ◽  
pp. 2592
Author(s):  
Xuemin Cheng ◽  
Yong Ren ◽  
Kaichang Cheng ◽  
Jie Cao ◽  
Qun Hao
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Classification Accuracy ◽  
Image Data ◽  
Recall Rate ◽  
Classification Performance ◽  
Support Vector ◽  
Coordinate Systems ◽  
Polar Coordinate

In this study, we propose a method for training convolutional neural networks to make them identify and classify images with higher classification accuracy. By combining the Cartesian and polar coordinate systems when describing the images, the method of recognition and classification for plankton images is discussed. The optimized classification and recognition networks are constructed. They are available for in situ plankton images, exploiting the advantages of both coordinate systems in the network training process. Fusing the two types of vectors and using them as the input for conventional machine learning models for classification, support vector machines (SVMs) are selected as the classifiers to combine these two features of vectors, coming from different image coordinate descriptions. The accuracy of the proposed model was markedly higher than those of the initial classical convolutional neural networks when using the in situ plankton image data, with the increases in classification accuracy and recall rate being 5.3% and 5.1% respectively. In addition, the proposed training method can improve the classification performance considerably when used on the public CIFAR-10 dataset.

scholarly journals LITHOLOGICAL CLASSIFICATION USING MULTI-SENSOR DATA AND CONVOLUTIONAL NEURAL NETWORKS

2019 ◽  
Vol XLII-2/W16 ◽  
pp. 55-59 ◽  
Author(s):  
M. Brandmeier ◽  
Y. Chen
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Sensor Data ◽  
Support Vector ◽  
Carbonate Sediments ◽  
Feature Maps ◽  
Aster Data ◽  
Sentinel 2

<p><strong>Abstract.</strong> Deep learning has been used successfully in computer vision problems, e.g. image classification, target detection and many more. We use deep learning in conjunction with ArcGIS to implement a model with advanced convolutional neural networks (CNN) for lithological mapping in the Mount Isa region (Australia). The area is ideal for spectral remote sensing as there is only sparse vegetation and besides freely available Sentinel-2 and ASTER data, several geophysical datasets are available from exploration campaigns. By fusing the data and thus covering a wide spectral range as well as capturing geophysical properties of rocks, we aim at improving classification accuracies and support geological mapping. We also evaluate the performance of the sensors on their own compared to a joint use as the Sentinel-2 satellites are relatively new and as of now there exist only few studies for geological applications. We developed an end-to-end deep learning model using Keras and Tensorflow that consists of several convolutional, pooling and deconvolutional layers. Our model was inspired by the family of U-Net architectures, where low-level feature maps (encoders) are concatenated with high-level ones (decoders), which enables precise localization. This type of network architecture was especially designed to effectively solve pixel-wise classification problems, which is appropriate for lithological classification. We spatially resampled and fused the multi-sensor remote sensing data with different bands and geophysical data into image cubes as input for our model. Pre-processing was done in ArcGIS and the final, fine-tuned model was imported into a toolbox to be used on further scenes directly in the GIS environment. The tool classifies each pixel of the multiband imagery into different types of rocks according to a defined probability threshold. Results highlight the power of using Sentinel-2 in conjunction with ASTER data with accuracies of 75% in comparison to only 70% and 73% for ASTER or Sentinel-2 data alone. These results are similar but examining the different classes shows that there are significant improvements for classes such as dolerite or carbonate sediments that are not that widely distributed in the area. Adding geophysical datasets reduced accuracies to 60%, probably due to an order of magnitude difference in spatial resolution. In comparison, Random Forest (RF) and Support Vector Machines (SVMs) that were trained on the same data only achieve accuracies of 46 % and 36 % respectively. Most insecurity is due to labelling errors and labels with mixed lithologies. However, results show that the U-Netmodel is a powerful alternative to other classifiers for medium-resolution multispectral data.</p>

Sign in / Sign up

Export Citation Format

Share Document