scholarly journals Removing Cosmic Spikes Using a Hyperspectral Upper-Bound Spectrum Method

2016 ◽  
Vol 71 (3) ◽  
pp. 507-519 ◽  
Author(s):  
Stephen M. Anthony ◽  
Jerilyn A. Timlin
Keyword(s):  
Hyperspectral Imaging ◽  
Principal Components ◽  
Upper Bound ◽  
Cosmic Ray ◽  
Hyperspectral Data ◽  
Hyperspectral Images ◽  
Data Sets ◽  
Imaging Data ◽  
Data Set ◽  
Spectrum Method

Cosmic ray spikes are especially problematic for hyperspectral imaging because of the large number of spikes often present and their negative effects upon subsequent chemometric analysis. Fortunately, while the large number of spectra acquired in a hyperspectral imaging data set increases the probability and number of cosmic spikes observed, the multitude of spectra can also aid in the effective recognition and removal of the cosmic spikes. Zhang and Ben-Amotz were perhaps the first to leverage the additional spatial dimension of hyperspectral data matrices (DM). They integrated principal component analysis (PCA) into the upper bound spectrum method (UBS), resulting in a hybrid method (UBS-DM) for hyperspectral images. Here, we expand upon their use of PCA, recognizing that principal components primarily present in only a few pixels most likely correspond to cosmic spikes. Eliminating the contribution of those principal components in those pixels improves the cosmic spike removal. Both simulated and experimental hyperspectral Raman image data sets are used to test the newly developed UBS-DM-hyperspectral (UBS-DM-HS) method which extends the UBS-DM method by leveraging characteristics of hyperspectral data sets. A comparison is provided between the performance of the UBS-DM-HS method and other methods suitable for despiking hyperspectral images, evaluating both their ability to remove cosmic ray spikes and the extent to which they introduce spectral bias.

scholarly journals An Overview of Hyperspectral Remote Sensing and its applications in various Disciplines

2016 ◽  
Vol 5 (2) ◽  
pp. 85 ◽  
Author(s):  
Alpana Shukla ◽  
Rajsi Kot
Keyword(s):  
Remote Sensing ◽  
Hyperspectral Imaging ◽  
Spatial Information ◽  
Coastal Zone Management ◽  
Hyperspectral Remote Sensing ◽  
Hyperspectral Data ◽  
Data Cube ◽  
Data Sets ◽  
Imaging Data ◽  
Advantages And Disadvantages

<div><p><em>Recent advances in remote sensing and geographic information has opened new directions for the development of hyperspectral sensors. Hyperspectral remote sensing, also known as imaging spectroscopy is a new technology. Hyperspectral imaging is currently being investigated by researchers and scientists for the detection and identification of vegetation, minerals, different objects and background.</em><em> Hyperspectral remote sensing combines imaging and spectroscopy in a single system which often includes large data sets and requires new processing methods. Hyperspectral data sets are generally made of about 100 to 200 spectral bands of relatively narrow bandwidths (5-10 nm), whereas, multispectral data sets are usually composed of about 5 to 10 bands of relatively large bandwidths (70-400 nm). Hyperspectral imagery is collected as a data cube with spatial information collected in the X-Y plane, and spectral information represented in the Z-direction. </em><em>Hyperspectral remote sensing is applicable in many different disciplines. It was originally developed for mining and geology; it has now spread into fields such as agriculture and forestry, ecology, coastal zone management, geology and mineral exploration. This paper presents an overview of hyperspectral imaging, data exploration and analysis, applications in various disciplines, advantages and disadvantages and future aspects of the technique.</em></p></div>

A Semi-Autonomous Method to Detect Cosmic Rays in Raman Hyperspectral Data Sets

2019 ◽  
Vol 73 (9) ◽  
pp. 1019-1027 ◽  
Author(s):  
Kyle Uckert ◽  
Rohit Bhartia ◽  
John Michel
Keyword(s):  
Cosmic Rays ◽  
Cosmic Ray ◽  
Hyperspectral Data ◽  
Hyperspectral Images ◽  
Data Sets ◽  
Baseline Drift ◽  
Novel Method ◽  
Deep Ultraviolet ◽  
Astronomical Images ◽  
Spectral Variance

Cosmic rays can degrade Raman hyperspectral images by introducing high-intensity noise to spectra, obfuscating the results of downstream analyses. We describe a novel method to detect cosmic rays in deep ultraviolet Raman hyperspectral data sets adapted from existing cosmic ray removal methods applied to astronomical images. This method identifies cosmic rays as outliers in the distribution of intensity values in each wavelength channel. In some cases, this algorithm fails to identify cosmic rays in data sets with high inter-spectral variance, uncorrected baseline drift, or few spectra. However, this method effectively identifies cosmic rays in spatially uncorrelated hyperspectral data sets more effectively than other cosmic ray rejection methods and can potentially be employed in commercial and robotic Raman systems to identify cosmic rays semi-autonomously.

scholarly journals An Algorithm for the Removal of Cosmic Ray Artifacts in Spectral Data Sets

2019 ◽  
Vol 73 (8) ◽  
pp. 893-901
Author(s):  
Sinead J. Barton ◽  
Bryan M. Hennelly
Keyword(s):  
Cosmic Ray ◽  
Data Sets ◽  
Biological Cells ◽  
Statistical Classification ◽  
Signal To Noise ◽  
Multivariate Statistical ◽  
Data Set ◽  
Artefact Removal ◽  
Single Capture ◽  
Acquisition Method

Cosmic ray artifacts may be present in all photo-electric readout systems. In spectroscopy, they present as random unidirectional sharp spikes that distort spectra and may have an affect on post-processing, possibly affecting the results of multivariate statistical classification. A number of methods have previously been proposed to remove cosmic ray artifacts from spectra but the goal of removing the artifacts while making no other change to the underlying spectrum is challenging. One of the most successful and commonly applied methods for the removal of comic ray artifacts involves the capture of two sequential spectra that are compared in order to identify spikes. The disadvantage of this approach is that at least two recordings are necessary, which may be problematic for dynamically changing spectra, and which can reduce the signal-to-noise (S/N) ratio when compared with a single recording of equivalent duration due to the inclusion of two instances of read noise. In this paper, a cosmic ray artefact removal algorithm is proposed that works in a similar way to the double acquisition method but requires only a single capture, so long as a data set of similar spectra is available. The method employs normalized covariance in order to identify a similar spectrum in the data set, from which a direct comparison reveals the presence of cosmic ray artifacts, which are then replaced with the corresponding values from the matching spectrum. The advantage of the proposed method over the double acquisition method is investigated in the context of the S/N ratio and is applied to various data sets of Raman spectra recorded from biological cells.

scholarly journals Ordinal classification for efficient plant stress prediction in hyperspectral data

2014 ◽  
Vol XL-7 ◽  
pp. 29-36 ◽  
Author(s):  
J. Behmann ◽  
P. Schmitter ◽  
J. Steinrücken ◽  
L. Plümer
Keyword(s):  
Linear Models ◽  
Plant Stress ◽  
Crop Protection ◽  
Local Stress ◽  
Prediction Performance ◽  
Hyperspectral Data ◽  
Hyperspectral Images ◽  
Support Vector ◽  
Data Set ◽  
High Prediction

Detection of crop stress from hyperspectral images is of high importance for breeding and precision crop protection. However, the continuous monitoring of stress in phenotyping facilities by hyperspectral imagers produces huge amounts of uninterpreted data. In order to derive a stress description from the images, interpreting algorithms with high prediction performance are required. Based on a static model, the local stress state of each pixel has to be predicted. Due to the low computational complexity, linear models are preferable. <br><br> In this paper, we focus on drought-induced stress which is represented by discrete stages of ordinal order. We present and compare five methods which are able to derive stress levels from hyperspectral images: One-vs.-one Support Vector Machine (SVM), one-vs.-all SVM, Support Vector Regression (SVR), Support Vector Ordinal Regression (SVORIM) and Linear Ordinal SVM classification. The methods are applied on two data sets - a real world set of drought stress in single barley plants and a simulated data set. It is shown, that Linear Ordinal SVM is a powerful tool for applications which require high prediction performance under limited resources. It is significantly more efficient than the one-vs.-one SVM and even more efficient than the less accurate one-vs.-all SVM. Compared to the very compact SVORIM model, it represents the senescence process much more accurate.

An Incremental Isomap Method for Hyperspectral Dimensionality Reduction and Classification

2021 ◽  
Vol 87 (6) ◽  
pp. 445-455
Author(s):  
Yi Ma ◽  
Zezhong Zheng ◽  
Yutang Ma ◽  
Mingcang Zhu ◽  
Ran Huang ◽  
...  
Keyword(s):  
Manifold Learning ◽  
Nearest Neighbor ◽  
Hyperspectral Image ◽  
Hyperspectral Data ◽  
Training Data ◽  
Support Vector ◽  
Data Sets ◽  
K Nearest Neighbor ◽  
Data Set ◽  
Data Points

Many manifold learning algorithms conduct an eigen vector analysis on a data-similarity matrix with a size of N×N, where N is the number of data points. Thus, the memory complexity of the analysis is no less than O(N2). We pres- ent in this article an incremental manifold learning approach to handle large hyperspectral data sets for land use identification. In our method, the number of dimensions for the high-dimensional hyperspectral-image data set is obtained with the training data set. A local curvature varia- tion algorithm is utilized to sample a subset of data points as landmarks. Then a manifold skeleton is identified based on the landmarks. Our method is validated on three AVIRIS hyperspectral data sets, outperforming the comparison algorithms with a k–nearest-neighbor classifier and achieving the second best performance with support vector machine.

scholarly journals Decomposing the Apoptosis Pathway Into Biologically Interpretable Principal Components

2018 ◽  
Vol 17 ◽  
pp. 117693511877108 ◽  
Author(s):  
Min Wang ◽  
Steven M Kornblau ◽  
Kevin R Coombes
Keyword(s):  
Principal Components ◽  
Myeloid Leukemia ◽  
Principal Component ◽  
R Package ◽  
Biological Data ◽  
Data Sets ◽  
Proteomics Data ◽  
Data Set ◽  
Apoptosis Pathway ◽  
Biological Interpretation

Principal component analysis (PCA) is one of the most common techniques in the analysis of biological data sets, but applying PCA raises 2 challenges. First, one must determine the number of significant principal components (PCs). Second, because each PC is a linear combination of genes, it rarely has a biological interpretation. Existing methods to determine the number of PCs are either subjective or computationally extensive. We review several methods and describe a new R package, PCDimension, that implements additional methods, the most important being an algorithm that extends and automates a graphical Bayesian method. Using simulations, we compared the methods. Our newly automated procedure is competitive with the best methods when considering both accuracy and speed and is the most accurate when the number of objects is small compared with the number of attributes. We applied the method to a proteomics data set from patients with acute myeloid leukemia. Proteins in the apoptosis pathway could be explained using 6 PCs. By clustering the proteins in PC space, we were able to replace the PCs by 6 “biological components,” 3 of which could be immediately interpreted from the current literature. We expect this approach combining PCA with clustering to be widely applicable.

scholarly journals Data Fusion Using a Multi-Sensor Sparse-Based Clustering Algorithm

2020 ◽  
Vol 12 (23) ◽  
pp. 4007
Author(s):  
Kasra Rafiezadeh Shahi ◽  
Pedram Ghamisi ◽  
Behnood Rasti ◽  
Robert Jackisch ◽  
Paul Scheunders ◽  
...  
Keyword(s):  
Clustering Algorithm ◽  
Spatial Information ◽  
Clustering Algorithms ◽  
Hyperspectral Data ◽  
Sensor Data ◽  
Data Sets ◽  
Data Types ◽  
Data Set ◽  
Multiple Data Sets ◽  
Imaging Sensors

The increasing amount of information acquired by imaging sensors in Earth Sciences results in the availability of a multitude of complementary data (e.g., spectral, spatial, elevation) for monitoring of the Earth’s surface. Many studies were devoted to investigating the usage of multi-sensor data sets in the performance of supervised learning-based approaches at various tasks (i.e., classification and regression) while unsupervised learning-based approaches have received less attention. In this paper, we propose a new approach to fuse multiple data sets from imaging sensors using a multi-sensor sparse-based clustering algorithm (Multi-SSC). A technique for the extraction of spatial features (i.e., morphological profiles (MPs) and invariant attribute profiles (IAPs)) is applied to high spatial-resolution data to derive the spatial and contextual information. This information is then fused with spectrally rich data such as multi- or hyperspectral data. In order to fuse multi-sensor data sets a hierarchical sparse subspace clustering approach is employed. More specifically, a lasso-based binary algorithm is used to fuse the spectral and spatial information prior to automatic clustering. The proposed framework ensures that the generated clustering map is smooth and preserves the spatial structures of the scene. In order to evaluate the generalization capability of the proposed approach, we investigate its performance not only on diverse scenes but also on different sensors and data types. The first two data sets are geological data sets, which consist of hyperspectral and RGB data. The third data set is the well-known benchmark Trento data set, including hyperspectral and LiDAR data. Experimental results indicate that this novel multi-sensor clustering algorithm can provide an accurate clustering map compared to the state-of-the-art sparse subspace-based clustering algorithms.

scholarly journals Technical Note: Regression Analysis of Proximal Hyperspectral Data to Predict Soil pH and Olsen P

Agriculture ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 55 ◽  
Author(s):  
Miles Grafton ◽  
Therese Kaul ◽  
Alan Palmer ◽  
Peter Bishop ◽  
Michael White
Keyword(s):  
Soil Nutrient ◽  
Large Data ◽  
Technical Note ◽  
Soil Samples ◽  
Hyperspectral Data ◽  
Data Sets ◽  
Regression Equations ◽  
Data Set ◽  
Hill Country ◽  
Olsen P

This work examines two large data sets to demonstrate that hyperspectral proximal devices may be able to measure soil nutrient. One data set has 3189 soil samples from four hill country pastoral farms and the second data set has 883 soil samples taken from a stratified nested grid survey. These were regressed with spectra from a proximal hyperspectral device measured on the same samples. This aim was to obtain wavelengths, which may be proxy indicators for measurements of soil nutrients. Olsen P and pH were regressed with 2150 wave bands between 350 nm and 2500 nm to find wavebands, which were significant indicators. The 100 most significant wavebands for each proxy were used to regress both data sets. The regression equations from the smaller data set were used to predict the values of pH and Olsen P to validate the larger data set. The predictions from the equations from the smaller data set were as good as the regression analyses from the large data set when applied to it. This may mean that, in the future, hyperspectral analysis may be a proxy to soil chemical analysis; or increase the intensity of soil testing by finding markers of fertility cheaply in the field.

scholarly journals Data segmentation based on the local intrinsic dimension

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Michele Allegra ◽  
Elena Facco ◽  
Francesco Denti ◽  
Alessandro Laio ◽  
Antonietta Mira
Keyword(s):  
High Dimensional Data ◽  
Large Data ◽  
Large Data Sets ◽  
High Dimensional ◽  
Data Sets ◽  
Imaging Data ◽  
Unsupervised Segmentation ◽  
Real World Data ◽  
Data Set ◽  
Intrinsic Dimension

Abstract One of the founding paradigms of machine learning is that a small number of variables is often sufficient to describe high-dimensional data. The minimum number of variables required is called the intrinsic dimension (ID) of the data. Contrary to common intuition, there are cases where the ID varies within the same data set. This fact has been highlighted in technical discussions, but seldom exploited to analyze large data sets and obtain insight into their structure. Here we develop a robust approach to discriminate regions with different local IDs and segment the points accordingly. Our approach is computationally efficient and can be proficiently used even on large data sets. We find that many real-world data sets contain regions with widely heterogeneous dimensions. These regions host points differing in core properties: folded versus unfolded configurations in a protein molecular dynamics trajectory, active versus non-active regions in brain imaging data, and firms with different financial risk in company balance sheets. A simple topological feature, the local ID, is thus sufficient to achieve an unsupervised segmentation of high-dimensional data, complementary to the one given by clustering algorithms.

scholarly journals RADIOMETRIC CORRECTION OF MULTITEMPORAL HYPERSPECTRAL UAS IMAGE MOSAICS OF SEEDLING STANDS

2017 ◽  
Vol XLII-3/W3 ◽  
pp. 113-118 ◽  
Author(s):  
L. Markelin ◽  
E. Honkavaara ◽  
R. Näsi ◽  
N. Viljanen ◽  
T. Rosnell ◽  
...  
Keyword(s):  
Precision Agriculture ◽  
Atmospheric Correction ◽  
Point Clouds ◽  
Hyperspectral Data ◽  
Reflectance Spectra ◽  
Data Sets ◽  
Data Set ◽  
Relative Correction ◽  
Small Areas ◽  
3D Point Clouds

Novel miniaturized multi- and hyperspectral imaging sensors on board of unmanned aerial vehicles have recently shown great potential in various environmental monitoring and measuring tasks such as precision agriculture and forest management. These systems can be used to collect dense 3D point clouds and spectral information over small areas such as single forest stands or sample plots. Accurate radiometric processing and atmospheric correction is required when data sets from different dates and sensors, collected in varying illumination conditions, are combined. Performance of novel radiometric block adjustment method, developed at Finnish Geospatial Research Institute, is evaluated with multitemporal hyperspectral data set of seedling stands collected during spring and summer 2016. Illumination conditions during campaigns varied from bright to overcast. We use two different methods to produce homogenous image mosaics and hyperspectral point clouds: image-wise relative correction and image-wise relative correction with BRDF. Radiometric datasets are converted to reflectance using reference panels and changes in reflectance spectra is analysed. Tested methods improved image mosaic homogeneity by 5&amp;thinsp;% to 25&amp;thinsp;%. Results show that the evaluated method can produce consistent reflectance mosaics and reflectance spectra shape between different areas and dates.

Sign in / Sign up

Export Citation Format

Share Document