Cloud detection for MIPAS using singular vector decomposition

Abstract. Satellite-borne high-spectral-resolution limb sounders, such as the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) onboard ENVISAT, provide information on clouds, especially optically thin clouds, which have been difficult to observe in the past. The aim of this work is to develop, implement and test a reliable cloud detection method for infrared spectra measured by MIPAS. Current MIPAS cloud detection methods used operationally have been developed to detect cloud effective filling more than 30% of the measurement field-of-view (FOV), under geometric and optical considerations – and hence are limited to detecting fairly thick cloud, or large physical extents of thin cloud. In order to resolve thin clouds, a new detection method using Singular Vector Decomposition (SVD) is formulated and tested. This new SVD detection method has been applied to a year's worth of MIPAS data, and qualitatively appears to be more sensitive to thin cloud than the current operational method.

Download Full-text

Cloud detection for MIPAS using singular vector decomposition

Atmospheric Measurement Techniques Discussions ◽

10.5194/amtd-2-1185-2009 ◽

2009 ◽

Vol 2 (2) ◽

pp. 1185-1219

Author(s):

J. Hurley ◽

A. Dudhia ◽

R. G. Grainger

Keyword(s):

Climate Models ◽

Singular Vector ◽

Detection Method ◽

Michelson Interferometer ◽

Detection Methods ◽

Cloud Detection ◽

Radiative Balance ◽

The Past ◽

Vector Decomposition ◽

Atmospheric Sounding

Abstract. Clouds are increasingly recognised for their influence on the radiative balance of the Earth and the implications that they have on possible climate change, as well as in air pollution and acid-rain production. However, clouds remain a major source of uncertainty in climate models. Satellite-borne high-resolution limb sounders, such as the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) onboard ENVISAT, provide information on clouds, especially optically thin clouds, which have been difficult to observe in the past. The aim of this work is to develop, implement and test a reliable cloud detection method for infrared spectra measured by MIPAS. Current MIPAS cloud detection methods used operationally have been developed to detect thick cloud filling more than 30% of the measurement field-of-view (FOV). In order to resolve thin clouds, a new detection method using Singular Vector Decomposition (SVD) is formulated and tested. A rigorous comparison of the current operational and newly-developed detection methods for MIPAS is carried out – and the new SVD detection method has been proven to be much more reliable than the current operational method, and very sensitive even to thin clouds only marginally filling the MIPAS FOV.

Download Full-text

Retrieving Sun-Induced Chlorophyll Fluorescence from Hyperspectral Data with TanSat Satellite

Sensors ◽

10.3390/s21144886 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4886

Author(s):

Shilei Li ◽

Maofang Gao ◽

Zhao-Liang Li

Keyword(s):

Chlorophyll Fluorescence ◽

Fluorescence Spectrum ◽

Singular Vector ◽

Low Frequency ◽

Information Criterion ◽

Hyperspectral Data ◽

High Spectral Resolution ◽

Remotely Sensed Data ◽

Spectral Window ◽

Vector Decomposition

A series of algorithms for satellite retrievals of sun-induced chlorophyll fluorescence (SIF) have been developed and applied to different sensors. However, research on SIF retrieval using hyperspectral data is performed in narrow spectral windows, assuming that SIF remains constant. In this paper, based on the singular vector decomposition (SVD) technique, we present an approach for retrieving SIF, which can be applied to remotely sensed data with ultra-high spectral resolution and in a broad spectral window without assuming that the SIF remains constant. The idea is to combine the first singular vector, the pivotal information of the non-fluorescence spectrum, with the low-frequency contribution of the atmosphere, plus a linear combination of the remaining singular vectors to express the non-fluorescence spectrum. Subject to instrument settings, the retrieval was performed within a spectral window of approximately 7 nm that contained only Fraunhofer lines. In our retrieval, hyperspectral data of the O2-A band from the first Chinese carbon dioxide observation satellite (TanSat) was used. The Bayesian Information Criterion (BIC) was introduced to self-adaptively determine the number of free parameters and reduce retrieval noise. SIF retrievals were compared with TanSat SIF and OCO-2 SIF. The results showed good consistency and rationality. A sensitivity analysis was also conducted to verify the performance of this approach. To summarize, the approach would provide more possibilities for retrieving SIF from hyperspectral data.

Download Full-text

Bias determination and precision validation of ozone profiles from MIPAS-Envisat retrieved with the IMK-IAA processor

Atmospheric Chemistry and Physics ◽

10.5194/acp-7-3639-2007 ◽

2007 ◽

Vol 7 (13) ◽

pp. 3639-3662 ◽

Cited By ~ 42

Author(s):

T. Steck ◽

T. von Clarmann ◽

H. Fischer ◽

B. Funke ◽

N. Glatthor ◽

...

Keyword(s):

Stratospheric Ozone ◽

Michelson Interferometer ◽

High Spectral Resolution ◽

Vapor Concentration ◽

Atmospheric Sounding ◽

Mean Differences ◽

Microwave Radiometers ◽

Vertical Ozone ◽

Vertical Ozone Profiles ◽

Polar Ozone

Abstract. This paper characterizes vertical ozone profiles retrieved with the IMK-IAA (Institute for Meteorology and Climate Research, Karlsruhe – Instituto de Astrofisica de Andalucia) science-oriented processor from high spectral resolution data (until March 2004) measured by the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) aboard the environmental satellite Envisat. Bias determination and precision validation is performed on the basis of correlative measurements by ground-based lidars, Fourier transform infrared spectrometers, and microwave radiometers as well as balloon-borne ozonesondes, the balloon-borne version of MIPAS, and two satellite instruments (Halogen Occultation Experiment and Polar Ozone and Aerosol Measurement III). Percentage mean differences between MIPAS and the comparison instruments for stratospheric ozone are generally within ±10%. The precision in this altitude region is estimated at values between 5 and 10% which gives an accuracy of 15 to 20%. Below 18 km, the spread of the percentage mean differences is larger and the precision degrades to values of more than 20% depending on altitude and latitude. The main reason for the degraded precision at low altitudes is attributed to undetected thin clouds which affect MIPAS retrievals, and to the influence of uncertainties in the water vapor concentration.

Download Full-text

A non-iterative linear retrieval for infrared high resolution limb sounders

Atmospheric Measurement Techniques Discussions ◽

10.5194/amtd-6-721-2013 ◽

2013 ◽

Vol 6 (1) ◽

pp. 721-766

Author(s):

L. Millán ◽

A. Dudhia

Keyword(s):

High Resolution ◽

Computing Time ◽

Michelson Interferometer ◽

High Spectral Resolution ◽

Radiative Transfer Model ◽

Spectral Signature ◽

Transfer Model ◽

Gas Concentration ◽

Atmospheric Sounding ◽

Retrieval Scheme

Abstract. Currently most of the high spectral resolution infrared limb sounders use subsets of the recorded spectra (microwindows) in their retrieval schemes to reduce the computing time of rerunning the radiative transfer model. A fast linear retrieval scheme is described which allows the whole spectral signature of the target molecule to be used. We determine how close the linearisation point needs to be to the solution in order to fall in the linear regime and also suggest an adjustment to the forward model and Jacobians to propagate the change in pressure and temperature on the gas concentration retrievals. As an example, this technique is implemented for the Michelson Interferometer for Passive Atmospheric Sounding instrument, but it is applicable to any high resolution limb sounder.

Download Full-text

A total sky cloud detection method using real clear sky background

Atmospheric Measurement Techniques Discussions ◽

10.5194/amtd-8-13073-2015 ◽

2015 ◽

Vol 8 (12) ◽

pp. 13073-13098 ◽

Cited By ~ 1

Author(s):

J. Yang ◽

Q. Min ◽

W. Lu ◽

Y. Ma ◽

W. Yao ◽

...

Keyword(s):

Detection Algorithm ◽

Brightness Distribution ◽

Detection Methods ◽

Cloud Detection ◽

Image Brightness ◽

Clear Sky ◽

Sensitivity Tests ◽

Thin Cloud ◽

Distribution Sensitivity ◽

Solar Position

Abstract. The brightness distribution of sky background is usually non-uniform, which creates many problems for traditional cloud detection methods including the failure of thin cloud detection in total sky images and significantly reducing retrieval accuracy in the circumsolar and near-horizon regions. This paper describes the development of a new cloud detection algorithm, named "clear sky background differencing (CSBD)", which is accomplished by differencing the original image and the corresponding clear sky background image using the images' green channel. First, a library of clear sky background images with a variety of solar elevation angles needs to be developed. The image rotation and image brightness adjustment algorithms are applied to ensure the two images being differenced have the same solar position and similar brightness distribution. Sensitivity tests show, as long as the positions of the sun in the two images are the same, the cloud detection results are satisfactory. Several experimental cases show that the CSBD algorithm obtains good cloud recognition results visually, especially for thin clouds.

Download Full-text

An Unsupervised Approach for Detecting Group Shilling Attacks in Recommender Systems Based on Topological Potential and Group Behaviour Features

Security and Communication Networks ◽

10.1155/2021/2907691 ◽

2021 ◽

Vol 2021 ◽

pp. 1-18

Author(s):

Hongyun Cai ◽

Fuzhi Zhang

Keyword(s):

Recommender Systems ◽

Detection Method ◽

Potential Method ◽

Detection Methods ◽

Clustering Method ◽

Group Behaviour ◽

The Past ◽

Group Members ◽

Unsupervised Approach ◽

Individual Features

To protect recommender systems against shilling attacks, a variety of detection methods have been proposed over the past decade. However, these methods focus mainly on individual features and rarely consider the lockstep behaviours among attack users, which suffer from low precision in detecting group shilling attacks. In this work, we propose a three-stage detection method based on strong lockstep behaviours among group members and group behaviour features for detecting group shilling attacks. First, we construct a weighted user relationship graph by combining direct and indirect collusive degrees between users. Second, we find all dense subgraphs in the user relationship graph to generate a set of suspicious groups by introducing a topological potential method. Finally, we use a clustering method to detect shilling groups by extracting group behaviour features. Extensive experiments on the Netflix and sampled Amazon review datasets show that the proposed approach is effective for detecting group shilling attacks in recommender systems, and the F1-measure on two datasets can reach over 99 percent and 76 percent, respectively.

Download Full-text

A total sky cloud detection method using real clear sky background

Atmospheric Measurement Techniques ◽

10.5194/amt-9-587-2016 ◽

2016 ◽

Vol 9 (2) ◽

pp. 587-597 ◽

Cited By ~ 8

Author(s):

Jun Yang ◽

Qilong Min ◽

Weitao Lu ◽

Ying Ma ◽

Wen Yao ◽

...

Keyword(s):

Detection Algorithm ◽

Brightness Distribution ◽

Detection Methods ◽

Cloud Detection ◽

Image Brightness ◽

Clear Sky ◽

Sensitivity Tests ◽

Thin Cloud ◽

Distribution Sensitivity ◽

Solar Position

Abstract. The brightness distribution of sky background is usually non-uniform, which creates many problems for traditional cloud detection methods, including the failure of thin cloud detection in total sky images and significantly reducing retrieval accuracy in the circumsolar and near-horizon regions. This paper describes the development of a new cloud detection algorithm, named "clear sky background differencing (CSBD)", which is accomplished by differencing the original image and the corresponding clear sky background image using the images' green channel. First, a library of clear sky background images with a variety of solar elevation angles needs to be developed. The image rotation and image brightness adjustment algorithms are applied to ensure the two images being differenced have the same solar position and similar brightness distribution. Sensitivity tests show that the cloud detection results are satisfactory when the two images have the same solar positions. Several experimental cases show that the CSBD algorithm obtains good cloud recognition results visually, especially for thin clouds.

Download Full-text

SFRS-Net: A Cloud-Detection Method Based on Deep Convolutional Neural Networks for GF-1 Remote-Sensing Images

Remote Sensing ◽

10.3390/rs13152910 ◽

2021 ◽

Vol 13 (15) ◽

pp. 2910

Author(s):

Xiaolong Li ◽

Hong Zheng ◽

Chuanzhao Han ◽

Wentao Zheng ◽

Hao Chen ◽

...

Keyword(s):

Remote Sensing ◽

Neural Networks ◽

Convolutional Neural Networks ◽

Detection Method ◽

Detection Methods ◽

Utilization Rate ◽

Optical Remote Sensing ◽

Cloud Detection ◽

Remote Sensing Images ◽

Deep Convolutional Neural Networks

Clouds constitute a major obstacle to the application of optical remote-sensing images as they destroy the continuity of the ground information in the images and reduce their utilization rate. Therefore, cloud detection has become an important preprocessing step for optical remote-sensing image applications. Due to the fact that the features of clouds in current cloud-detection methods are mostly manually interpreted and the information in remote-sensing images is complex, the accuracy and generalization of current cloud-detection methods are unsatisfactory. As cloud detection aims to extract cloud regions from the background, it can be regarded as a semantic segmentation problem. A cloud-detection method based on deep convolutional neural networks (DCNN)—that is, a spatial folding–unfolding remote-sensing network (SFRS-Net)—is introduced in the paper, and the reason for the inaccuracy of DCNN during cloud region segmentation and the concept of space folding/unfolding is presented. The backbone network of the proposed method adopts an encoder–decoder structure, in which the pooling operation in the encoder is replaced by a folding operation, and the upsampling operation in the decoder is replaced by an unfolding operation. As a result, the accuracy of cloud detection is improved, while the generalization is guaranteed. In the experiment, the multispectral data of the GaoFen-1 (GF-1) satellite is collected to form a dataset, and the overall accuracy (OA) of this method reaches 96.98%, which is a satisfactory result. This study aims to develop a method that is suitable for cloud detection and can complement other cloud-detection methods, providing a reference for researchers interested in cloud detection of remote-sensing images.

Download Full-text

Retrieval of macrophysical cloud parameters from MIPAS: algorithm description and preliminary validation

Atmospheric Measurement Techniques Discussions ◽

10.5194/amtd-3-3877-2010 ◽

2010 ◽

Vol 3 (4) ◽

pp. 3877-3906

Author(s):

J. Hurley ◽

A. Dudhia ◽

R. G. Grainger

Keyword(s):

Atmospheric Chemistry ◽

A Priori ◽

Michelson Interferometer ◽

Optimal Estimation ◽

Additional Information ◽

The Past ◽

Cloud Parameters ◽

Preliminary Validation ◽

Atmospheric Sounding ◽

Cloud Top Temperature

Abstract. The Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) onboard ENVISAT has the potential to be particularly useful for studying high, thin clouds, which have been difficult to observe in the past. This paper details the development, implementation and testing of an optimal-estimation-type retrieval for three macrophysical cloud parameters (cloud top height, cloud top temperature and cloud extinction coefficient) from infrared spectra measured by MIPAS, employing additional information derived to improve the choice of a priori. The retrieval is applied and initially validated on MIPAS data. From application to MIPAS data, the retrieved cloud top heights are assessed to be accurate to within 50 m, the cloud top temperatures to within 0.5 K and extinction coefficients to within a factor of 15%. This algorithm has been adopted by the European Space Agency's ''MIPclouds'' project, which itself recognises the potential of MIPAS beyond monitoring atmospheric chemistry and seeks to study clouds themselves rigorously using MIPAS.

Download Full-text

Infrared limb emission measurements of aerosol in the troposphere and stratosphere

Atmospheric Measurement Techniques ◽

10.5194/amt-9-4399-2016 ◽

2016 ◽

Vol 9 (9) ◽

pp. 4399-4423 ◽

Cited By ~ 12

Author(s):

Sabine Griessbach ◽

Lars Hoffmann ◽

Reinhold Spang ◽

Marc von Hobe ◽

Rolf Müller ◽

...

Keyword(s):

Volcanic Ash ◽

Lower Stratosphere ◽

Michelson Interferometer ◽

Emission Spectra ◽

High Spectral Resolution ◽

Volcanic Aerosol ◽

Atmospheric Infrared Sounder ◽

Ice Clouds ◽

Upper Troposphere ◽

Atmospheric Sounding

Abstract. Altitude-resolved aerosol detection in the upper troposphere and lower stratosphere (UTLS) is a challenging task for remote sensing instruments. Infrared limb emission measurements provide vertically resolved global measurements at day- and nighttime in the UTLS. For high-spectral-resolution infrared limb instruments we present here a new method to detect aerosol and separate between ice and non-ice particles. The method is based on an improved aerosol–cloud index that identifies infrared limb emission spectra affected by non-ice aerosol or ice clouds. For the discrimination between non-ice aerosol and ice clouds we employed brightness temperature difference correlations. The discrimination thresholds for this method were derived from radiative transfer simulations (including scattering) and Michelson Interferometer for Passive Atmospheric Sounding (MIPAS)/Envisat measurements obtained in 2011. We demonstrate the value of this approach for observations of volcanic ash and sulfate aerosol originating from the Grímsvötn (Iceland, 64° N), Puyehue–Cordón Caulle (Chile, 40° S), and Nabro (Eritrea, 13° N) eruptions in May and June 2011 by comparing the MIPAS volcanic aerosol detections with Atmospheric Infrared Sounder (AIRS) volcanic ash and SO2 measurements.

Download Full-text