Hyperspectral data compression based upon the principal component analysis

The paper is aimed at developing an algorithm of hyperspectral data compression that combines small losses with high compression rate. The algorithm relies on a principal component analysis and a method of exhaustion. The principal components are singular vectors of an initial signal matrix, which are found by the method of exhaustion. A retrieved signal matrix is formed in parallel. The process continues until a required retrieval error is attained. The algorithm is described in detail and input and output parameters are specified. Testing is performed using AVIRIS data (Airborne Visible-Infrared Imaging Spectrometer). Three images of differently looking sky (clear sky, partly clouded sky, and overcast skies) are analyzed. For each image, testing is performed for all spectral bands and for a set of bands from which high water-vapour absorption bands are excluded. Retrieval errors versus compression rates are presented. The error formulas include the root mean square deviation, the noise-to-signal ratio, the mean structural similarity index, and the mean relative deviation. It is shown that the retrieval errors decrease by more than an order of magnitude if spectral bands with high gas absorption are disregarded. It is shown that the reason is that weak signals in the absorption bands are measured with great errors, leading to a weak dependence between the spectra in different spatial pixels. A mean cosine distance between the spectra in different spatial pixels is suggested to be used to assess the image compressibility.

Simulating the synergy of microwave radiometer and differential absorption radar for advancing water vapor profiling in cloudy trade-wind conditions

<p>Continuous, high vertical resolution water vapor profile measurements are key for advancing the understanding of how clouds interact with their environment through convection, precipitation and circulation processes.  Yet, current ground-based observation systems are limited by low temporal resolution in the case of soundings, signal saturation at cloud base in the case of optical sensors, or too coarse vertical resolution in the case of passive microwave measurements. Overcoming the limitations of each single sensor, we assess the synergistic benefits of combining ground-based microwave radiometer (MWR) and the novel Differential Absorption Radar technique, based on synthetic measurements generated for typical trade wind conditions as observed during the EUREC<sup>4</sup>A field study.</p><p>Based on the single and multiple cloud layer conditions observed at Barbados Cloud Observatory, we use the passive and active microwave transfer model PAMTRA to generate synthetic measurements of the K-band MWR channels, as well as for a G-band dual-frequency radar instrument operating at frequencies of 167 and 174.8 GHz.  The synthetic brightness temperatures and radar dual-frequency ratios are combined in an optimal estimation framework to retrieve the absolute humidity profile. Varying the observation vector setup, the synergy benefits are assessed by comparing the synergistic information content (Degrees of Freedom for Signal, DFS) and retrieval errors to the respective single-instrument configuration, and by evaluating the retrieved profile using the initial sounding profile.</p><p>In single-cloud conditions, the total synergistic retrieval information content increases by more than one DFS compared to a MWR-only retrieval. While the radar measurements dominate the retrieval below and throughout the cloud layer, the MWR drives the retrieval above the cloud layer. The synergy further enhances the information content above the cloud layer by up to 15% compared to the MWR-only retrieval, accompanied by decreased retrieval errors of up to 10%. Cases of a shallow cloud layer topped by a stratiform outflow confirm the identified patterns. The radar measurements further increase the information content between the cloud layers by up to 25%. In this case, the results suggest an improved partitioning of the water vapor amount below and above the trade inversion. </p><p>Current G-band radar signal attenuation in moist tropical conditions are expected to reduce the feasible synergy potential in a real application. Yet, increased radar signal sensitivities, adjusted frequency pairs, or drier atmospheric conditions motivate the application of this synergy concept to real measurements for advancing ground-based water vapor profiling in cloudy conditions.</p>

Retrieval of Daytime Total Column Water Vapour from OLCI Measurements over Land Surfaces

A new retrieval of total column water vapour (TCWV) from daytime measurements over land of the Ocean and Land Colour Instrument (OLCI) on-board the Copernicus Sentinel-3 missions is presented. The Copernicus Sentinel-3 OLCI Water Vapour product (COWa) retrieval algorithm is based on the differential absorption technique, relating TCWV to the radiance ratio of non-absorbing band and nearby water vapour absorbing band and was previously also successfully applied to other passive imagers Medium Resolution Imaging Spectrometer (MERIS) and Moderate Resolution Imaging Spectroradiometer (MODIS). One of the main advantages of the OLCI instrument regarding improved TCWV retrievals lies in the use of more than one absorbing band. Furthermore, the COWa retrieval algorithm is based on the full Optimal Estimation (OE) method, providing pixel-based uncertainty estimates, and transferable to other Near-Infrared (NIR) based TCWV observations. Three independent global TCWV data sets, i.e., Aerosol Robotic Network (AERONET), Atmospheric Radiation Measurement (ARM) and U.S. SuomiNet, and a German Global Navigation Satellite System (GNSS) TCWV data set, all obtained from ground-based observations, serve as reference data sets for the validation. Comparisons show an overall good agreement, with absolute biases between 0.07 and 1.31 kg/m2 and root mean square errors (RMSE) between 1.35 and 3.26 kg/m2. This is a clear improvement in comparison to the operational OLCI TCWV Level 2 product, for which the bias and RMSEs range between 1.10 and 2.55 kg/m2 and 2.08 and 3.70 kg/m2, respectively. A first evaluation of pixel-based uncertainties indicates good estimated uncertainties for lower retrieval errors, while the uncertainties seem to be overestimated for higher retrieval errors.

Motorist Understanding of Pavement Centre Lines and their Effect on Driving Behaviour

Human factors and more generally driver errors account for the largest number of road accidents. Driver errors are external human factors that can contribute to specific error types selected from slip, lapse, mistake and violation. Action and information retrieval errors are both examples of driver errors. The failure to interpret correctly an intended road marking’s message causes driver misunderstanding and lead to a driver error. Centre lines are examples of such markings and if misread or unrecognised may cause unintentional driver violations and unsafe driving. This study focused on the examining of driver understanding of road markings, and the influence of centre lines on their driving behaviour. This study determined that drivers had a much better understanding of the overtaking messages intended by road markings, than the directional flow message. Drivers demonstrated that they relied more on signs and other drivers to determine whether the road is a two-way or not. This study demonstrated that the presence of both centre lines and edge lines have a positive effect on a driver in handling and controlling of their vehicles’ position. It was postulated from this study that the absence of the edge lines has a more significant effect on a vehicle’s position than the absence of centre lines.

Impact of Molecular Spectroscopy on Carbon Monoxide Abundances from TROPOMI

The impact of SEOM–IAS (Scientific Exploitation of Operational Missions–Improved Atmospheric Spectroscopy) spectroscopic information on CO columns from TROPOMI (Tropospheric Monitoring Instrument) shortwave infrared (SWIR) observations was examined. HITRAN 2016 (High Resolution Transmission) and GEISA 2015 (Gestion et Etude des Informations Spectroscopiques Atmosphériques 2015) were used as a reference upon which the spectral fitting residuals, retrieval errors and inferred quantities were assessed. It was found that SEOM–IAS significantly improves the quality of the CO retrieval by reducing the residuals to TROPOMI observations. The magnitude of the impact is dependent on the climatological region and spectroscopic reference used. The difference in the CO columns was found to be rather small, although discrepancies reveal, for selected scenes, in particular, for observations with elevated molecular concentrations. A brief comparison to Total Column Carbon Observing Network (TCCON) and Network for the Detection of Atmospheric Composition Change (NDACC) also demonstrated that both spectroscopies cause similar columns; however, the smaller retrieval errors in the SEOM with Speed-Dependent Rautian and line-Mixing (SDRM) inferred CO turned out to be beneficial in the comparison of post-processed mole fractions with ground-based references.

Noise-sensitivity Analysis and Improvement of Automatic Retrieval of Temperature and Emissivity Using Spectral Smoothness

There are numerous algorithms that can be used to retrieve land surface temperature (LST) and land surface emissivity (LSE) from hyperspectral thermal infrared (HTIR) data. The algorithms are sensitive to a number of factors, where noise is difficult to handle due to its unpredictability. Although there is a lot of research regarding the influence of noise on retrieval errors, few studies have focused on the mechanism. In this study, we selected the automatic retrieval of temperature and emissivity using spectral smoothness (ARTEMISS) algorithm—the representative of the iterative spectral smoothness temperature-emissivity separation algorithm family—as the research object and proposed an improved algorithm. First, we analyzed the influence mechanism of noise on the retrieval errors of ARTEMISS in theory. Second, we carried out a simulation and inversion experiment and analyzed the relationship between instrument spectral resolution, noise level, the ARTEMISS parameter setting and the retrieval errors separately. Last, we proposed an improved method (resolution-degrade-based spectral smoothness algorithm, RDSS) based on the mechanism and law of the influence of noise on retrieval errors and provided corresponding suggestions on instrument design. The results show that RDSS improves the accuracy of temperature inversion and is more effective for thermal infrared data with a high noise level and high spectral resolution, which can reduce the LST inversion error by up to 0.75 K and the LSE median absolute deviation (MAD) by 31%. In the presence of noise in HTIR data, the RDSS algorithm performs better than the ARTEMISS algorithm in terms of temperature-emissivity separation.

Effect of snow microstructure variability on Ku-band radar snow water equivalent retrievals

Spatial variability in snowpack properties negatively impacts our capacity to make direct measurements of snow water equivalent (SWE) using satellites. A comprehensive data set of snow microstructure (94 profiles at 36 sites) and snow layer thickness (9000 vertical profiles across nine trenches) collected over two winters at Trail Valley Creek, NWT, Canada, was applied in synthetic radiative transfer experiments. This allowed for robust assessment of the impact of estimation accuracy of unknown snow microstructural characteristics on the viability of SWE retrievals. Depth hoar layer thickness varied over the shortest horizontal distances, controlled by subnivean vegetation and topography, while variability in total snowpack thickness approximated that of wind slab layers. Mean horizontal correlation lengths of layer thickness were less than a metre for all layers. Depth hoar was consistently ∼30 % of total depth, and with increasing total depth the proportion of wind slab increased at the expense of the decreasing surface snow layer. Distinct differences were evident between distributions of layer properties; a single median value represented density and specific surface area (SSA) of each layer well. Spatial variability in microstructure of depth hoar layers dominated SWE retrieval errors. A depth hoar SSA estimate of around 7 % under the median value was needed to accurately retrieve SWE. In shallow snowpacks <0.6 m, depth hoar SSA estimates of ±5 %–10 % around the optimal retrieval SSA allowed SWE retrievals within a tolerance of ±30 mm. Where snowpacks were deeper than ∼30 cm, accurate values of representative SSA for depth hoar became critical as retrieval errors were exceeded if the median depth hoar SSA was applied.

Toward autonomous surface-based infrared remote sensing of polar clouds: Retrievals of cloud microphysical properties

Improvements to climate model results in polar regions require improved knowledge of cloud microphysical properties. Surface-based infrared radiance spectrometers have been used to retrieve cloud microphysical properties in polar regions, but measurements are sparse. Reductions in cost and power requirements to allow more widespread measurements could be aided by reducing instrument resolution. Here we explore the effect of errors and instrument resolution on cloud microphysical property retrievals from downwelling infrared radiances for resolutions of 0.1 to 8 cm−1. Retrievals are tested on 331 radiance simulations characteristic of the Arctic, including mixed-phase, vertically inhomogeneous, and liquid-topped clouds and a variety of ice habits. Results indicate that measurement biases lead to biases in retrieved properties that are not represented by the retrieval error covariance matrix. Retrieval errors are high if mixed-phase is assumed throughout liquid-topped ice clouds. Errors due to assuming ice habit is spherical are progressively larger for solid columns, plates, and hollow bullet rosettes. Using retrieved cloud heights, particularly when errors are imposed, increases retrieval errors but decreases sensitivity to incorrect ice habits and vertical variation. Results indicate that retrieval accuracy is unaffected by resolution from 0.1 to 2 cm−1, after which it decreases only slightly. At a resolution of 4 cm−1, for typical errors expected in temperature (0.2 K) and water vapour (3 %), and assuming radiation bias and noise of 0.2 mW/(m2 sr cm−1), using retrieved cloud heights, error estimates are 0.1 ± 0.6 for optical depth, 0.0 ± 0.3 for ice fraction, 0 &amp;plusmnl 2 μm for effective radius of liquid, and 2 ± 2 μm for effective radius of ice. These results indicate that a moderately low resolution, portable, surface-based infrared spectrometer could provide microphysical properties to help constrain climate models.