scholarly journals Generalization of the complete data fusion to multi-target retrieval of atmospheric parameters and application to FORUM and IASI-NG simulated measurements

2021 ◽  
Vol 276 ◽  
pp. 107925
Author(s):  
Cecilia Tirelli ◽  
Simone Ceccherini ◽  
Nicola Zoppetti ◽  
Samuele Del Bianco ◽  
Ugo Cortesi
Keyword(s):  
Data Fusion ◽  
Complete Data ◽  
Atmospheric Parameters

scholarly journals Statistical Data Fusion for Cross-Tabulation

1997 ◽  
Vol 34 (4) ◽  
pp. 485-498 ◽  
Author(s):  
Wagner A. Kamakura ◽  
Michel Wedel
Keyword(s):  
Data Fusion ◽  
Statistical Data ◽  
Statistical Tests ◽  
Complete Data ◽  
Discrete Variables ◽  
Fusion Model ◽  
Multiple Imputations ◽  
Tests Of Association ◽  
Cross Tabulation ◽  
Statistical Data Fusion

The authors address the situation in which a researcher wants to cross-tabulate two sets of discrete variables collected in independent samples, but a subset of the variables is common to both samples. The authors propose a statistical data-fusion model that allows for statistical tests of association using multiple imputations. The authors illustrate this approach with an application in which they compare the cross-tabulation results from fused data with those obtained from complete data. Their approach is also compared to the traditional hot-deck procedure.

The Complete Data Fusion as a ready to use tool for the exploitation of atmospheric Sentinel ozone profiles

2020 ◽  
Author(s):  
Nicola Zoppetti ◽  
Simone Ceccherini ◽  
Flavio Barbara ◽  
Samuele Del Bianco ◽  
Marco Gai ◽  
...  
Keyword(s):  
Data Fusion ◽  
A Priori ◽  
Synthetic Data ◽  
Simulated Data ◽  
Real Data ◽  
Low Earth Orbit ◽  
Complete Data ◽  
Atmospheric Composition ◽  
Horizon 2020 ◽  
Level 3

<p>Remote sounding of atmospheric composition makes use of satellite measurements with very heterogeneous characteristics. In particular, the determination of vertical profiles of gases in the atmosphere can be performed using measurements acquired in different spectral bands and with different observation geometries. The most rigorous way to combine heterogeneous measurements of the same quantity in a single Level 2 (L2) product is simultaneous retrieval. The main drawback of simultaneous retrieval is its complexity, due to the necessity to embed the forward models of different instruments into the same retrieval application. To overcome this shortcoming, we developed a data fusion method, referred to as Complete Data Fusion (CDF), to provide an efficient and adaptable alternative to simultaneous retrieval. In general, the CDF input is any number of profiles retrieved with the optimal estimation technique, characterized by their a priori information, covariance matrix (CM), and averaging kernel (AK) matrix. The output of the CDF is a single product also characterized by an a priori, a CM and an AK matrix, which collect all the available information content. To account for the geo-temporal differences and different vertical grids of the fusing profiles, a coincidence and an interpolation error have to be included in the error budget.<br>In the first part of the work, the CDF method is applied to ozone profiles simulated in the thermal infrared and ultraviolet bands, according to the specifications of the Sentinel 4 (geostationary) and Sentinel 5 (low Earth orbit) missions of the Copernicus program. The simulated data have been produced in the context of the Advanced Ultraviolet Radiation and Ozone Retrieval for Applications (AURORA) project funded by the European Commission in the framework of the Horizon 2020 program. The use of synthetic data and the assumption of negligible systematic error in the simulated measurements allow studying the behavior of the CDF in ideal conditions. The use of synthetic data allows evaluating the performance of the algorithm also in terms of differences between the products of interest and the reference truth, represented by the atmospheric scenario used in the procedure to simulate the L2 products. This analysis aims at demonstrating the potential benefits of the CDF for the synergy of products measured by different platforms in a close future realistic scenario, when the Sentinel 4, 5/5p ozone profiles will be available.<br>In the second part of this work, the CDF is applied to a set of real measurements of ozone acquired by GOME-2 onboard the MetOp-B platform. The quality of the CDF products, obtained for the first time from operational products, is compared with that of the original GOME-2 products. This aims to demonstrate the concrete applicability of the CDF to real data and its possible use to generate Level-3 (or higher) gridded products.<br>The results discussed in this presentation offer a first consolidated picture of the actual and potential value of an innovative technique for post-retrieval processing and generation of Level-3 (or higher) products from the atmospheric Sentinel data.</p>

scholarly journals Importance of interpolation and coincidence errors in data fusion

2017 ◽  
Author(s):  
Simone Ceccherini ◽  
Bruno Carli ◽  
Cecilia Tirelli ◽  
Nicola Zoppetti ◽  
Samuele Del Bianco ◽  
...  
Keyword(s):  
Data Fusion ◽  
Degrees Of Freedom ◽  
Thermal Infrared ◽  
Complete Data ◽  
Forward Model ◽  
Fusion Method ◽  
Model Errors ◽  
The Impact ◽  
Vertical Grids

Abstract. The Complete Data Fusion method is applied to ozone profiles obtained from simulated measurements in the ultraviolet and in the thermal infrared in the framework of the Sentinel 4 mission of the Copernicus programme. We observe that the quality of the fused products is degraded when the fusing profiles are either retrieved on different vertical grids or referred to different true profiles. To address this shortcoming, a generalization of the complete data fusion method, which takes into account interpolation and coincidence errors, is presented. This upgrade overcomes the encountered problems and provides products of good quality when the fusing profiles are both retrieved on different vertical grids and referred to different true profiles. The impact of the interpolation and coincidence errors on number of degrees of freedom and errors of the fused profile is also analyzed. The approach developed here to account for the interpolation and coincidence errors can also be followed to include other error components, such as forward model errors.

scholarly journals The cost function of the data fusion process and its application

2019 ◽  
Vol 12 (5) ◽  
pp. 2967-2977 ◽  
Author(s):  
Simone Ceccherini ◽  
Nicola Zoppetti ◽  
Bruno Carli ◽  
Ugo Cortesi ◽  
Samuele Del Bianco ◽  
...  
Keyword(s):  
Data Fusion ◽  
Cost Function ◽  
Covariance Matrix ◽  
Expected Value ◽  
Covariance Matrices ◽  
Complete Data ◽  
Sufficient Statistics ◽  
The Cost ◽  
Analytical Expressions

Abstract. When the complete data fusion method is used to fuse inconsistent measurements, it is necessary to add to the measurement covariance matrix of each fusing profile a covariance matrix that takes into account the inconsistencies. A realistic estimate of these inconsistency covariance matrices is required for effectual fused products. We evaluate the possibility of assisting the estimate of the inconsistency covariance matrices using the value of the cost function minimized in the complete data fusion. The analytical expressions of expected value and variance of the cost function are derived. Modelling the inconsistency covariance matrix with one parameter, we determine the value of the parameter that makes the reduced cost function equal to its expected value and use the variance to assign an error to this determination. The quality of the inconsistency covariance matrix determined in this way is tested for simulated measurements of ozone profiles obtained in the thermal infrared in the framework of the Sentinel-4 mission of the Copernicus programme. As expected, the method requires sufficient statistics and poor results are obtained when a small number of profiles are being fused together, but very good results are obtained when the fusion involves a large number of profiles.

scholarly journals The cost function of the data fusion process and its application

2019 ◽  
Author(s):  
Simone Ceccherini ◽  
Nicola Zoppetti ◽  
Bruno Carli ◽  
Ugo Cortesi ◽  
Samuele Del Bianco ◽  
...  
Keyword(s):  
Data Fusion ◽  
Cost Function ◽  
Covariance Matrix ◽  
Expected Value ◽  
Covariance Matrices ◽  
Complete Data ◽  
Sufficient Statistics ◽  
The Cost ◽  
Analytical Expressions

Abstract. When the complete data fusion method is used to fuse inconsistent measurements, it is necessary to add to the measurement covariance matrix of each fusing profile a covariance matrix that takes into account the inconsistencies. A realistic estimate of these inconsistency covariance matrices is required for effectual fused products. We evaluate the possibility of assisting the estimate of the inconsistency covariance matrices using the value of the cost function minimized in the complete data fusion. The analytical expressions of expected value and variance of the cost function are derived. Modelling the inconsistency covariance matrix with one parameter, we determine the value of the parameter that makes the reduced cost function equal to its expected value and use the variance to assign an error to this determination. The quality of the inconsistency covariance matrix determined in this way is tested for simulated measurements of ozone profiles obtained in the thermal infrared in the framework of the Sentinel 4 mission of the Copernicus programme. As expected, the method requires a sufficient statistics and poor results are obtained when a small numbers of profiles are being fused together, but very good results are obtained when the fusion involves a large number of profiles.

scholarly journals Comment on “Synergetic use of IASI and TROPOMI space borne sensors for generating a tropospheric methane profile product”

2021 ◽  
Author(s):  
Simone Ceccherini
Keyword(s):  
Kalman Filter ◽  
Data Fusion ◽  
Covariance Matrices ◽  
Complete Data ◽  
Fusion Method ◽  
Error Covariance ◽  
Noise Error

Abstract. A great interest is growing about methods that combine measurements from two or more instruments that observe the same species either in different spectral regions or with different geometries. Recently, a method based on the Kalman filter has been proposed to combine IASI and TROPOMI methane products. We show that this method is equivalent to the Complete Data Fusion method. Therefore, the choice between these two methods is driven only by the advantages of the different implementations. From the comparison of the two methods a generalization of the Complete Data Fusion formula, which is valid also in the case that the noise error covariance matrices of the fused products are singular, is derived.

scholarly journals Importance of interpolation and coincidence errors in data fusion

2018 ◽  
Vol 11 (2) ◽  
pp. 1009-1017 ◽  
Author(s):  
Simone Ceccherini ◽  
Bruno Carli ◽  
Cecilia Tirelli ◽  
Nicola Zoppetti ◽  
Samuele Del Bianco ◽  
...  
Keyword(s):  
Data Fusion ◽  
Degrees Of Freedom ◽  
Thermal Infrared ◽  
Complete Data ◽  
Forward Model ◽  
Fusion Method ◽  
Model Errors ◽  
The Impact ◽  
Vertical Grids

Abstract. The complete data fusion (CDF) method is applied to ozone profiles obtained from simulated measurements in the ultraviolet and in the thermal infrared in the framework of the Sentinel 4 mission of the Copernicus programme. We observe that the quality of the fused products is degraded when the fusing profiles are either retrieved on different vertical grids or referred to different true profiles. To address this shortcoming, a generalization of the complete data fusion method, which takes into account interpolation and coincidence errors, is presented. This upgrade overcomes the encountered problems and provides products of good quality when the fusing profiles are both retrieved on different vertical grids and referred to different true profiles. The impact of the interpolation and coincidence errors on number of degrees of freedom and errors of the fused profile is also analysed. The approach developed here to account for the interpolation and coincidence errors can also be followed to include other error components, such as forward model errors.

scholarly journals Data Fusion Analysis of Sentinel-4 and Sentinel-5 Simulated Ozone Data

2020 ◽  
Vol 37 (4) ◽  
pp. 573-587 ◽  
Author(s):  
Cecilia Tirelli ◽  
Simone Ceccherini ◽  
Nicola Zoppetti ◽  
Samuele Del Bianco ◽  
Marco Gai ◽  
...  
Keyword(s):  
Data Fusion ◽  
Low Earth Orbit ◽  
Complete Data ◽  
Earth Orbit ◽  
Fusion Algorithm ◽  
Ozone Data ◽  
Infrared Spectral ◽  
Fusion Analysis ◽  
Spectral Ranges

AbstractThe complete data fusion method, generalized to the case of fusing profiles of atmospheric variables retrieved on different vertical grids and referred to different true values, is applied to ozone profiles retrieved from simulated measurements in the ultraviolet, visible, and thermal infrared spectral ranges for the Sentinel-4 and Sentinel-5 missions of the Copernicus program. In this study, the production and characterization of combined low Earth orbit (Sentinel-5) and geostationary Earth orbit (Sentinel-4) fused ozone data is performed. Fused and standard products have been compared and a performance assessment of the generalized complete data fusion is presented. The analysis of the output products of the complete data fusion algorithm and of the standard processing using quality quantifiers demonstrates that the generalized complete data fusion algorithm provides products of better quality when compared with standard products.

scholarly journals Feature Selection and Classification for High-Dimensional Incomplete Multimodal Data

2018 ◽  
Vol 2018 ◽  
pp. 1-9 ◽  
Author(s):  
Wan-Yu Deng ◽  
Dan Liu ◽  
Ying-Ying Dong
Keyword(s):  
Feature Selection ◽  
Data Fusion ◽  
Classification Accuracy ◽  
Missing Values ◽  
High Dimensional Data ◽  
Complete Data ◽  
Experimental Results ◽  
High Dimensional ◽  
Multimodal Data ◽  
Fusion Methods

Due to missing values, incomplete dataset is ubiquitous in multimodal scene. Complete data is a prerequisite of the most existing multimodality data fusion methods. For incomplete multimodal high-dimensional data, we propose a feature selection and classification method. Our method mainly focuses on extracting the most relevant features from the high-dimensional features and then improving the classification accuracy. The experimental results show that our method produces considerably better performance on incomplete multimodal data such as ADNI dataset and Office dataset, compared to the case of complete data.

scholarly journals Supplementary material to "The Complete Data Fusion for a Full Exploitation of Copernicus Atmospheric Sentinel Level 2 Products"

2019 ◽  
Author(s):  
Nicola Zoppetti ◽  
Simone Ceccherini ◽  
Bruno Carli ◽  
Samuele Del Bianco ◽  
Marco Gai ◽  
...  
Keyword(s):  
Data Fusion ◽  
Complete Data ◽  
Supplementary Material ◽  
Level 2
Sign in / Sign up

Export Citation Format

Share Document