Synergy conformal prediction applied to large-scale bioactivity datasets and in federated learning

Confidence predictors can deliver predictions with the associated confidence required for decision making and can play an important role in drug discovery and toxicity predictions. In this work we investigate a recently introduced version of conformal prediction, synergy conformal prediction, focusing on the predictive performance when applied to bioactivity data. We compare the performance to other variants of conformal predictors for multiple partitioned datasets and demonstrate the utility of synergy conformal predictors for federated learning where data cannot be pooled in one location. Our results show that synergy conformal predictors based on training data randomly sampled with replacement can compete with other conformal setups, while using completely separate training sets often results in worse performance. However, in a federated setup where no method has access to all the data, synergy conformal prediction is shown to give promising results. Based on our study, we conclude that synergy conformal predictors are a valuable addition to the conformal prediction toolbox.

On the data synergy effect of large-sample multi-physics catchment modeling with machine learning

<p>Watersheds in the world are often perceived as being unique from each other, requiring customized study for each basin. Models uniquely built for each watershed, in general, cannot be leveraged for other watersheds. It is also a customary practice in hydrology and related geoscientific disciplines to divide the whole domain into multiple regimes and study each region separately, in an approach sometimes called regionalization or stratification. However, in the era of big-data machine learning, models can learn across regions and identify commonalities and differences. In this presentation, we first show that machine learning can derive highly functional continental-scale models for streamflow, evapotranspiration, and water quality variables. Next, through two hydrologic examples (soil moisture and streamflow), we argue that unification can often significantly outperform stratification, and systematically examine an effect we call data synergy, where the results of the DL models improved when data were pooled together from characteristically different regions and variables. In fact, the performance of the DL models benefited from some diversity in training data even with similar data quantity. However, allowing heterogeneous training data makes eligible much larger training datasets, which is an inherent advantage of DL. We also share our recent developments in advancing hydrologic deep learning and machine learning driven parameterization.</p>

Tropical Tropospheric Ozone from Sentinel-5P TROPOMI data: Synergy of CCD/CSL retrievals

<p>The TROPOspheric Monitoring Instrument (TROPOMI), on board the Sentinel 5 precursor (S5p) satellite, was launched in October 2017. The TROPOMI instrument has high spatial resolution and daily coverage of the Earth. About two years of level 2 data (versions up to 2.1.4) of OFFL GODFIT ozone and OCRA/ROCINN CRB (fraction and height) are available. Using these datasets, we derive tropical tropospheric ozone using the convective CCD cloud differential method for tropical tropospheric column ozone (TTCO) [DU] and the CSL cloud slicing method for upper tropospheric ozone volume mixing ratios (TUTO) [ppbv].</p><p>The CCD algorithm was optimized for TROPOMI with respect to the reference sector Above Cloud Column Ozone field (ACCO) by adjusting it in time and latitude space in order to reduce data gaps in the daily ACCO vectors. Daily total ozone gridded data with a latitude/longitude resolution of 0.5°/1° are used to minimize the error from stratospheric ozone changes.</p><p>The CSL algorithm (CHOVA: Cloud Height induced Ozone Variation Analysis) was developed to fully exploit the S5p instruments characteristics. The data is spatially sampled to a 2° latitude/longitude grid. A temporal sampling of cloud/ozone data is not necessary anymore due to the high amount of S5p measurements. Comparisons with NASA/GSFC SHADOZ ozone sondes show good agreement (low bias and high dispersion) for both methods taking into account the principal differences between sonde point measurements and satellite sampled mean value. The CHOVA results from the pacific sector are now used as input for the CCD method to adjust the height dependent columns to a fixed pressure level.    </p><p>The work on TROPOMI/S5P geophysical products is funded by ESA and national contributions from the Netherlands, Germany, Belgium, and Finland.</p>