K nearest neighbour in merging satellite rainfall estimates from diverse sources in sparsely gauged basins

2021 ◽  
Author(s):  
Biswa Bhattacharya ◽  
Junaid Ahmad
Keyword(s):  
Bias Correction ◽  
Spatial Scales ◽  
Error Variance ◽  
Indus Basin ◽  
Nearest Neighbour ◽  
Data Merging ◽  
Basin Scale ◽  
Satellite Rainfall ◽  
Merging Method ◽  
Rainfall Estimates

<p>Satellite based rainfall estimates (SBRE) are used as an alternative to gauge rainfall in hydrological studies particularly for basins with data issues. However, these data products exhibit errors which cannot always be corrected by bias correction methods such as Ratio Bias Correction (RBC). Data fusion or data merging can be a potentially good approach in merging various satellite rainfall products to obtain a fused dataset, which can benefit from all the data sources and may minimise the error in rainfall estimates. Data merging methods which are commonly applied in meteorology and hydrology are: Arithmetic merging method (AMM), Inverse error squared weighting (IESW) and Error variance (EV). Among these methods EV is popular, which merges can be used to merge bias corrected SBREs using the minimisation of variance principle.</p><p>In this research we investigated the possibility of using K nearest neighbour as a data merging method. Four satellite rainfall products were used in this study namely CMORPH, PERSIANN CDR, TRMM 3B42 and MSWEP. MSWEP was used as a reference dataset for comparing the merged rainfall dataset since it is also a merged product. All these products were downloaded at 0.25° x 0.25° spatial scale and daily temporal scale. Satellite products are known to behave differently at different temporal and spatial scales. Based on the climatic and physiographic features the Indus basin was divided into four zones. Rainfall products were compared at daily, weekly, fortnightly, monthly and seasonal scales whereas spatial scales were gauge location, zonal scales and basin scale. The RBC method was used to correct the biasness of satellite products by correcting the products at monthly and seasonal scale. Wth bias correction the daily normalised root mean square error (NRMSE) was reduced up to 20% for CMORPH, 22% for PERSIANN CDR and 14% for TRMM at the Indus basin scale for monthly scale which is why the monthly bias corrected data was used for merging. Merging of satellite products can be fruitful to benefit from the strength of each product and minimize the weakness of products. Four different merging methods i.e. Arithmetic merging method (AMM), Inverse error squared weighting (IESW), Error variance (EV) and K Nearest Neighbour method (KNN) were used and performance was checked in two seasons i.e. non-wet and wet season. AMM and EV methods performed similarly whereas IESW performed poorly at zonal scales. KNN merging method outperformed all other merging methods and gave lowest error across the basin. Daily NRMSE was reduced to 0.3 at Indus basin scale with KNN method, AMM and EV reduced the error to 0.45 in comparison to error produced by CMORPH, PERSIANN CDR and TRMM of 0.8, 0.65 and 0.53 respectively in the wet season. KNN merged product gave lowest error at daily scale in calibration and validation period which justifies that merging improves rainfall estimates in sparsely gauged basin.</p><p> </p><p><strong>Key words:</strong> Merging, data fusion, K nearest neighbour, KNN, error variance, Indus.</p>

Merging of satellite rainfall estimates from diverse sources with K nearest neighbour in sparsely gauged basins

2020 ◽  
Author(s):  
Biswa Bhattacharya ◽  
Junaid Ahmad
Keyword(s):  
Error Variance ◽  
Indus Basin ◽  
Nearest Neighbour ◽  
Data Merging ◽  
Basin Scale ◽  
Satellite Rainfall ◽  
Data Point ◽  
Gauged Basins ◽  
Merging Method ◽  
Rainfall Estimates

<p>Satellite based rainfall estimates (SBRE) are used as an alternative to gauge rainfall in hydrological studies particularly for basins with data issues. However, these data products exhibit errors which cannot be always corrected by bias correction methods such as Ratio Bias Correction (RBC). Data fusion or data merging can be a potentially good approach in merging various SBREs to obtain a fused dataset, which can benefit from all the data sources and may minimise the error in rainfall estimates. Data merging methods which are commonly applied in meteorology and hydrology are: Arithmetic merging method (AMM), Inverse error squared weighting (IESW) and Error variance (EV). Among these methods EV is popular, which merges bias corrected SBREs using the minimisation of variance principle.</p><p>In this research we propose using K Nearest Neighbour (KNN) as a data merging method. KNN has a particular advantage as it does not depend upon any specific statistical model to merge data and presents a great flexibility as the value of K (the number of neighbours to be chosen) can be varied to suit the purpose (for example, choosing different K values for different seasons). In this research it is proposed to compute the distances of bias corrected SBREs of the training data from the gauge data and to assign the SBRE with the minimum distance as the class C where C = 1, 2, 3,…, number of SBREs. In validation each data point consisting of a value of each SBRE may be compared with the data points from the training set and the class of the data point(s) closest to this data point is assigned as the class of the validation data point.</p><p>The KNN approach as a data merging method was applied to the Indus basin in Pakistan. Three satellite rainfall products CMORPH, PERSIANN CDR and TRMM 3B42 with 0.25° x 0.25° spatial and daily temporal resolution were used. Based on the climatic and physiographic features the Indus basin was divided into four zones. Rainfall products were compared at daily, weekly, fortnightly, monthly and seasonally whereas spatial scales were gauge location, zonal scales and basin scale. The RBC method was used to correct the bias. The KNN method with K=1, 3 and 5 was used and compared with other merging methods namely AMM, IESW and EV. The results were compared in two seasons i.e. non-wet and wet season. AMM and EV methods performed similarly whereas IESW performed poorly at zonal scales. KNN merging method outperformed all other merging methods and gave lowest error across the basin. The daily normalised root mean square error at the Indus basin scale was reduced to 0.3, 0.45 and 0.45 respectively with KNN, AMM and EV whereas this error was 0.8, 0.65 and 0.53 respectively in CMORPH, PERSIANN CDR and TRMM datasets. The KNN merged product gave lowest error at daily scale in calibration and validation period which justifies that merging with KNN improves rainfall estimates in sparsely gauged basins.</p><p> </p><p><strong>Key words:</strong> Merging, data fusion, K nearest neighbour, KNN, error variance, Indus.</p>

scholarly journals Evaluation of High-Resolution Satellite Rainfall Products through Streamflow Simulation in a Hydrological Modeling of a Small Mountainous Watershed in Ethiopia

2012 ◽  
Vol 13 (1) ◽  
pp. 338-350 ◽  
Author(s):  
Menberu M. Bitew ◽  
Mekonnen Gebremichael ◽  
Lula T. Ghebremichael ◽  
Yared A. Bayissa
Keyword(s):  
High Resolution ◽  
Real Time ◽  
Bias Correction ◽  
Hydrological Modeling ◽  
Rain Gauge ◽  
Hydrologic Model ◽  
Model Simulations ◽  
Mountainous Watershed ◽  
Satellite Rainfall ◽  
Rainfall Estimates

Abstract This study focuses on evaluating four widely used global high-resolution satellite rainfall products [the Climate Prediction Center’s morphing technique (CMORPH) product, the Tropical Rainfall Measuring Mission (TRMM) Multisatellite Precipitation Analysis (TMPA) near-real-time product (3B42RT), the TMPA method post-real-time research version product (3B42), and the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) product] with a spatial resolution of 0.25° and temporal resolution of 3 h through their streamflow simulations in the Soil and Water Assessment Tool (SWAT) hydrologic model of a 299-km2 mountainous watershed in Ethiopia. Results show significant biases in the satellite rainfall estimates. The 3B42RT and CMORPH products perform better than the 3B42 and PERSIANN. The predictive ability of each of the satellite rainfall was examined using a SWAT model calibrated in two different approaches: with rain gauge rainfall as input, and with each of the satellite rainfall products as input. Significant improvements in model streamflow simulations are obtained when the model is calibrated with input-specific rainfall data than with rain gauge data. Calibrating SWAT with satellite rainfall estimates results in curve number values that are by far higher than the standard tabulated values, and therefore caution must be exercised when using standard tabulated parameter values with satellite rainfall inputs. The study also reveals that bias correction of satellite rainfall estimates significantly improves the model simulations. The best-performing model simulations based on satellite rainfall inputs are obtained after bias correction and model recalibration.

scholarly journals Bayesian Bias Correction of Satellite Rainfall Estimates for Climate Studies

2018 ◽  
Vol 10 (7) ◽  
pp. 1074 ◽  
Author(s):  
Margaret Kimani ◽  
Joost Hoedjes ◽  
Zhongbo Su
Keyword(s):  
Bias Correction ◽  
Satellite Rainfall ◽  
Climate Studies ◽  
Rainfall Estimates

scholarly journals Bias correction of satellite rainfall estimation using a radar-gauge product

2010 ◽  
Vol 7 (6) ◽  
pp. 8913-8945 ◽  
Author(s):  
K. Tesfagiorgis ◽  
S. E. Mahani ◽  
R. Khanbilvardi
Keyword(s):  
Bias Correction ◽  
Bias Field ◽  
Systematic Errors ◽  
Daily Basis ◽  
Random Errors ◽  
Field Generation ◽  
Shuffled Complex Evolution ◽  
Hydrologic Prediction ◽  
Satellite Rainfall ◽  
Rainfall Estimates

Abstract. Satellite rainfall estimates can be used in operational hydrologic prediction, but are prone to systematic errors. The goal of this study is to seamlessly blend a radar-gauge product with a corrected satellite product that fills gaps in radar coverage. To blend different rainfall products, they should have similar bias features. The paper presents a pixel by pixel method, which aims to correct biases in hourly satellite rainfall products using a radar-gauge rainfall product. Bias factors are calculated for corresponding rainy pixels, and a desired number of them are randomly selected for the analysis. Bias fields are generated using the selected bias factors. The method takes into account spatial variation and random errors in biases. Bias field parameters were determined on a daily basis using the Shuffled Complex Evolution optimization algorithm. To include more sources of errors, ensembles of bias factors were generated and applied before bias field generation. The procedure of the method was demonstrated using a satellite and a radar-gauge rainfall data for several rainy events in 2006 for the Oklahoma region. The method was compared with bias corrections using interpolation without ensembles, the ratio of mean and maximum ratio. Results show the method outperformed the other techniques such as mean ratio, maximum ratio and bias field generation by interpolation.

scholarly journals Bias correction schemes for CMORPH satellite rainfall estimates in the Zambezi River Basin

2016 ◽  
Author(s):  
W. Gumindoga ◽  
T. H. M. Rientjes ◽  
A. T. Haile ◽  
H. Makurira ◽  
P. Reggiani
Keyword(s):  
River Basin ◽  
Bias Correction ◽  
Large Scale ◽  
Statistical Error ◽  
Correction Method ◽  
Correction Scheme ◽  
Satellite Rainfall ◽  
Zambezi River Basin ◽  
Rainfall Estimates ◽  
Zambezi River

Abstract. Obtaining reliable records of rainfall from satellite rainfall estimates (SREs) is a challenge as SREs are an indirect rainfall estimate from visible, infrared (IR), and/or microwave (MW) based information of cloud properties. SREs also contain inherent biases which exaggerate or underestimate actual rainfall values hence the need to apply bias correction methods to improve accuracies. We evaluate the performance of five bias correction schemes for CMORPH satellite-based rainfall estimates. We use 54 raingauge stations in the Zambezi Basin for the period 1998–2013 for comparison and correction. Analysis shows that SREs better match to gauged estimates in the Upper Zambezi Basin than the Lower and Middle Zambezi basins but performance is not clearly related to elevation. Findings indicate that rainfall in the Upper Zambezi Basin is best estimated by an additive bias correction scheme (Distribution transformation). The linear based (Spatio-temporal) bias correction scheme successfully corrected the daily mean of CMORPH estimates for 70 % of the stations and also was most effective in reducing the rainfall bias. The nonlinear bias correction schemes (Power transform and the Quantile based empirical-statistical error correction method) proved most effective in reproducing the rainfall totals. Analyses through bias correction indicate that bias of CMORPH estimates has elevation and seasonality tendencies across the Zambezi river basin area of large scale.

scholarly journals Effect of Bias Correction of Satellite-Rainfall Estimates on Runoff Simulations at the Source of the Upper Blue Nile

2014 ◽  
Vol 6 (7) ◽  
pp. 6688-6708 ◽  
Author(s):  
Emad Habib ◽  
Alemseged Haile ◽  
Nazmus Sazib ◽  
Yu Zhang ◽  
Tom Rientjes
Keyword(s):  
Bias Correction ◽  
Blue Nile ◽  
Satellite Rainfall ◽  
Rainfall Estimates

scholarly journals A susceptibility-based rainfall threshold approach for landslide occurrence

2019 ◽  
Vol 19 (4) ◽  
pp. 775-789 ◽  
Author(s):  
Elise Monsieurs ◽  
Olivier Dewitte ◽  
Alain Demoulin
Keyword(s):  
Landslide Susceptibility ◽  
Rainfall Threshold ◽  
Exceedance Probability ◽  
Equal Weight ◽  
Antecedent Rainfall ◽  
Rainfall Thresholds ◽  
Landslide Occurrence ◽  
Continental Scale ◽  
Satellite Rainfall ◽  
Rainfall Estimates

Abstract. Rainfall threshold determination is a pressing issue in the landslide scientific community. While major improvements have been made towards more reproducible techniques for the identification of triggering conditions for landsliding, the now well-established rainfall intensity or event-duration thresholds for landsliding suffer from several limitations. Here, we propose a new approach of the frequentist method for threshold definition based on satellite-derived antecedent rainfall estimates directly coupled with landslide susceptibility data. Adopting a bootstrap statistical technique for the identification of threshold uncertainties at different exceedance probability levels, it results in thresholds expressed as AR = (α±Δα)⋅S(β±Δβ), where AR is antecedent rainfall (mm), S is landslide susceptibility, α and β are scaling parameters, and Δα and Δβ are their uncertainties. The main improvements of this approach consist in (1) using spatially continuous satellite rainfall data, (2) giving equal weight to rainfall characteristics and ground susceptibility factors in the definition of spatially varying rainfall thresholds, (3) proposing an exponential antecedent rainfall function that involves past daily rainfall in the exponent to account for the different lasting effect of large versus small rainfall, (4) quantitatively exploiting the lower parts of the cloud of data points, most meaningful for threshold estimation, and (5) merging the uncertainty on landslide date with the fit uncertainty in a single error estimation. We apply our approach in the western branch of the East African Rift based on landslides that occurred between 2001 and 2018, satellite rainfall estimates from the Tropical Rainfall Measurement Mission Multi-satellite Precipitation Analysis (TMPA 3B42 RT), and the continental-scale map of landslide susceptibility of Broeckx et al. (2018) and provide the first regional rainfall thresholds for landsliding in tropical Africa.

scholarly journals Temperature Projections over the Indus River Basin of Pakistan Using Statistical Downscaling

Atmosphere ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 195
Author(s):  
Muhammad Saleem Pomee ◽  
Elke Hertig
Keyword(s):  
21St Century ◽  
Statistical Downscaling ◽  
General Circulation ◽  
Winter Season ◽  
Circulation Model ◽  
Indus Basin ◽  
Indus River ◽  
Temperature Changes ◽  
Basin Scale ◽  
Maximum Heating

We assessed maximum (Tmax) and minimum (Tmin) temperatures over Pakistan’s Indus basin during the 21st century using statistical downscaling. A particular focus was given to spatiotemporal heterogeneity, reference and General Circulation Model (GCM) uncertainties, and statistical skills of regression models using an observational profile that could significantly be improved by recent high-altitude observatories. First, we characterized the basin into homogeneous climate regions using K-means clustering. Predictors from ERA-Interim reanalysis were then used to model observed temperatures skillfully and quantify reference and GCM uncertainties. Thermodynamical (dynamical) variables mainly governed reference (GCM) uncertainties. The GCM predictors under RCP4.5 and RCP8.5 scenarios were used as “new” predictors in statistical models to project ensemble temperature changes. Our analysis projected non-uniform warming but could not validate elevation-dependent warming (EDW) at the basin scale. We obtained more significant warming during the westerly-dominated seasons, with maximum heating during the winter season through Tmin changes. The most striking feature is a low-warming monsoon (with the possibility of no change to slight cooling) over the Upper Indus Basin (UIB). Therefore, the likelihood of continuing the anomalous UIB behavior during the primary melt season may not entirely be ruled out at the end of the 21st century under RCP8.5.

Identification of Synchronicity in Deterministic Chaotic Attractors for the Downscaling Process in the Bogotá River Basin

2021 ◽  
Author(s):  
Santiago Duarte ◽  
Gerald Corzo ◽  
Germán Santos
Keyword(s):  
Time Series ◽  
Time Delay ◽  
River Basin ◽  
Spatial Scales ◽  
Climatic Variables ◽  
Chaotic Attractors ◽  
Multiple Attractors ◽  
Nearest Neighbours ◽  
Basin Scale ◽  
Non Linear

<p>Bogotá’s River Basin, it’s an important basin in Cundinamarca, Colombia’s central region. Due to the complexity of the dynamical climatic system in tropical regions, can be difficult to predict and use the information of GCMs at the basin scale. This region is especially influenced by ENSO and non-linear climatic oscillation phenomena. Furthermore, considering that climatic processes are essentially non-linear and possibly chaotic, it may reduce the effectiveness of downscaling techniques in this region. </p><p>In this study, we try to apply chaotic downscaling to see if we could identify synchronicity that will allow us to better predict. It was possible to identify clearly the best time aggregation that can capture at the best the maximum relations between the variables at different spatial scales. Aside this research proposes a new combination of multiple attractors. Few analyses have been made to evaluate the existence of synchronicity between two or more attractors. And less analysis has considered the chaotic behaviour in attractors derived from climatic time series at different spatial scales. </p><p>Thus, we evaluate general synchronization between multiple attractors of various climate time series. The Mutual False Nearest Neighbours parameter (MFNN) is used to test the “Synchronicity Level” (existence of any type of synchronization) between two different attractors. Two climatic variables were selected for the analysis: Precipitation and Temperature. Likewise, two information sources are used: At the basin scale, local climatic-gauge stations with daily data and at global scale, the output of the MPI-ESM-MR model with a spatial resolution of 1.875°x1.875° for both climatic variables (1850-2005). In the downscaling process, two RCP (Representative Concentration Pathways)  scenarios are used, RCP 4.5 and RCP 8.5.</p><p>For the attractor’s reconstruction, the time-delay is obtained through the  Autocorrelation and the Mutual Information functions. The False Nearest Neighbors method (FNN) allowed finding the embedding dimension to unfold the attractor. This information was used to identify deterministic chaos at different times (e.g. 1, 2, 3 and 5 days) and spatial scales using the Lyapunov exponents. These results were used to test the synchronicity between the various chaotic attractor’s sets using the MFNN method and time-delay relations. An optimization function was used to find the attractor’s distance relation that increases the synchronicity between the attractors.  These results provided the potential of synchronicity in chaotic attractors to improve rainfall and temperature downscaling results at aggregated daily-time steps. Knowledge of loss information related to multiple reconstructed attractors can provide a better construction of downscaling models. This is new information for the downscaling process. Furthermore, synchronicity can improve the selection of neighbours for nearest-neighbours methods looking at the behaviour of synchronized attractors. This analysis can also allow the classification of unique patterns and relationships between climatic variables at different temporal and spatial scales.</p>

Sign in / Sign up

Export Citation Format

Share Document