Data-driven scale extrapolation: estimating yearly discharge for a large region by small sub-basins

Abstract. Large-scale hydrological models and land surface models are so far the only tools for assessing current and future water resources. Those models estimate discharge with large uncertainties, due to the complex interaction between climate and hydrology, the limited availability and quality of data, as well as model uncertainties. A new purely data-driven scale-extrapolation method to estimate discharge for a large region solely from selected small sub-basins, which are typically 1–2 orders of magnitude smaller than the large region, is proposed. Those small sub-basins contain sufficient information, not only on climate and land surface, but also on hydrological characteristics for the large basin. In the Baltic Sea drainage basin, best discharge estimation for the gauged area was achieved with sub-basins that cover 5% of the gauged area. There exist multiple sets of sub-basins whose climate and hydrology resemble those of the gauged area equally well. Those multiple sets estimate annual discharge for the gauged area consistently well with 6 % average error. The scale-extrapolation method is completely data-driven; therefore it does not force any modelling error into the prediction. The multiple predictions are expected to bracket the inherent variations and uncertainties of the climate and hydrology of the basin.

Download Full-text

Data-based discharge extrapolation: estimating annual discharge for a partially gauged large river basin from its small sub-basins

Hydrology and Earth System Sciences Discussions ◽

10.5194/hessd-9-6829-2012 ◽

2012 ◽

Vol 9 (6) ◽

pp. 6829-6856

Author(s):

L. Gong

Keyword(s):

Water Resources ◽

Land Surface ◽

Large Scale ◽

Drainage Basin ◽

Large River ◽

Extrapolation Method ◽

Average Error ◽

Sufficient Information ◽

Baltic Sea Drainage Basin ◽

Annual Discharge

Abstract. Large-scale hydrological models and land surface models are by far the only tools for accessing future water resources in climate change impact studies. Those models estimate discharge with large uncertainties, due to the complex interaction between climate and hydrology, the limited quality and availability of data, as well as model uncertainties. A new purely data-based scale-extrapolation method is proposed, to estimate water resources for a large basin solely from selected small sub-basins, which are typically two-orders-of-magnitude smaller than the large basin. Those small sub-basins contain sufficient information, not only on climate and land surface, but also on hydrological characteristics for the large basin In the Baltic Sea drainage basin, best discharge estimation for the gauged area was achieved with sub-basins that cover 2–4% of the gauged area. There exist multiple sets of sub-basins that resemble the climate and hydrology of the basin equally well. Those multiple sets estimate annual discharge for gauged area consistently well with 5% average error. The scale-extrapolation method is completely data-based; therefore it does not force any modelling error into the prediction. The multiple predictions are expected to bracket the inherent variations and uncertainties of the climate and hydrology of the basin. The method can be applied in both un-gauged basins and un-gauged periods with uncertainty estimation.

Download Full-text

"Blueprint" for the UP Modelling System for Large Scale Hydrology

Hydrology and Earth System Sciences ◽

10.5194/hess-1-55-1997 ◽

1997 ◽

Vol 1 (1) ◽

pp. 55-69 ◽

Cited By ~ 17

Author(s):

J. Ewen

Keyword(s):

Land Surface ◽

Large Scale ◽

Weather Forecasting ◽

Hydrological Cycle ◽

Sufficient Information ◽

Forecasting Models ◽

Wide Range ◽

Physically Based ◽

Up Elements ◽

The Impact

Abstract. There are at least two needs to be met by the current research efforts on large scale hydrological modelling. The first is for practical conceptual land-surface hydrology schemes for use with existing operational climate and weather forecasting models, to replace the overly simple schemes often used in such models. The second is for models of large scale hydrology which are properly sensitive to changes in physical properties and inputs measured (or predicted) over a wide range of scales, from the point-scale upwards, yet are simple enough in structure to be coupled to climate and weather forecasting models. Such models of large scale hydrology are needed for studying the environmental impact of pollution and changes in climate and land-use, especially the impact On water resources. The UP system (name derived from Upsealed Physically-based) is an attempt to satisfy the second need. It uses a physically-based approach and has a simple structure, yet incorporates sufficient information on sub-grid behaviour to make it a useful tool for the study of environmental impacts over a wide range of scales. The system uses a new approach to large scale modelling, giving physically-based predictions of hourly flows, storages, saturated areas, etc., for regions covering hundreds of thousands of square kilometres. The basic component of the system is the UP element. This has seven water storage compartments (one each for the snowpack, vegetation canopy, surface water, root zone, unsaturated percolation, interflow and groundwater) and allows all the main processes of the terrestrial phase of the hydrological cycle to be represented. A region is modelled as a collection of UP elements, linked by a river routing scheme. Each compartment represents a fixed zone within the area covered by the UP element, and each is related to a physical process such as groundwater flow. Most of the parameterizations for the compartments are in the form of look-up tables, linking the outputs from the compartments to state variables such as the current storage in the compartment. These parameterizations are, in the main, derived from results from physically-based, distributed models applied to the zones (e.g. a groundwater compartment is parameterized using a groundwater model). For large regions modelled using many UP elements, the UP parameters are regionalized using a classification scheme, thus reducing the overall effort spent in parameterization. The development of the UP system is a long-term project involving research into physically-based parameterization of large scale hydrology models, including the effects of sub-grid spatial variations. The first stage involved developing a "blueprint" for the UP element, based on experience with physically-based, distributed river basin modelling and reviews of existing techniques and modelling approaches for large scale and linked atmosphere-hydrology modelling. This paper describes the UP element and the concepts and ideas behind the development of the UP system and, briefly, describes some of the research and development work currently in progress on UP and its parameterization.

Download Full-text

A Fused Variable by Night Light Images and MODIS Products for Improving Urban Built-Up Area Extraction

Technologies ◽

10.3390/technologies9020040 ◽

2021 ◽

Vol 9 (2) ◽

pp. 40

Author(s):

Guang Yang ◽

Yuntao Ma ◽

Jiaqi Hu

Keyword(s):

Surface Temperature ◽

Land Surface ◽

Vegetation Index ◽

Normalized Difference Vegetation Index ◽

Extraction Methods ◽

Average Error ◽

Geographic Coordinate System ◽

Optimal Thresholds ◽

Moderate Resolution Imaging Spectroradiometer ◽

Spatial Locations

The boundary of urban built-up areas is the baseline data of a city. Rapid and accurate monitoring of urban built-up areas is the prerequisite for the boundary control and the layout of urban spaces. In recent years, the night light satellite sensors have been employed in urban built-up area extraction. However, the existing extraction methods have not fully considered the properties that directly reflect the urban built-up areas, like the land surface temperature. This research first converted multi-source data into a uniform projection, geographic coordinate system and resampling size. Then, a fused variable that integrated the Defense Meteorological Satellite Program/Operational Linescan System (DMSP/OLS) night light images, the Moderate-resolution Imaging Spectroradiometer (MODIS) surface temperature product and the normalized difference vegetation index (NDVI) product was designed to extract the built-up areas. The fusion results showed that the values of the proposed index presented a sharper gradient within a smaller spatial range, compared with the only night light images. The extraction results were tested in both the area sizes and the spatial locations. The proposed index performed better in both accuracies (average error rate 1.10%) and visual perspective. We further discussed the regularity of the optimal thresholds in the final boundary determination. The optimal thresholds of the proposed index were more stable in different cases on the premise of higher accuracies.

Download Full-text

Accelerating In-Transit Co-Processing for Scientific Simulations Using Region-Based Data-Driven Analysis

Algorithms ◽

10.3390/a14050154 ◽

2021 ◽

Vol 14 (5) ◽

pp. 154

Author(s):

Marcus Walldén ◽

Masao Okita ◽

Fumihiko Ino ◽

Dimitris Drikakis ◽

Ioannis Kokkinakis

Keyword(s):

Large Scale ◽

Data Driven ◽

Data Sets ◽

Output Constraints ◽

Data Driven Approach ◽

Scientific Simulations ◽

Multiple Metrics ◽

In Transit ◽

Multiple Compression ◽

Large Scale Simulations

Increasing processing capabilities and input/output constraints of supercomputers have increased the use of co-processing approaches, i.e., visualizing and analyzing data sets of simulations on the fly. We present a method that evaluates the importance of different regions of simulation data and a data-driven approach that uses the proposed method to accelerate in-transit co-processing of large-scale simulations. We use the importance metrics to simultaneously employ multiple compression methods on different data regions to accelerate the in-transit co-processing. Our approach strives to adaptively compress data on the fly and uses load balancing to counteract memory imbalances. We demonstrate the method’s efficiency through a fluid mechanics application, a Richtmyer–Meshkov instability simulation, showing how to accelerate the in-transit co-processing of simulations. The results show that the proposed method expeditiously can identify regions of interest, even when using multiple metrics. Our approach achieved a speedup of 1.29× in a lossless scenario. The data decompression time was sped up by 2× compared to using a single compression method uniformly.

Download Full-text

Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

International Journal of Artificial Intelligence in Education ◽

10.1007/s40593-021-00267-x ◽

2021 ◽

Author(s):

Ekaterina Kochmar ◽

Dung Do Vu ◽

Robert Belfer ◽

Varun Gupta ◽

Iulian Vlad Serban ◽

...

Keyword(s):

Machine Learning ◽

Student Performance ◽

Language Processing ◽

Intelligent Tutoring Systems ◽

Large Scale ◽

Intelligent Tutoring ◽

Performance Outcomes ◽

Data Driven ◽

Personalized Feedback ◽

Tutoring Systems

AbstractIntelligent tutoring systems (ITS) have been shown to be highly effective at promoting learning as compared to other computer-based instructional approaches. However, many ITS rely heavily on expert design and hand-crafted rules. This makes them difficult to build and transfer across domains and limits their potential efficacy. In this paper, we investigate how feedback in a large-scale ITS can be automatically generated in a data-driven way, and more specifically how personalization of feedback can lead to improvements in student performance outcomes. First, in this paper we propose a machine learning approach to generate personalized feedback in an automated way, which takes individual needs of students into account, while alleviating the need of expert intervention and design of hand-crafted rules. We leverage state-of-the-art machine learning and natural language processing techniques to provide students with personalized feedback using hints and Wikipedia-based explanations. Second, we demonstrate that personalized feedback leads to improved success rates at solving exercises in practice: our personalized feedback model is used in , a large-scale dialogue-based ITS with around 20,000 students launched in 2019. We present the results of experiments with students and show that the automated, data-driven, personalized feedback leads to a significant overall improvement of 22.95% in student performance outcomes and substantial improvements in the subjective evaluation of the feedback.

Download Full-text

Data-Driven Energy Use Estimation in Large Scale Transportation Networks

Proceedings of the 2nd ACM/EIGSCC Symposium on Smart Cities and Communities - SCC '19 ◽

10.1145/3357492.3358632 ◽

2019 ◽

Author(s):

Bin Wang ◽

Cy Chan ◽

Divya Somasi ◽

Jane Macfarlane ◽

Eric Rask

Keyword(s):

Large Scale ◽

Energy Use ◽

Transportation Networks ◽

Data Driven

Download Full-text

Improving the management of type 2 diabetes through large-scale general practice: the role of a data-driven and technology-enabled education programme

BMJ Open Quality ◽

10.1136/bmjoq-2020-001087 ◽

2021 ◽

Vol 10 (1) ◽

pp. e001087

Author(s):

Tarek F Radwan ◽

Yvette Agyako ◽

Alireza Ettefaghian ◽

Tahira Kamran ◽

Omar Din ◽

...

Keyword(s):

Type 2 Diabetes ◽

Primary Care ◽

Large Scale ◽

Education Programme ◽

Educational Programme ◽

Data Driven ◽

Treatment Targets ◽

Care Processes ◽

Data Driven Approach

A quality improvement (QI) scheme was launched in 2017, covering a large group of 25 general practices working with a deprived registered population. The aim was to improve the measurable quality of care in a population where type 2 diabetes (T2D) care had previously proved challenging. A complex set of QI interventions were co-designed by a team of primary care clinicians and educationalists and managers. These interventions included organisation-wide goal setting, using a data-driven approach, ensuring staff engagement, implementing an educational programme for pharmacists, facilitating web-based QI learning at-scale and using methods which ensured sustainability. This programme was used to optimise the management of T2D through improving the eight care processes and three treatment targets which form part of the annual national diabetes audit for patients with T2D. With the implemented improvement interventions, there was significant improvement in all care processes and all treatment targets for patients with diabetes. Achievement of all the eight care processes improved by 46.0% (p<0.001) while achievement of all three treatment targets improved by 13.5% (p<0.001). The QI programme provides an example of a data-driven large-scale multicomponent intervention delivered in primary care in ethnically diverse and socially deprived areas.

Download Full-text

Reservoir-Induced Land Deformation: Case Study from the Grand Ethiopian Renaissance Dam

Remote Sensing ◽

10.3390/rs13050874 ◽

2021 ◽

Vol 13 (5) ◽

pp. 874

Author(s):

Yu Chen ◽

Mohamed Ahmed ◽

Natthachet Tangdamrongsub ◽

Dorina Murgulet

Keyword(s):

Land Surface ◽

Large Scale ◽

Surface Deformation ◽

Vertical Displacement ◽

Nile River ◽

Reservoir Volume ◽

Novel Approach ◽

Land Deformation ◽

Large Scale Land ◽

The Impact

The Nile River stretches from south to north throughout the Nile River Basin (NRB) in Northeast Africa. Ethiopia, where the Blue Nile originates, has begun the construction of the Grand Ethiopian Renaissance Dam (GERD), which will be used to generate electricity. However, the impact of the GERD on land deformation caused by significant water relocation has not been rigorously considered in the scientific research. In this study, we develop a novel approach for predicting large-scale land deformation induced by the construction of the GERD reservoir. We also investigate the limitations of using the Gravity Recovery and Climate Experiment Follow On (GRACE-FO) mission to detect GERD-induced land deformation. We simulated three land deformation scenarios related to filling the expected reservoir volume, 70 km3, using 5-, 10-, and 15-year filling scenarios. The results indicated: (i) trends in downward vertical displacement estimated at −17.79 ± 0.02, −8.90 ± 0.09, and −5.94 ± 0.05 mm/year, for the 5-, 10-, and 15-year filling scenarios, respectively; (ii) the western (eastern) parts of the GERD reservoir are estimated to move toward the reservoir’s center by +0.98 ± 0.01 (−0.98 ± 0.01), +0.48 ± 0.00 (−0.48 ± 0.00), and +0.33 ± 0.00 (−0.33 ± 0.00) mm/year, under the 5-, 10- and 15-year filling strategies, respectively; (iii) the northern part of the GERD reservoir is moving southward by +1.28 ± 0.02, +0.64 ± 0.01, and +0.43 ± 0.00 mm/year, while the southern part is moving northward by −3.75 ± 0.04, −1.87 ± 0.02, and −1.25 ± 0.01 mm/year, during the three examined scenarios, respectively; and (iv) the GRACE-FO mission can only detect 15% of the large-scale land deformation produced by the GERD reservoir. Methods and results demonstrated in this study provide insights into possible impacts of reservoir impoundment on land surface deformation, which can be adopted into the GERD project or similar future dam construction plans.

Download Full-text

Cities Exacerbate Climate Warming

Urban Science ◽

10.3390/urbansci5010027 ◽

2021 ◽

Vol 5 (1) ◽

pp. 27

Author(s):

Lahouari Bounoua ◽

Kurtis Thome ◽

Joseph Nigro

Keyword(s):

Surface Temperature ◽

Land Surface ◽

Large Scale ◽

Climate Models ◽

Warming Trend ◽

Climate Mitigation ◽

Temperature Changes ◽

Surface Warming ◽

The Us

Urbanization is a complex land transformation not explicitly resolved within large-scale climate models. Long-term timeseries of high-resolution satellite data are essential to characterize urbanization within land surface models and to assess its contribution to surface temperature changes. The potential for additional surface warming from urbanization-induced land use change is investigated and decoupled from that due to change in climate over the continental US using a decadal timescale. We show that, aggregated over the US, the summer mean urban-induced surface temperature increased by 0.15 °C, with a warming of 0.24 °C in cities built in vegetated areas and a cooling of 0.25 °C in cities built in non-vegetated arid areas. This temperature change is comparable in magnitude to the 0.13 °C/decade global warming trend observed over the last 50 years caused by increased CO2. We also show that the effect of urban-induced change on surface temperature is felt above and beyond that of the CO2 effect. Our results suggest that climate mitigation policies must consider urbanization feedback to put a limit on the worldwide mean temperature increase.

Download Full-text