A Review of Machine Learning Applications in Land Surface Modeling

Machine learning (ML), as an artificial intelligence tool, has acquired significant progress in data-driven research in Earth sciences. Land Surface Models (LSMs) are important components of the climate models, which help to capture the water, energy, and momentum exchange between the land surface and the atmosphere, providing lower boundary conditions to the atmospheric models. The objectives of this review paper are to highlight the areas of improvement in land modeling using ML and discuss the crucial ML techniques in detail. Literature searches were conducted using the relevant key words to obtain an extensive list of articles. The bibliographic lists of these articles were also considered. To date, ML-based techniques have been able to upgrade the performance of LSMs and reduce uncertainties by improving evapotranspiration and heat fluxes estimation, parameter optimization, better crop yield prediction, and model benchmarking. Widely used ML techniques used for these purposes include Artificial Neural Networks and Random Forests. We conclude that further improvements in land modeling are possible in terms of high-resolution data preparation, parameter calibration, uncertainty reduction, efficient model performance, and data assimilation using ML. In addition to the traditional techniques, convolutional neural networks, long short-term memory, and other deep learning methods can be implemented.

Download Full-text

A mass- and energy-conserving framework for using machine learning to speed computations: a photochemistry example

Geoscientific Model Development ◽

10.5194/gmd-13-4435-2020 ◽

2020 ◽

Vol 13 (9) ◽

pp. 4435-4442

Author(s):

Patrick Obin Sturm ◽

Anthony S. Wexler

Keyword(s):

Machine Learning ◽

Land Surface ◽

Climate Models ◽

Property Values ◽

Model Performance ◽

Chemical Properties ◽

Atmospheric Composition ◽

Mathematical Framework ◽

Environmental Models ◽

Computational Performance

Abstract. Large air quality models and large climate models simulate the physical and chemical properties of the ocean, land surface, and/or atmosphere to predict atmospheric composition, energy balance and the future of our planet. All of these models employ some form of operator splitting, also called the method of fractional steps, in their structure, which enables each physical or chemical process to be simulated in a separate operator or module within the overall model. In this structure, each of the modules calculates property changes for a fixed period of time; that is, property values are passed into the module, which calculates how they change for a period of time and then returns the new property values, all in round-robin between the various modules of the model. Some of these modules require the vast majority of the computer resources consumed by the entire model, so increasing their computational efficiency can either improve the model's computational performance, enable more realistic physical or chemical representations in the module, or a combination of these two. Recent efforts have attempted to replace these modules with ones that use machine learning tools to memorize the input–output relationships of the most time-consuming modules. One shortcoming of some of the original modules and their machine-learned replacements is lack of adherence to conservation principles that are essential to model performance. In this work, we derive a mathematical framework for machine-learned replacements that conserves properties – say mass, atoms, or energy – to machine precision. This framework can be used to develop machine-learned operator replacements in environmental models.

Download Full-text

A Mass- and Energy-Conserving Framework for Using Machine Learning to Speed Computations

10.5194/gmd-2020-83 ◽

2020 ◽

Author(s):

Patrick Obin Sturm ◽

Anthony S. Wexler

Keyword(s):

Machine Learning ◽

Land Surface ◽

Climate Models ◽

Property Values ◽

Model Performance ◽

Chemical Properties ◽

Atmospheric Composition ◽

Mathematical Framework ◽

Environmental Models ◽

Computational Performance

Abstract. Large air quality models and large climate models simulate the physical and chemical properties of the ocean, land surface and/or atmosphere to predict atmospheric composition, energy balance, and the future of our planet. All of these models employ some form of operator splitting, also called the method of fractional steps, in their structure, which enables each physical or chemical process to be simulated in a separate operator or module within the overall model. In this structure, each of the modules calculates property changes for a fixed period of time; that is, property values are passed into the module which calculates how they change for a period of time and then returns the new property values, all in round robin between the various modules of the model. Some of these modules require the vast majority of the computer resources consumed by the entire model so increasing their computational efficiency can either improve the model's computational performance or enable more realistic physical or chemical representations in the module, or a combination of these two. Recent efforts have attempted to replace these modules with ones that use machine learning tools to memorize the input-output relationships of the most time-consuming modules. One shortcoming of some of the original modules and their machine learned replacements is lack of adherence to conservation principles that are essential to model performance. In this work, we derive a mathematical framework for machine learned replacements that conserves properties, say mass, atoms, or energy, to machine precision. This framework can be used to develop machine learned operator replacements in environmental models.

Download Full-text

Predicting Future Occurrence of Acute Hypotensive Episodes Using Noninvasive and Invasive Features

Military Medicine ◽

10.1093/milmed/usaa418 ◽

2021 ◽

Vol 186 (Supplement_1) ◽

pp. 445-451

Author(s):

Yifei Sun ◽

Navid Rashedi ◽

Vikrant Vaze ◽

Parikshit Shah ◽

Ryan Halter ◽

...

Keyword(s):

Machine Learning ◽

Support Vector Machine ◽

Real World ◽

Short Term Memory ◽

Model Performance ◽

Learning Technologies ◽

Machine Learning Algorithms ◽

Support Vector ◽

K Nearest Neighbor ◽

Continuous Map

ABSTRACT Introduction Early prediction of the acute hypotensive episode (AHE) in critically ill patients has the potential to improve outcomes. In this study, we apply different machine learning algorithms to the MIMIC III Physionet dataset, containing more than 60,000 real-world intensive care unit records, to test commonly used machine learning technologies and compare their performances. Materials and Methods Five classification methods including K-nearest neighbor, logistic regression, support vector machine, random forest, and a deep learning method called long short-term memory are applied to predict an AHE 30 minutes in advance. An analysis comparing model performance when including versus excluding invasive features was conducted. To further study the pattern of the underlying mean arterial pressure (MAP), we apply a regression method to predict the continuous MAP values using linear regression over the next 60 minutes. Results Support vector machine yields the best performance in terms of recall (84%). Including the invasive features in the classification improves the performance significantly with both recall and precision increasing by more than 20 percentage points. We were able to predict the MAP with a root mean square error (a frequently used measure of the differences between the predicted values and the observed values) of 10 mmHg 60 minutes in the future. After converting continuous MAP predictions into AHE binary predictions, we achieve a 91% recall and 68% precision. In addition to predicting AHE, the MAP predictions provide clinically useful information regarding the timing and severity of the AHE occurrence. Conclusion We were able to predict AHE with precision and recall above 80% 30 minutes in advance with the large real-world dataset. The prediction of regression model can provide a more fine-grained, interpretable signal to practitioners. Model performance is improved by the inclusion of invasive features in predicting AHE, when compared to predicting the AHE based on only the available, restricted set of noninvasive technologies. This demonstrates the importance of exploring more noninvasive technologies for AHE prediction.

Download Full-text

Forecasting the risk at infractions: an ensemble comparison of machine learning approach

Industrial Management & Data Systems ◽

10.1108/imds-10-2020-0603 ◽

2021 ◽

Vol ahead-of-print (ahead-of-print) ◽

Author(s):

Lei Li ◽

Desheng Wu

Keyword(s):

Machine Learning ◽

Prediction Models ◽

Short Term Memory ◽

Model Performance ◽

Large Data ◽

Support Vector ◽

Learning Approaches ◽

Content Type ◽

Day To Day Operations ◽

Prediction Approach

PurposeThe infraction of securities regulations (ISRs) of listed firms in their day-to-day operations and management has become one of common problems. This paper proposed several machine learning approaches to forecast the risk at infractions of listed corporates to solve financial problems that are not effective and precise in supervision.Design/methodology/approachThe overall proposed research framework designed for forecasting the infractions (ISRs) include data collection and cleaning, feature engineering, data split, prediction approach application and model performance evaluation. We select Logistic Regression, Naïve Bayes, Random Forest, Support Vector Machines, Artificial Neural Network and Long Short-Term Memory Networks (LSTMs) as ISRs prediction models.FindingsThe research results show that prediction performance of proposed models with the prior infractions provides a significant improvement of the ISRs than those without prior, especially for large sample set. The results also indicate when judging whether a company has infractions, we should pay attention to novel artificial intelligence methods, previous infractions of the company, and large data sets.Originality/valueThe findings could be utilized to address the problems of identifying listed corporates' ISRs at hand to a certain degree. Overall, results elucidate the value of the prior infraction of securities regulations (ISRs). This shows the importance of including more data sources when constructing distress models and not only focus on building increasingly more complex models on the same data. This is also beneficial to the regulatory authorities.

Download Full-text

Applying neural network for identification of land surface model parameters

10.5194/egusphere-egu21-16355 ◽

2021 ◽

Author(s):

Ruslan Chernyshev ◽

Mikhail Krinitskiy ◽

Viktor Stepanenko

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Partial Differential Equations ◽

Differential Equations ◽

Thermodynamic Model ◽

Land Surface ◽

Land Surface Model ◽

Surface Model ◽

Partial Differential

This work is devoted to development of neural networks for identification of partial differential equations (PDE) solved in the land surface scheme of INM RAS Earth System model (ESM). Atmospheric and climate models are in the top of the most demanding for supercomputing resources among research applications. Spatial resolution and a multitude of physical parameterizations used in ESMs continuously increase. Most of parameters are still poorly constrained, many of them cannot be measured directly. To optimize model calibration time, using neural networks looks a promising approach. Neural networks are already in wide use in satellite imaginary (Su Jeong Lee, et al, 2015; Krinitskiy M. et al, 2018) and for calibrating parameters of land surface models (Yohei Sawada el al, 2019). Neural networks have demonstrated high efficiency in solving conventional problems of mathematical physics (Lucie P. Aarts el al, 2001; Raissi M. et al, 2020).&#160;We develop a neural networks for optimizing parameters of nonlinear soil heat and moisture transport equation set. For developing we used Python3 based programming tools implemented on GPUs and Ascend platform, provided by Huawei. Because of using hybrid approach combining neural network and classical thermodynamic equations, the major purpose was finding the way to correctly calculate backpropagation gradient of error function, because model trains and is being validated on the same temperature data, while model output is heat equation parameter, which is typically not known. Neural network model has been runtime trained using reference thermodynamic model calculation with prescribed parameters, every next thermodynamic model step has been used for fitting the neural network until it reaches the loss function tolerance.Literature:1. &#160; &#160; Aarts, L.P., van der Veer, P. &#8220;Neural Network Method for Solving Partial Differential Equations&#8221;. Neural Processing Letters 14, 261&#8211;271 (2001). https://doi.org/10.1023/A:10127841298832. &#160; &#160; Raissi, M., P. Perdikaris and G. Karniadakis. &#8220;Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations.&#8221; ArXiv abs/1711.10561 (2017): n. pag.3. &#160; &#160; Lee, S.J., Ahn, MH. & Lee, Y. Application of an artificial neural network for a direct estimation of atmospheric instability from a next-generation imager. Adv. Atmos. Sci. 33, 221&#8211;232 (2016). https://doi.org/10.1007/s00376-015-5084-94. &#160; &#160; Krinitskiy M, Verezemskaya P, Grashchenkov K, Tilinina N, Gulev S, Lazzara M. Deep Convolutional Neural Networks Capabilities for Binary Classification of Polar Mesocyclones in Satellite Mosaics. Atmosphere. 2018; 9(11):426.5. &#160; &#160; Sawada, Y.. &#8220;Machine learning accelerates parameter optimization and uncertainty assessment of a land surface model.&#8221; ArXiv abs/1909.04196 (2019): n. pag.6. &#160; &#160; Shufen Pan et al. Evaluation of global terrestrial evapotranspiration using state-of-the-art approaches in remote sensing, machine learning and land surface modeling. Hydrol. Earth Syst. Sci., 24, 1485&#8211;1509 (2020)7. &#160; &#160; Chaney, Nathaniel & Herman, Jonathan & Ek, M. & Wood, Eric. (2016). Deriving Global Parameter Estimates for the Noah Land Surface Model using FLUXNET and Machine Learning: Improving Noah LSM Parameters. Journal of Geophysical Research: Atmospheres. 121. 10.1002/2016JD024821.&#160;&#160;

Download Full-text

Using machine-learning risk prediction models to triage the acuity of undifferentiated patients entering the emergency care system: a systematic review

Diagnostic and Prognostic Research ◽

10.1186/s41512-020-00084-1 ◽

2020 ◽

Vol 4 (1) ◽

Author(s):

Jamie Miles ◽

Janette Turner ◽

Richard Jacques ◽

Julia Williams ◽

Suzanne Mason

Keyword(s):

Machine Learning ◽

Systematic Review ◽

Neural Networks ◽

Logistic Regression ◽

Emergency Care ◽

Model Performance ◽

Machine Learning Methods ◽

C Statistic ◽

Emergency Care System ◽

Care System

Abstract Background The primary objective of this review is to assess the accuracy of machine learning methods in their application of triaging the acuity of patients presenting in the Emergency Care System (ECS). The population are patients that have contacted the ambulance service or turned up at the Emergency Department. The index test is a machine-learning algorithm that aims to stratify the acuity of incoming patients at initial triage. This is in comparison to either an existing decision support tool, clinical opinion or in the absence of these, no comparator. The outcome of this review is the calibration, discrimination and classification statistics. Methods Only derivation studies (with or without internal validation) were included. MEDLINE, CINAHL, PubMed and the grey literature were searched on the 14th December 2019. Risk of bias was assessed using the PROBAST tool and data was extracted using the CHARMS checklist. Discrimination (C-statistic) was a commonly reported model performance measure and therefore these statistics were represented as a range within each machine learning method. The majority of studies had poorly reported outcomes and thus a narrative synthesis of results was performed. Results There was a total of 92 models (from 25 studies) included in the review. There were two main triage outcomes: hospitalisation (56 models), and critical care need (25 models). For hospitalisation, neural networks and tree-based methods both had a median C-statistic of 0.81 (IQR 0.80-0.84, 0.79-0.82). Logistic regression had a median C-statistic of 0.80 (0.74-0.83). For critical care need, neural networks had a median C-statistic of 0.89 (0.86-0.91), tree based 0.85 (0.84-0.88), and logistic regression 0.83 (0.79-0.84). Conclusions Machine-learning methods appear accurate in triaging undifferentiated patients entering the Emergency Care System. There was no clear benefit of using one technique over another; however, models derived by logistic regression were more transparent in reporting model performance. Future studies should adhere to reporting guidelines and use these at the protocol design stage. Registration and funding This systematic review is registered on the International prospective register of systematic reviews (PROSPERO) and can be accessed online at the following URL: https://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42020168696 This study was funded by the NIHR as part of a Clinical Doctoral Research Fellowship.

Download Full-text

Development of a Unified Land Model for Prediction of Surface Hydrology and Land–Atmosphere Interactions

Journal of Hydrometeorology ◽

10.1175/2011jhm1361.1 ◽

2011 ◽

Vol 12 (6) ◽

pp. 1299-1320 ◽

Cited By ~ 23

Author(s):

Ben Livneh ◽

Pedro J. Restrepo ◽

Dennis P. Lettenmaier

Keyword(s):

Soil Moisture ◽

Land Surface ◽

Climate Models ◽

A Priori ◽

Model Performance ◽

Surface Flux ◽

Surface Fluxes ◽

Frozen Soil ◽

Surface Model ◽

Soil Moisture Accounting

Abstract A unified land model (ULM) is described that combines the surface flux parameterizations in the Noah land surface model (used in most of NOAA’s coupled weather and climate models) with the Sacramento Soil Moisture Accounting model (Sac; used for hydrologic prediction within the National Weather Service). The motivation was to develop a model that has a history of strong hydrologic performance while having the ability to be run in the coupled land–atmosphere environment. ULM takes the vegetation, snow model, frozen soil, and evapotranspiration schemes from Noah and merges them with the soil moisture accounting scheme from Sac. ULM surface fluxes, soil moisture, and streamflow simulations were evaluated through comparisons with observations from the Ameriflux (surface flux), Illinois Climate Network (soil moisture), and Model Parameter Estimation Experiment (MOPEX; streamflow) datasets. Initially, a priori parameters from Sac and Noah were used, which resulted in ULM surface flux simulations that were comparable to those produced by Noah (Sac does not predict surface energy fluxes). ULM with the a priori parameters had streamflow simulation skill that was generally similar to Sac’s, although it was slightly better (worse) for wetter (more arid) basins. ULM model performance using a set of parameters identified via a Monte Carlo search procedure lead to substantial improvements relative to the a priori parameters. A scheme for transfer of parameters from streamflow simulations to nearby flux and soil moisture measurement points was also evaluated; this approach did not yield conclusive improvements relative to the a priori parameters.

Download Full-text

One stomatal model to rule them all? Evaluating competing hypotheses to regulate the exchange of carbon and water against experimental data

10.5194/egusphere-egu2020-678 ◽

2020 ◽

Author(s):

Manon Sabot ◽

Martin De Kauwe ◽

Belinda Medlyn ◽

Andy Pitman

Keyword(s):

Stomatal Conductance ◽

Land Surface ◽

Model Comparison ◽

Climate Models ◽

Global Climate ◽

Model Performance ◽

Vapour Pressure Deficit ◽

Global Climate Models ◽

Model Complexity ◽

Coupled Climate Models

Nearly 2/3 of the annual global evapotranspiration (ET) over land arises from the vegetation. Yet, coupled-climate models only attribute between 22% &#8211; 58% of the annual terrestrial ET to plants. In coupled-climate models, the exchange of carbon and water between the terrestrial biosphere and the atmosphere is simulated by land-surface models (LSMs). Within those LSMs, stomatal conductance (gs) models allow plants to regulate their transpiration and carbon uptake, but most are empirically linked to climate, soil moisture availabilty, and CO2. Therefore, how and which gs&#160;schemes are implemented within LSMs is a key source of model uncertainty. This uncertainty has led to considerable investment in theory development in the recent years; multiple alternative hypotheses of optimal leaf-level regulation of gas exchange have been proposed as solutions to reduce existing model biases. However, a systematic inter-model evaluation is lacking (i.e. inter-model comparison within a single framework is needed to understand how different mechanistic assumptions across these new gs&#160;models affect plant behaviour). Here, we asked how, and under what conditions, nine novel optimal gs&#160;models differ from one another. The models were trained to match under average conditions before being subjected to: (i) a dry-down, (ii) high vapour pressure deficit, and (iii) elevated CO2. These experiments allowed us to identify the models&#8217; specific responses and sensitivities. To further assess whether the models&#8217; responses were realistic, we tested them against photosynthetic and hydraulic field data measured along mesic-xeric gradients in Europe and Australia. Finally, we evaluated model performance versus model complexity and the amount of information taken in by each model, which enables us to make recommendations regarding the use of stomatal conductance schemes in global climate models.

Download Full-text

Deep Neural Network Driven Speech Classification for Relevance Detection in Automatic Medical Documentation

Studies in Health Technology and Informatics - Public Health and Informatics ◽

10.3233/shti210121 ◽

2021 ◽

Author(s):

Suhail Ahamed ◽

Gabriele Weiler ◽

Karl Boden ◽

Kai Januschowski ◽

Matthias Stennes ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Short Term Memory ◽

Model Performance ◽

Medical Documentation ◽

Report Generation ◽

The Neural Network ◽

Computational Capability ◽

Speech Classification

The automation of medical documentation is a highly desirable process, especially as it could avert significant temporal and monetary expenses in healthcare. With the help of complex modelling and high computational capability, Automatic Speech Recognition (ASR) and deep learning have made several promising attempts to this end. However, a factor that significantly determines the efficiency of these systems is the volume of speech that is processed in each medical examination. In the course of this study, we found that over half of the speech, recorded during follow-up examinations of patients treated with Intra-Vitreal Injections, was not relevant for medical documentation. In this paper, we evaluate the application of Convolutional and Long Short-Term Memory (LSTM) neural networks for the development of a speech classification module aimed at identifying speech relevant for medical report generation. In this regard, various topology parameters are tested and the effect of the model performance on different speaker attributes is analyzed. The results indicate that Convolutional Neural Networks (CNNs) are more successful than LSTM networks, and achieve a validation accuracy of 92.41%. Furthermore, on evaluation of the robustness of the model to gender, accent and unknown speakers, the neural network generalized satisfactorily.

Download Full-text

Applying Deep Neural Networks and Ensemble Machine Learning Methods to Forecast Airborne Ambrosia Pollen

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph16111992 ◽

2019 ◽

Vol 16 (11) ◽

pp. 1992 ◽

Cited By ~ 6

Author(s):

Gebreab K. Zewdie ◽

David J. Lary ◽

Estelle Levetin ◽

Gemechu F. Garuma

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Land Surface ◽

Deep Neural Networks ◽

Airborne Pollen ◽

Training Data ◽

Gradient Boosting ◽

Learning Approaches ◽

Ambrosia Pollen ◽

Extreme Gradient Boosting

Allergies to airborne pollen are a significant issue affecting millions of Americans. Consequently, accurately predicting the daily concentration of airborne pollen is of significant public benefit in providing timely alerts. This study presents a method for the robust estimation of the concentration of airborne Ambrosia pollen using a suite of machine learning approaches including deep learning and ensemble learners. Each of these machine learning approaches utilize data from the European Centre for Medium-Range Weather Forecasts (ECMWF) atmospheric weather and land surface reanalysis. The machine learning approaches used for developing a suite of empirical models are deep neural networks, extreme gradient boosting, random forests and Bayesian ridge regression methods for developing our predictive model. The training data included twenty-four years of daily pollen concentration measurements together with ECMWF weather and land surface reanalysis data from 1987 to 2011 is used to develop the machine learning predictive models. The last six years of the dataset from 2012 to 2017 is used to independently test the performance of the machine learning models. The correlation coefficients between the estimated and actual pollen abundance for the independent validation datasets for the deep neural networks, random forest, extreme gradient boosting and Bayesian ridge were 0.82, 0.81, 0.81 and 0.75 respectively, showing that machine learning can be used to effectively forecast the concentrations of airborne pollen.

Download Full-text