Predicting energy and carbon fluxes using LSTM networks

Global land-atmosphere energy and carbon fluxes are key drivers of the Earth&#8217;s climate system. Their assessment over a wide range of climates and biomes is therefore essential (i) for a better understanding and characterization of land-atmosphere exchanges and feedbacks and (ii) for examining the effect of climate change on the global water, energy and carbon cycles.&#160;Large-sample datasets such as the FLUXNET2015 dataset (Pastorello et al., 2020) foster the use of machine learning (ML) techniques as a powerful addition to existing physically-based modelling approaches. Several studies have investigated ML techniques for assessing energy and carbon fluxes, and while across-site variability and the mean seasonal cycle are typically well predicted, deviations from mean seasonal behaviour remains challenging (Tramontana et al., 2016).&#160;In this study we examine the importance of memory effects for predicting energy and carbon fluxes at half-hourly and daily temporal resolutions. To this end, we train a Long Short-Term Memory (LSTM, Hochreiter and Schmidthuber, 1997), a recurrent neural network with explicit memory, that is particularly suited for time series predictions due to its capability to store information over longer (time) sequences. We train the LSTM on a large number of FLUXNET sites part of the FLUXNET2015 dataset using local meteorological forcings and static site attributes derived from remote sensing and reanalysis data.&#160;We evaluate model performance out-of-sample (leaving out individual sites) in a 10-fold cross-validation. Additionally, we compare results from the LSTM with results from another ML technique, XGBoost (Chen and Guestrin, 2016), that does not contain system memory. By analysing the differences in model performances of both approaches across various biomes, we investigate under which conditions the inclusion of memory might be beneficial for modelling energy and carbon fluxes.&#160;References:Chen, Tianqi, and Carlos Guestrin. "Xgboost: A scalable tree boosting system." Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 2016.Hochreiter, Sepp, and J&#252;rgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780.Pastorello, Gilberto, et al. "The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data." Scientific data 7.1 (2020): 1-27Tramontana, Gianluca, et al. "Predicting carbon dioxide and energy fluxes across global &#160; FLUXNET sites with regression algorithms." Biogeosciences 13.14 (2016): 4291-4313.

Download Full-text

Knowledge transfer from simulation to reality via Long Short-Term Memory networks: Estimating groundwater table depth anomalies over Europe

10.5194/egusphere-egu21-590 ◽

2021 ◽

Author(s):

Yueling Ma ◽

Carsten Montzka ◽

Bagher Bayat ◽

Stefan Kollet

Keyword(s):

Short Term Memory ◽

Scientific Data ◽

Groundwater Table ◽

Short Term ◽

Term Memory ◽

Continental Scale ◽

Groundwater Table Depth ◽

Long Short Term Memory ◽

Simulation Results

Near real-time groundwater table depth measurements are scarce over Europe, leading to challenges in monitoring groundwater resources at the continental scale. In this study, we leveraged knowledge learned from simulation results by Long Short-Term Memory (LSTM) networks to estimate monthly groundwater table depth anomaly (wtda) data over Europe. The LSTM networks were trained, validated, and tested at individual pixels on anomaly data derived from daily integrated hydrologic simulation results over Europe from 1996 to 2016, with a spatial resolution of 0.11&#176; (Furusho-Percot et al., 2019), to predict monthly wtda based on monthly precipitation anomalies (pra) and soil moisture anomalies (&#952;a). Without additional training, we directly fed the networks with averaged monthly pra and &#952;a data from 1996 to 2016 obtained from commonly available observational datasets and reanalysis products, and compared the network outputs with available borehole in situ measured wtda. The LSTM network estimates show good agreement with the in situ observations, resulting in Pearson correlation coefficients of regional averaged wtda data in seven PRUDENCE regions ranging from 42% to 76%, which are ~ 10% higher than the original simulation results except for the Iberian Peninsula. Our study demonstrates the potential of LSTM networks to transfer knowledge from simulation to reality for the estimation of wtda over Europe. The proposed method can be used to provide spatiotemporally continuous information at large spatial scales in case of sparse ground-based observations, which is common for groundwater table depth measurements. Moreover, the results highlight the advantage of combining physically-based models with machine learning techniques in data processing.&#160;Reference:Furusho-Percot, C., Goergen, K., Hartick, C., Kulkarni, K., Keune, J. and Kollet, S. (2019). Pan-European groundwater to atmosphere terrestrial systems climatology from a physically consistent simulation. Scientific Data, 6(1).

Download Full-text

Modeling of groundwater table depth anomalies using Long Short-Term Memory networks over Europe

10.5194/egusphere-egu2020-5367 ◽

2020 ◽

Author(s):

Yueling Ma ◽

Carsten Montzka ◽

Bagher Bayat ◽

Stefan Kollet

Keyword(s):

Real Time ◽

Short Term Memory ◽

Groundwater Resources ◽

Scientific Data ◽

Great Promise ◽

Monthly Precipitation ◽

Short Term ◽

Term Memory ◽

Continental Scale ◽

Long Short Term Memory

Groundwater is the dominant source of fresh water in many European countries. However, due to a lack of near-real-time water table depth (wtd) observations, monitoring of groundwater resources is not feasible at the continental scale. Thus, an alternative approach is required to produce wtd data from other available observations near-real-time. In this study, we propose Long Short-Term Memory (LSTM) networks to model monthly wtd anomalies over Europe utilizing monthly precipitation anomalies as input. LSTM networks are a special type of artificial neural networks, showing great promise in exploiting long-term dependencies between time series, which is expected in the response of groundwater to precipitation. To establish the methodology, spatially and temporally continuous data from terrestrial simulations at the continental scale were applied with a spatial resolution of 0.11&#176;, ranging from the year 1996 to 2016 (Furusho-Percot et al., 2019). They were divided into a training set (1996 &#8211; 2012), a validation set (2012 &#8211; 2014) and a testing set (2015 -2016) to construct local models on selected pixels over eight PRUDENCE regions. The outputs of the LSTM networks showed good agreement with the simulation results in locations with a shallow wtd (~3m). It is important to note, the quality of the models was strongly affected by the amount of snow cover. Moreover, with the introduction of monthly evapotranspiration anomalies as additional input, pronounced improvements of the network performances were only obtained in more arid regions (i.e., Iberian Peninsula and Mediterranean). Our results demonstrate the potential of LSTM networks to produce high-quality wtd anomalies from hydrometeorological variables that are monitored at the large scale and part of operational forecasting systems potentially facilitating the implementation of an efficient groundwater monitoring system over Europe.Reference:Furusho-Percot, C., Goergen, K., Hartick, C., Kulkarni, K., Keune, J. and Kollet, S. (2019). Pan-European groundwater to atmosphere terrestrial systems climatology from a physically consistent simulation. Scientific Data, 6(1).

Download Full-text

Research on Safety Prediction of Sector Traffic Operation Based on a Long Short Term Memory Model

Applied Sciences ◽

10.3390/app11115141 ◽

2021 ◽

Vol 11 (11) ◽

pp. 5141

Author(s):

Wenying Lyu ◽

Honghai Zhang ◽

Junqiang Wan ◽

Lei Yang

Keyword(s):

Time Series ◽

Traffic Safety ◽

Prediction Models ◽

Short Term Memory ◽

Model Performance ◽

Risk Identification ◽

Civil Aviation ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Traffic safety has been thought of as a basic feature of transportation, recent developments in civil aviation have emphasized the need for risk identification and safety prediction. This study aims to increase en-route flight safety through the development of prediction models for flight conflicts. Firstly, flight conflicts time series and traffic parameters are extracted from historical ADS-B data. In the second step, a Long Short-Term Memory (LSTM) model is trained to make a one-step-ahead prediction on the flight conflict time series. The results show that the LSTM model has the greatest prediction effect (MAE 0.3901) with comparison to other models. Based on that, we add traffic parameters (volume, density, velocity) into the LSTM model as new input variables and issue a comprehensive analysis of the relative predictive power of traffic parameters. The accuracy of prediction model is validated with a mean error of less than 3%. Based on the improvements of model performance brought by traffic parameters, LSTM models with a single traffic parameter are proposed for further discussion. The results illustrate that volume is the most important factor in promoting prediction accuracy and density has an advantage of improvement in the aspect of model stability.

Download Full-text

Prediction of gait trajectories based on the Long Short Term Memory neural networks

PLoS ONE ◽

10.1371/journal.pone.0255597 ◽

2021 ◽

Vol 16 (8) ◽

pp. e0255597

Author(s):

Abdelrahman Zaroug ◽

Alessandro Garofolini ◽

Daniel T. H. Lai ◽

Kurt Mudie ◽

Rezaul Begg

Keyword(s):

Lower Limb ◽

Short Term Memory ◽

Series Data ◽

Short Term ◽

Term Memory ◽

Wide Range ◽

Limb Kinematics ◽

Long Short Term Memory ◽

Balance Loss ◽

Lower Limb Kinematics

The forecasting of lower limb trajectories can improve the operation of assistive devices and minimise the risk of tripping and balance loss. The aim of this work was to examine four Long Short Term Memory (LSTM) neural network architectures (Vanilla, Stacked, Bidirectional and Autoencoders) in predicting the future trajectories of lower limb kinematics, i.e. Angular Velocity (AV) and Linear Acceleration (LA). Kinematics data of foot, shank and thigh (LA and AV) were collected from 13 male and 3 female participants (28 ± 4 years old, 1.72 ± 0.07 m in height, 66 ± 10 kg in mass) who walked for 10 minutes at preferred walking speed (4.34 ± 0.43 km.h-1) and at an imposed speed (5km.h-1, 15.4% ± 7.6% faster) on a 0% gradient treadmill. The sliding window technique was adopted for training and testing the LSTM models with total kinematics time-series data of 10,500 strides. Results based on leave-one-out cross validation, suggested that the LSTM autoencoders is the top predictor of the lower limb kinematics trajectories (i.e. up to 0.1s). The normalised mean squared error was evaluated on trajectory predictions at each time-step and it obtained 2.82–5.31% for the LSTM autoencoders. The ability to predict future lower limb motions may have a wide range of applications including the design and control of bionics allowing improved human-machine interface and mitigating the risk of falls and balance loss.

Download Full-text

Bottleneck Based Gridlock Prediction in an Urban Road Network Using Long Short-Term Memory

Electronics ◽

10.3390/electronics9091412 ◽

2020 ◽

Vol 9 (9) ◽

pp. 1412

Author(s):

Ei Ei Mon ◽

Hideya Ochiai ◽

Chaiyachet Saivichit ◽

Chaodit Aswakul

Keyword(s):

Road Network ◽

Short Term Memory ◽

Road Networks ◽

Sample Sizes ◽

Short Term ◽

Urban Road Network ◽

Term Memory ◽

Urban Road ◽

Wide Range ◽

Long Short Term Memory

The traffic bottlenecks in urban road networks are more challenging to investigate and discover than in freeways or simple arterial networks. A bottleneck indicates the congestion evolution and queue formation, which consequently disturb travel delay and degrade the urban traffic environment and safety. For urban road networks, sensors are needed to cover a wide range of areas, especially for bottleneck and gridlock analysis, requiring high installation and maintenance costs. The emerging widespread availability of GPS vehicles significantly helps to overcome the geographic coverage and spacing limitations of traditional fixed-location detector data. Therefore, this study investigated GPS vehicles that have passed through the links in the simulated gridlock-looped intersection area. The sample size estimation is fundamental to any traffic engineering analysis. Therefore, this study tried a different number of sample sizes to analyze the severe congestion state of gridlock. Traffic condition prediction is one of the primary components of intelligent transportation systems. In this study, the Long Short-Term Memory (LSTM) neural network was applied to predict gridlock based on bottleneck states of intersections in the simulated urban road network. This study chose to work on the Chula-Sathorn SUMO Simulator (Chula-SSS) dataset. It was calibrated with the past actual traffic data collection by using the Simulation of Urban MObility (SUMO) software. The experiments show that LSTM provides satisfactory results for gridlock prediction with temporal dependencies. The reported prediction error is based on long-range time dependencies on the respective sample sizes using the calibrated Chula-SSS dataset. On the other hand, the low sampling rate of GPS trajectories gives high RMSE and MAE error, but with reduced computation time. Analyzing the percentage of simulated GPS data with different random seed numbers suggests the possibility of gridlock identification and reports satisfying prediction errors.

Download Full-text

Time Series Segmentation Based on Stationarity Analysis to Improve New Samples Prediction

Sensors ◽

10.3390/s21217333 ◽

2021 ◽

Vol 21 (21) ◽

pp. 7333

Author(s):

Ricardo Petri Silva ◽

Bruno Bogaz Zarpelão ◽

Alberto Cano ◽

Sylvio Barbon Junior

Keyword(s):

Time Series ◽

Short Term Memory ◽

Machine Learning Algorithms ◽

Detection Methods ◽

Sequential Data ◽

Short Term ◽

Term Memory ◽

Wide Range ◽

Long Short Term Memory ◽

Adf Test

A wide range of applications based on sequential data, named time series, have become increasingly popular in recent years, mainly those based on the Internet of Things (IoT). Several different machine learning algorithms exploit the patterns extracted from sequential data to support multiple tasks. However, this data can suffer from unreliable readings that can lead to low accuracy models due to the low-quality training sets available. Detecting the change point between high representative segments is an important ally to find and thread biased subsequences. By constructing a framework based on the Augmented Dickey-Fuller (ADF) test for data stationarity, two proposals to automatically segment subsequences in a time series were developed. The former proposal, called Change Detector segmentation, relies on change detection methods of data stream mining. The latter, called ADF-based segmentation, is constructed on a new change detector derived from the ADF test only. Experiments over real-file IoT databases and benchmarks showed the improvement provided by our proposals for prediction tasks with traditional Autoregressive integrated moving average (ARIMA) and Deep Learning (Long short-term memory and Temporal Convolutional Networks) methods. Results obtained by the Long short-term memory predictive model reduced the relative prediction error from 1 to 0.67, compared to time series without segmentation.

Download Full-text

Attention-Based BiLSTM for Negation Handling in Sentimen Analysis

IJCCS (Indonesian Journal of Computing and Cybernetics Systems) ◽

10.22146/ijccs.60733 ◽

2020 ◽

Vol 14 (4) ◽

pp. 397

Author(s):

Riszki Wijayatun Pratiwi ◽

Yunita Sari ◽

Yohanes Suyanto

Keyword(s):

Feature Extraction ◽

Sentiment Analysis ◽

Extraction Method ◽

Short Term Memory ◽

Model Performance ◽

Short Term ◽

Term Memory ◽

Feature Extraction Method ◽

Long Short Term Memory

Research on sentiment analysis in recent years has increased. However, in sentiment analysis research there are still few ideas about the handling of negation, one of which is in the Indonesian sentence. This results in sentences that contain elements of the word negation have not found the exact polarity.The purpose of this research is to analyze the effect of the negation word in Indonesian. Based on positive, neutral and negative classes, using attention-based Long Short Term Memory and word2vec feature extraction method with continuous bag-of-word (CBOW) architecture. The dataset used is data from Twitter. Model performance is seen in the accuracy value.The use of word2vec with CBOW architecture and the addition of layer attention to the Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BiLSTM) methods obtained an accuracy of 78.16% and for BiLSTM resulted in an accuracy of 79.68%. whereas in the FSW algorithm is 73.50% and FWL 73.79%. It can be concluded that attention based BiLSTM has the highest accuracy, but the addition of layer attention in the Long Short Term Memory method is not too significant for negation handling. because the addition of the attention layer cannot determine the words that you want to pay attention to.

Download Full-text

Comparison of Long Short-Term Memory and Weighted Regressions on Time, Discharge, and Season Models for Nitrate-N Load Estimation

Sustainability ◽

10.3390/su12155942 ◽

2020 ◽

Vol 12 (15) ◽

pp. 5942

Author(s):

Kichul Jung ◽

Myoung-Jin Um ◽

Momcilo Markus ◽

Daeryong Park

Keyword(s):

Water Quality ◽

Short Term Memory ◽

Water Quality Management ◽

Model Performance ◽

The United States ◽

Percentage Error ◽

Short Term ◽

Term Memory ◽

Nitrate N ◽

Long Short Term Memory

The long short-term memory (LSTM) model has been widely used for a broad range of applications entailing the estimation of variables in different fields to improve water quality management in rivers. The main objectives of this study are (1) to develop a novel LSTM-based model for the estimation of nitrate-N loads, which adversely affect water resources, and (2) to evaluate the performance of the model by comparing it with that of Monte Carlo sub-sampling and the weighted regressions on time discharge and season (WRTDS) model. We evaluated the model performance using various numbers of hidden layers, ranging from one to four, in the LSTM model to determine the appropriate number of hidden layers; furthermore, we applied the sampling frequencies of 6, 12, and 24 to assess their impact. Seven polluted river basins in the United States were used for analysis, and the relative root mean squared error (rRMSE) and the mean percentage error (MPE) metrics were applied for the validation of the model estimates. The proposed model achieved accurate nitrate-N load estimates using three to four hidden layers, and improved model performance was observed when the sampling frequency was increased. The differences among the results obtained using the LSTM model were examined based on a binning technique via a log-log plot of nitrate-N concentration against discharge. The binning analysis showed that the slope obtained from the average rates of discharge and low discharge values apparently influenced the estimates. Furthermore, box plot analyses of the statistical indices such as rRMSE and MPE demonstrate that the LSTM model seems to exhibit better performance than the WRTDS model. The results of the examination demonstrate that the LSTM model may be a good alternative with regard to estimating nitrate-N loads for the control of water quality constituents.

Download Full-text

Improving Fine-Grained Opinion Mining Approach with a Deep Constituency Tree-Long Short Term Memory Network and Word Embedding

Recent Advances in Computer Science and Communications ◽

10.2174/2666255813999200922142212 ◽

2020 ◽

Vol 13 ◽

Author(s):

Dalila Bouras ◽

Mohamed Amroune ◽

Hakim Bendjenna ◽

Issam Bendib

Keyword(s):

Opinion Mining ◽

Short Term Memory ◽

State Of The Art ◽

Syntactic Structure ◽

Product Reviews ◽

Short Term ◽

Term Memory ◽

Fine Grained ◽

Wide Range ◽

Long Short Term Memory

Objective: One key task of fine-grained opinion mining on product review is to extract product aspects and their corresponding opinion expressed by users. Previous work has demonstrated that precise modeling of opinion targets within the surrounding context can improve performances. However, how to effectively and efficiently learn hidden word semantics and better represent targets and the context still needs to be further studied. Recent years have seen a revival of the long short-term memory (LSTM), with its effectiveness being demonstrated on a wide range of problems. However, LSTM based approaches are still limited to linear data processing since it processes the information sequentially. As a result, they may perform poorly on user-generated texts, such as product reviews, tweets, etc., whose syntactic structure is not precise.To tackle this challenge, Methods: In this research paper, we propose a constituency tree long short term memory neural network-based approach. We compare our model with state-of-the-art baselines on SemEval 2014 datasets. Results: Experiment results show that our models obtain competitive performances compared to various supervised LSTM architectures. Conclusion: Our work contributes to the improvement of state-of-the-art aspect-level opinion mining methods and offers a new approach to support human decision-making process based on opinion mining results.

Download Full-text

A Particulate Matter Concentration Prediction Model Based on Long Short-Term Memory and an Artificial Neural Network

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18136801 ◽

2021 ◽

Vol 18 (13) ◽

pp. 6801

Author(s):

Junbeom Park ◽

Seongju Chang

Keyword(s):

Neural Network ◽

Artificial Neural Network ◽

Particulate Matter ◽

Industrial Development ◽

Short Term Memory ◽

Model Performance ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Artificial Neural

Many countries are concerned about high particulate matter (PM) concentrations caused by rapid industrial development, which can harm both human health and the environment. To manage PM, the prediction of PM concentrations based on historical data is actively being conducted. Existing technologies for predicting PM mostly assess the model performance for the prediction of existing PM concentrations; however, PM must be forecast in advance, before it becomes highly concentrated and causes damage to the citizens living in the affected regions. Thus, it is necessary to conduct research on an index that can illustrate whether the PM concentration will increase or decrease. We developed a model that can predict whether the PM concentration might increase or decrease after a certain time, specifically for PM2.5 (fine PM) generated by anthropogenic volatile organic compounds. An algorithm that can select a model on an hourly basis, based on the long short-term memory (LSTM) and artificial neural network (ANN) models, was developed. The proposed algorithm exhibited a higher F1-score than the LSTM, ANN, or random forest models alone. The model developed in this study could be used to predict future regional PM concentration levels more effectively.

Download Full-text