scholarly journals Groundwater Level Forecasting with Artificial Neural Networks: A Comparison of LSTM, CNN and NARX

2020 ◽  
Author(s):  
Andreas Wunsch ◽  
Tanja Liesch ◽  
Stefan Broda
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Groundwater Level ◽  
Short Term Memory ◽  
State Of The Art ◽  
Predictive Ability ◽  
Data Set ◽  
Groundwater Levels ◽  
Narx Models ◽  
Artificial Neural

Abstract. It is now well established to use shallow artificial neural networks (ANN) to obtain accurate and reliable groundwater level forecasts, which are an important tool for sustainable groundwater management. However, we observe an increasing shift from conventional shallow ANNs to state-of-the-art deep learning (DL) techniques, but a direct comparison of the performance is often lacking. Although they have already clearly proven their suitability, especially shallow recurrent networks frequently seem to be excluded from the study design despite the euphoria about new DL techniques and its successes in various disciplines. Therefore, we aim to provide an overview on the predictive ability in terms of groundwater levels of shallow conventional recurrent ANN namely nonlinear autoregressive networks with exogenous inputs (NARX), and popular state-of-the-art DL-techniques such as long short-term memory (LSTM) and convolutional neural networks (CNN). We compare both the performance on sequence-to-value (seq2val) and sequence-to-sequence (seq2seq) forecasting on a 4-year period, while using only few, widely available and easy to measure meteorological input parameters, which makes our approach widely applicable. We observe that for seq2val forecasts NARX models on average perform best, however, CNNs are much faster and only slightly worse in terms of accuracy. For seq2seq forecasts, mostly NARX outperform both DL-models and even almost reach the speed of CNNs. However, NARX are the least robust against initialization effects, which nevertheless can be handled easily using ensemble forecasting. We showed that shallow neural networks, such as NARX, should not be neglected in comparison to DL-techniques; however, LSTMs and CNNs might perform substantially better with a larger data set, where DL really can demonstrate its strengths, which is rarely available in the groundwater domain though.

scholarly journals Artificial Neural Networks in Drug Design 11: Influence of Learning Rate and Momentum Factor on the Predictive Ability

2000 ◽  
Vol 68 (1) ◽  
pp. 57-64 ◽  
Author(s):  
D. Kaiser ◽  
C. Tmej ◽  
P. Chiba ◽  
K.-J. Schaper ◽  
G. Ecker
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Predictive Power ◽  
Predictive Ability ◽  
Learning Rate ◽  
Data Set ◽  
Lead Compounds ◽  
Learning Rates ◽  
Artificial Neural ◽  
Nanomolar Range

A data set of 48 propafenone-type modulators of multidrug resistance was used to investigate the influence of learning rate and momentum factor on the predictive power of artificial neural networks of different architecture. Generally, small learning rates and medium sized momentum factors are preferred. Some of the networks showed higher cross validated Q2 values than the corresponding linear model (0.87 vs. 0.83). Screening of a 158 compound virtual library identified several new lead compounds with activities in the nanomolar range.

scholarly journals Groundwater level forecasting with artificial neural networks: a comparison of long short-term memory (LSTM), convolutional neural networks (CNNs), and non-linear autoregressive networks with exogenous input (NARX)

2021 ◽  
Vol 25 (3) ◽  
pp. 1671-1687
Author(s):  
Andreas Wunsch ◽  
Tanja Liesch ◽  
Stefan Broda
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Convolutional Neural Networks ◽  
Groundwater Level ◽  
Short Term Memory ◽  
State Of The Art ◽  
Short Term ◽  
Term Memory ◽  
Non Linear ◽  
Long Short Term Memory

Abstract. It is now well established to use shallow artificial neural networks (ANNs) to obtain accurate and reliable groundwater level forecasts, which are an important tool for sustainable groundwater management. However, we observe an increasing shift from conventional shallow ANNs to state-of-the-art deep-learning (DL) techniques, but a direct comparison of the performance is often lacking. Although they have already clearly proven their suitability, shallow recurrent networks frequently seem to be excluded from the study design due to the euphoria about new DL techniques and its successes in various disciplines. Therefore, we aim to provide an overview on the predictive ability in terms of groundwater levels of shallow conventional recurrent ANNs, namely non-linear autoregressive networks with exogenous input (NARX) and popular state-of-the-art DL techniques such as long short-term memory (LSTM) and convolutional neural networks (CNNs). We compare the performance on both sequence-to-value (seq2val) and sequence-to-sequence (seq2seq) forecasting on a 4-year period while using only few, widely available and easy to measure meteorological input parameters, which makes our approach widely applicable. Further, we also investigate the data dependency in terms of time series length of the different ANN architectures. For seq2val forecasts, NARX models on average perform best; however, CNNs are much faster and only slightly worse in terms of accuracy. For seq2seq forecasts, mostly NARX outperform both DL models and even almost reach the speed of CNNs. However, NARX are the least robust against initialization effects, which nevertheless can be handled easily using ensemble forecasting. We showed that shallow neural networks, such as NARX, should not be neglected in comparison to DL techniques especially when only small amounts of training data are available, where they can clearly outperform LSTMs and CNNs; however, LSTMs and CNNs might perform substantially better with a larger dataset, where DL really can demonstrate its strengths, which is rarely available in the groundwater domain though.

Artificial neural networks applied for solidified soils data prediction: a bibliometric and systematic review

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Vinicius Luiz Pacheco ◽  
Lucimara Bragagnolo ◽  
Antonio Thomé
Keyword(s):  
Systematic Review ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
State Of The Art ◽  
Computational Cost ◽  
Soil Improvement ◽  
The State ◽  
Data Set ◽  
Content Type ◽  
Artificial Neural

Purpose The purpose of this article is to analyze the state-of-the art in a systematic way, identifying the main research groups and their related topics. The types of studies found are fundamental for understanding the application of artificial neural networks (ANNs) in cemented soils and the potential for using the technique, as well as the feasibility of extrapolation to new geotechnical or civil and environmental engineering segments. Design/methodology/approach This work is characterized as being bibliometric and systematic research of an exploratory perspective of state-of-the-art. It also persuades the qualitative and quantitative data analysis of cemented soil improvement, biocemented or microbially induced calcite precipitation (MICP) soil improvement by prediction/modeling by ANN. This study sought to compile and study the state of the art of the topic which possibilities to have a critical view about the theme. To do so, two main databases were analyzed: Scopus and Web of Science. Systematic review techniques, as well as bibliometric indicators, were implemented. Findings This paper connected the network between the achievements of the researches and illustrated the main application of ANNs in soil improvement prediction, specifically on cemented-based soils and biocemented soils (e.g. MICP technique). Also, as a bibliometric and systematic review, this work could achieve the key points in the absence of researches involving soil-ANN, and it provided the understanding of the lack of exploratory studies to be approached in the near future. Research limitations/implications Because of the research topic the article suggested other applications of ANNs in geotechnical engineering, such as other tests not related to geomechanical resistance such as unconfined compression test test and triaxial test. Practical implications This article systematically and critically presents some interesting points in the direction of future research, such as the non-approach to the use of ANNs in biocementation processes, such as MICP. Social implications Regarding the social environment, the paper brings approaches on methods that somehow mitigate the computational use, or elements necessary for geotechnical improvement of the soil, thereby optimizing the same consequently. Originality/value Neural networks have been studied for a long time in engineering, but the current computational power has increased the implementation for several engineering applications. Besides that, soil cementation is a widespread technique and its prediction modes often require high computational strength, such parameters can be mitigated with the use of ANNs, because artificial intelligence seeks learning from the implementation of the data set, reducing computational cost and increasing accuracy.

Investigation into the production and conformation traits associated with clinical mastitis using artificial neural networks

2000 ◽  
Vol 80 (3) ◽  
pp. 415-426 ◽  
Author(s):  
X. Z. Yang ◽  
R. Lacroix ◽  
K. M. Wade
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Milk Yield ◽  
Predictive Ability ◽  
Dairy Herd ◽  
Clinical Mastitis ◽  
Neural Nets ◽  
Sensitivity Analyses ◽  
Data Set ◽  
Artificial Neural

A data set comprising milk-recording and conformation data was used to investigate the usefulness of artificial neural networks in detecting influential variables in the prediction of incidences of clinical mastitis. Specifically, these data contained test-day records from dairy herd analysis, phenotypic cow scores for conformation and genetic conformation proofs for cows and their sires. The data were analysed using the milk-recording data only, the conformation data only, and a combination of the two. Results from sensitivity analyses, performed with trained neural nets, indicated that stage of lactation, milk yield on test day, cumulative milk yield and somatic cell count were the major production factors influencing the ability to detect the occurrence of clinical mastitis. Among the conformation traits, such variables as phenotypic scores for rear-teat placement, dairy character and size, cow proof for dairy character, sire reliability for final score and sire proofs for pin-setting (desirability) and loin strength were found to have some influence on the network's predictive ability, although they were all very minor in relation to the production variables mentioned. As a group, cow genetic proofs seemed more important than either sire genetic proofs or cow phenotypic scores. Given the neural network's general abilities to determine the major factors related to the presence or absence of mastitis on a given test day, it may be appropriate to investigate the possibility of using this technology for actual prediction purposes. Key words: Artificial neural networks, clinical mastitis, milk-recording data, conformation traits, sensitivity analysis, milk production

scholarly journals Groundwater Estimation from Major Physical Hydrology Components Using Artificial Neural Networks and Deep Learning

Water ◽  
2019 ◽  
Vol 12 (1) ◽  
pp. 5 ◽  
Author(s):  
Hassan Afzaal ◽  
Aitazaz A. Farooque ◽  
Farhat Abbas ◽  
Bishnu Acharya ◽  
Travis Esau
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Deep Learning ◽  
Prince Edward Island ◽  
Short Term Memory ◽  
Dew Point ◽  
Groundwater Levels ◽  
Creek Watershed ◽  
Artificial Neural ◽  
The Baltic

Precise estimation of physical hydrology components including groundwater levels (GWLs) is a challenging task, especially in relatively non-contiguous watersheds. This study estimates GWLs with deep learning and artificial neural networks (ANNs), namely a multilayer perceptron (MLP), long short term memory (LSTM), and a convolutional neural network (CNN) with four different input variable combinations for two watersheds (Baltic River and Long Creek) in Prince Edward Island, Canada. Variables including stream level, stream flow, precipitation, relative humidity, mean temperature, evapotranspiration, heat degree days, dew point temperature, and evapotranspiration for the 2011–2017 period were used as input variables. Using a hit and trial approach and various hyperparameters, all ANNs were trained from scratched (2011–2015) and validated (2016–2017). The stream level was the major contributor to GWL fluctuation for the Baltic River and Long Creek watersheds (R2 = 50.8 and 49.1%, respectively). The MLP performed better in validation for Baltic River and Long Creek watersheds (RMSE = 0.471 and 1.15, respectively). Increased number of variables from 1 to 4 improved the RMSE for the Baltic River watershed by 11% and for the Long Creek watershed by 1.6%. The deep learning techniques introduced in this study to estimate GWL fluctuations are convenient and accurate as compared to collection of periodic dips based on the groundwater monitoring wells for groundwater inventory control and management.

scholarly journals Application of Artificial Neural Networks to Predict Beach Nourishment Volume Requirements

2021 ◽  
Vol 9 (8) ◽  
pp. 786
Author(s):  
Damjan Bujak ◽  
Tonko Bogovac ◽  
Dalibor Carević ◽  
Suzana Ilic ◽  
Goran Lončar
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Spatial Variability ◽  
Coastal Management ◽  
Wind Wave ◽  
Predictive Ability ◽  
Test Set ◽  
Adriatic Coast ◽  
Important Input ◽  
Artificial Neural

The volume of material required for the construction of new and expansion of existing beach sites is an important parameter for coastal management. This information may play a crucial role when deciding which beach sites to develop. This work examines whether artificial neural networks (ANNs) can predict the spatial variability of nourishment requirements on the Croatian coast. We use survey data of the nourishment volume requirements and gravel diameter from 2016 to 2020, fetch length, beach area and orientation derived from national maps which vary from location to location due to a complex coastal configuration on the East Adriatic coast, and wind, tide, and rainfall data from nearby meteorological/oceanographic stations to train and test ANNs. The results reported here confirm that an ANN can adequately predict the spatial variability of observed nourishment volumes (R and MSE for the test set equal 0.87 and 2.24 × 104, respectively). The contributions of different parameters to the ANN’s predictive ability were examined. Apart from the most obvious parameters like the beach length and the beach areas, the fetch length proved to be the most important input contribution to ANN’s predictive ability, followed by the beach orientation. Fetch length and beach orientation are parameters governing the wind wave height and direction and hence are proxies for forcing.

scholarly journals Opportunities and Constraints in Applying Artificial Neural Networks (ANNs) in Food Authentication. Honey—A Case Study

2021 ◽  
Vol 11 (15) ◽  
pp. 6723
Author(s):  
Ariana Raluca Hategan ◽  
Romulus Puscas ◽  
Gabriela Cristea ◽  
Adriana Dehelean ◽  
Francois Guyon ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Training Sample ◽  
Training Data ◽  
Elemental Content ◽  
Neural Structure ◽  
Food Authentication ◽  
Data Set ◽  
Artificial Neural

The present work aims to test the potential of the application of Artificial Neural Networks (ANNs) for food authentication. For this purpose, honey was chosen as the working matrix. The samples were originated from two countries: Romania (50) and France (53), having as floral origins: acacia, linden, honeydew, colza, galium verum, coriander, sunflower, thyme, raspberry, lavender and chestnut. The ANNs were built on the isotope and elemental content of the investigated honey samples. This approach conducted to the development of a prediction model for geographical recognition with an accuracy of 96%. Alongside this work, distinct models were developed and tested, with the aim of identifying the most suitable configurations for this application. In this regard, improvements have been continuously performed; the most important of them consisted in overcoming the unwanted phenomenon of over-fitting, observed for the training data set. This was achieved by identifying appropriate values for the number of iterations over the training data and for the size and number of the hidden layers and by introducing of a dropout layer in the configuration of the neural structure. As a conclusion, ANNs can be successfully applied in food authenticity control, but with a degree of caution with respect to the “over optimization” of the correct classification percentage for the training sample set, which can lead to an over-fitted model.

scholarly journals Artificial Neural Networks, Sequence-to-Sequence LSTMs, and Exogenous Variables as Analytical Tools for NO2 (Air Pollution) Forecasting: A Case Study in the Bay of Algeciras (Spain)

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1770
Author(s):  
Javier González-Enrique ◽  
Juan Jesús Ruiz-Aguilar ◽  
José Antonio Moscoso-López ◽  
Daniel Urda ◽  
Lipika Deka ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Short Term Memory ◽  
Prediction Method ◽  
Monitoring Network ◽  
Exogenous Variables ◽  
Prediction Horizon ◽  
Artificial Neural ◽  
Input Variables ◽  
Analytical Tools

This study aims to produce accurate predictions of the NO2 concentrations at a specific station of a monitoring network located in the Bay of Algeciras (Spain). Artificial neural networks (ANNs) and sequence-to-sequence long short-term memory networks (LSTMs) were used to create the forecasting models. Additionally, a new prediction method was proposed combining LSTMs using a rolling window scheme with a cross-validation procedure for time series (LSTM-CVT). Two different strategies were followed regarding the input variables: using NO2 from the station or employing NO2 and other pollutants data from any station of the network plus meteorological variables. The ANN and LSTM-CVT exogenous models used lagged datasets of different window sizes. Several feature ranking methods were used to select the top lagged variables and include them in the final exogenous datasets. Prediction horizons of t + 1, t + 4 and t + 8 were employed. The exogenous variables inclusion enhanced the model’s performance, especially for t + 4 (ρ ≈ 0.68 to ρ ≈ 0.74) and t + 8 (ρ ≈ 0.59 to ρ ≈ 0.66). The proposed LSTM-CVT method delivered promising results as the best performing models per prediction horizon employed this new methodology. Additionally, per each parameter combination, it obtained lower error values than ANNs in 85% of the cases.

Sign in / Sign up

Export Citation Format

Share Document