scholarly journals Exploring the Sensitivity of Recurrent Neural Network Models for Forecasting Land Cover Change

Land ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 282
Author(s):  
Alysha van Duynhoven ◽  
Suzana Dragićević
Keyword(s):  
Land Cover ◽  
Short Term Memory ◽  
Network Models ◽  
Optimization Techniques ◽  
Neural Network Models ◽  
Method Performance ◽  
Timeseries Analysis ◽  
Method Optimization ◽  
Long Short Term Memory ◽  
Real World Datasets

Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) architectures, have obtained successful outcomes in timeseries analysis tasks. While RNNs demonstrated favourable performance for Land Cover (LC) change analyses, few studies have explored or quantified the geospatial data characteristics required to utilize this method. Likewise, many studies utilize overall measures of accuracy rather than metrics accounting for the slow or sparse changes of LC that are typically observed. Therefore, the main objective of this study is to evaluate the performance of LSTM models for forecasting LC changes by conducting a sensitivity analysis involving hypothetical and real-world datasets. The intent of this assessment is to explore the implications of varying temporal resolutions and LC classes. Additionally, changing these input data characteristics impacts the number of timesteps and LC change rates provided to the respective models. Kappa variants are selected to explore the capacity of LSTM models for forecasting transitions or persistence of LC. Results demonstrate the adverse effects of coarser temporal resolutions and high LC class cardinality on method performance, despite method optimization techniques applied. This study suggests various characteristics of geospatial datasets that should be present before considering LSTM methods for LC change forecasting.

scholarly journals Analyzing the Effects of Temporal Resolution and Classification Confidence for Modeling Land Cover Change with Long Short-Term Memory Networks

2019 ◽  
Vol 11 (23) ◽  
pp. 2784 ◽  
Author(s):  
Alysha van Duynhoven ◽  
Suzana Dragićević
Keyword(s):  
Land Cover ◽  
Land Cover Change ◽  
Temporal Resolution ◽  
Short Term Memory ◽  
Geospatial Data ◽  
Short Term ◽  
Method Performance ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Data Layers

Land cover change (LCC) is typically characterized by infrequent changes over space and time. Data-driven methods such as deep learning (DL) approaches have proven effective in many domains for predictive and classification tasks. When applied to geospatial data, sequential DL methods such as long short-term memory (LSTM) have yielded promising results in remote sensing and GIScience studies. However, the characteristics of geospatial datasets selected for use with these methods have demonstrated important implications on method performance. The number of data layers available, the rate of LCC, and inherent errors resulting from classification procedures are expected to influence model performance. Yet, it is unknown how these can affect compatibility with the LSTM method. As such, the main objective of this study is to explore the capacity of LSTM to forecast patterns that have emerged from LCC dynamics given varying temporal resolutions, persistent land cover classes, and auxiliary data layers pertaining to classification confidence. Stacked LSTM modeling approaches are applied to 17-year MODIS land cover datasets focused on the province of British Columbia, Canada. This geospatial data is reclassified to four major land cover (LC) classes during pre-processing procedures. The evaluation considers the dataset at variable temporal resolutions to demonstrate the significance of geospatial data characteristics on LSTM method performance in several scenarios. Results indicate that LSTM can be utilized for forecasting LCC patterns when there are few limitations on temporal intervals of the datasets provided. Likewise, this study demonstrates improved performance measures when there are classes that do not change. Furthermore, providing classification confidence data as ancillary input also demonstrated improved results when the number of timesteps or temporal resolution is limited. This study contributes to future applications of DL and LSTM methods for forecasting LCC.

scholarly journals Reinforcement adaptation of an attention-based neural natural language generator for spoken dialogue systems

2019 ◽  
Vol 10 (1) ◽  
pp. 1-19
Author(s):  
Matthieu Riou ◽  
Bassam Jabaian ◽  
Stéphane Huet ◽  
Fabrice Lefèvre
Keyword(s):  
Natural Language ◽  
Short Term Memory ◽  
Network Models ◽  
Dialogue Systems ◽  
Neural Network Models ◽  
Spoken Dialogue Systems ◽  
Spoken Dialogue ◽  
Automatic Transcription ◽  
Long Short Term Memory ◽  
Expected Payoffs

Following some recent propositions to handle natural language generation in spoken dialogue systems with long short-term memory recurrent neural network models~\citep{Wen2016a} we first investigate a variant thereof with the objective of a better integration of the attention subnetwork. Then our next objective is to propose and evaluate a framework to adapt the NLG module online through direct interactions with the users. When doing so the basic way is to ask the user to utter an alternative sentence to express a particular dialogue act. But then the system has to decide between using an automatic transcription or to ask for a manual transcription. To do so a reinforcement learning approach based on an adversarial bandit scheme is retained. We show that by defining appropriately the rewards as a linear combination of expected payoffs and costs of acquiring the new data provided by the user, a system design can balance between improving the system's performance towards a better match with the user's preferences and the burden associated with it. Then the actual benefits of this system is assessed with a human evaluation, showing that the addition of more diverse utterances allows to produce sentences more satisfying for the user.

Deep Neural Models and Retrofitting for Arabic Text Categorization

2020 ◽  
Vol 16 (2) ◽  
pp. 74-86 ◽  
Author(s):  
Fatima-Zahra El-Alami ◽  
Said Ouatik El Alaoui ◽  
Noureddine En-Nahnahi
Keyword(s):  
Text Categorization ◽  
Short Term Memory ◽  
Network Models ◽  
Arabic Text ◽  
Short Term ◽  
Lexical Resources ◽  
Neural Network Models ◽  
Online Data ◽  
Long Short Term Memory ◽  
Vector Representations

Arabic text categorization is an important task in text mining particularly with the fast-increasing quantity of the Arabic online data. Deep neural network models have shown promising performance and indicated great data modeling capacities in managing large and substantial datasets. This article investigates convolution neural networks (CNNs), long short-term memory (LSTM) and their combination for Arabic text categorization. This work additionally handles the morphological variety of Arabic words by exploring the word embeddings model using position weights and subword information. To guarantee the nearest vector representations for connected words, this article adopts a strategy for refining Arabic vector space representations using semantic information embedded in lexical resources. Several experiments utilizing different architectures have been conducted on the OSAC dataset. The obtained results show the effectiveness of CNN-LSTM without and with retrofitting for Arabic text categorization in comparison with major competing methods.

scholarly journals Comparing Neural Network Models for Photovoltaic Power Generation Prediction

2021 ◽  
Author(s):  
Carlos Henrique Torres Andrade ◽  
Tiago Figueiredo Vieira ◽  
Ícaro Bezzera Queiroz Araújo ◽  
Gustavo Costa Gomes Melo ◽  
Erick de Andrade Barboza ◽  
...  
Keyword(s):  
Alternative Energy ◽  
Fossil Fuels ◽  
Short Term Memory ◽  
Network Models ◽  
Optimization Approach ◽  
Neural Network Models ◽  
Alternative Energy Sources ◽  
Comparison Results ◽  
Long Short Term Memory ◽  
The City

Research on alternative energy sources has been increasing for the past years due to environmental concerns and the depletion of fossil fuels. Since photovoltaic generation is intermittent, one needs to predict solar incidence to alleviate problems due to demand surges in conventional distribution systems.Many works have used Long Short-Term Memory (LSTMs) to predict generation. However, to minimize computational costs related to retraining and inference, LSTMs might not be optimal. Therefore, in this work, we compare the performance of MLP (Multilayer Perceptron), Recurrent Neural Networks (RNNs), and LSTMs for the task mentioned above. We used the solar radiance measured throughout 2020 in the city of Maceió (Brazil), taking into account periods of 2 hours for training to predict the next 5-minutes. Hyperparameters were fine-tuned using an optimization approach based on Bayesian inference to promote a fair comparison. Results showed that the MLP has satisfactory performance, requiring much less time to train and forecast. Such results can contribute, for example, to improving response time in embedded systems.

scholarly journals Residual Life Prediction for Induction Furnace by Sequential Encoder with s-Convolutional LSTM

Processes ◽  
2021 ◽  
Vol 9 (7) ◽  
pp. 1121
Author(s):  
Yulim Choi ◽  
Hyeonho Kwun ◽  
Dohee Kim ◽  
Eunju Lee ◽  
Hyerim Bae
Keyword(s):  
Life Prediction ◽  
Induction Furnace ◽  
Short Term Memory ◽  
Residual Life ◽  
Network Models ◽  
Sensor Data ◽  
Short Term ◽  
Neural Network Models ◽  
Long Short Term Memory ◽  
Convolutional Lstm

Induction furnaces are widely used for melting scrapped steel in small foundries and their use has recently become more frequent. The maintenance of induction furnaces is usually based on empirical decisions of the operator and an explosion can occur through operator error. To prevent an explosion, previous studies have utilized statistical models but have been unable to generalize the problem and have achieved a low accuracy. Herein, we propose a data-driven method for induction furnaces by proposing a novel 2D matrix called a sequential feature matrix(s-encoder) and multi-channel convolutional long short-term memory (s-ConLSTM). First, the sensor data and operation data are converted into sequential feature matrices. Then, N-sequential feature matrices are imported into the convolutional LSTM model to predict the residual life of the induction furnace wall. Based on our experimental results, our method outperforms general neural network models and enhances the safe use of induction furnaces.

scholarly journals Prediction Power of Logistic Regression (LR) and Multi-Layer Perceptron (MLP) Models in Exploring Driving Forces of Urban Expansion to Be Sustainable in Estonia

2021 ◽  
Vol 14 (1) ◽  
pp. 160
Author(s):  
Najmeh Mozaffaree Pour ◽  
Tõnu Oja
Keyword(s):  
Land Cover ◽  
Driving Forces ◽  
Urban Expansion ◽  
Network Models ◽  
Influential Factors ◽  
Residential Areas ◽  
Neural Network Models ◽  
Independent Variables ◽  
The Core ◽  
Mlp Model

Estonia mainly experienced urban expansion after regaining independence in 1991. Employing the CORINE Land Cover dataset to analyze the dynamic changes in land use/land cover (LULC) in Estonia over 28 years revealed that urban land increased by 33.96% in Harju County and by 19.50% in Tartu County. Therefore, after three decades of LULC changes, the large number of shifts from agricultural and forest land to urban ones in an unplanned manner have become of great concern. To this end, understanding how LULC change contributes to urban expansion will provide helpful information for policy-making in LULC and help make better decisions for future transitions in urban expansion orientation and plan for more sustainable cities. Many different factors govern urban expansion; however, physical and proximity factors play a significant role in explaining the spatial complexity of this phenomenon in Estonia. In this research, it was claimed that urban expansion was affected by the 12 proximity driving forces. In this regard, we applied LR and MLP neural network models to investigate the prediction power of these models and find the influential factors driving urban expansion in two Estonian counties. Using LR determined that the independent variables “distance from main roads (X7)”, “distance from the core of main cities of Tallinn and Tartu land (X2)”, and “distance from water land (X11)” had a higher negative correlation with urban expansion in both counties. Indeed, this investigation requires thinking towards constructing a balance between urban expansion and its driving forces in the long term in the way of sustainability. Using the MLP model determined that the “distance from existing residential areas (X10)” in Harju County and the “distance from the core of Tartu (X2)” in Tartu County were the most influential driving forces. The LR model showed the prediction power of these variables to be 37% for Harju County and 45% for Tartu County. In comparison, the MLP model predicted nearly 80% of variability by independent variables for Harju County and approximately 50% for Tartu County, expressing the greater power of independent variables. Therefore, applying these two models helped us better understand the causative nature of urban expansion in Harju County and Tartu County in Estonia, which requires more spatial planning regulation to ensure sustainability.

The Impact of Land Cover Data on Rainfall-Runoff Prediction Using an Entity-Aware-LSTM

2021 ◽  
Author(s):  
Sebastian Drost ◽  
Fabian Netzel ◽  
Andreas Wytzisk-Ahrens ◽  
Christoph Mudersbach
Keyword(s):  
Land Cover ◽  
Short Term Memory ◽  
Land Cover Changes ◽  
Rainfall Runoff ◽  
Corine Land Cover ◽  
Land Cover Data ◽  
Modelling Problem ◽  
Long Short Term Memory ◽  
The Impact ◽  
Process Based Models

<p>The application of Deep Learning methods for modelling rainfall-runoff have reached great advances in the last years. Especially, long short-term memory (LSTM) networks have gained enhanced attention for time-series prediction. The architecture of this special kind of recurrent neural network is optimized for learning long-term dependencies from large time-series datasets. Thus, different studies proved the applicability of LSTM networks for rainfall-runoff predictions and showed, that they are capable of outperforming other types of neural networks (Hu et al., 2018).</p><p>Understanding the impact of land-cover changes on rainfall-runoff dynamics is an important task. Such a hydrological modelling problem typically is solved with process-based models by varying model-parameters related to land-cover-incidents at different points in time. Kratzert et al. (2019) proposed an adaption of the standard LSTM architecture, called Entity-Aware-LSTM (EA-LSTM), which can take static catchment attributes as input features to overcome the regional modelling problem and provides a promising approach for similar use cases. Hence, our contribution aims to analyse the suitability of EA-LSTM for assessing the effect of land-cover changes.</p><p>In different experimental setups, we train standard LSTM and EA-LSTM networks for multiple small subbasins, that are associated to the Wupper region in Germany. Gridded daily precipitation data from the REGNIE dataset (Rauthe et al., 2013), provided by the German Weather Service (DWD), is used as model input to predict the daily discharge for each subbasin. For training the EA-LSTM we use land cover information from the European CORINE Land Cover (CLC) inventory as static input features. The CLC inventory includes Europe-wide timeseries of land cover in 44 classes as well as land cover changes for different time periods (Büttner, 2014). The percentage proportion of each land cover class within a subbasin serves as static input features. To evaluate the impact of land cover data on rainfall-runoff prediction, we compare the results of the EA-LSTM with those of the standard LSTM considering different statistical measures as well as the Nash–Sutcliffe efficiency (NSE).</p><p>In addition, we test the ability of the EA-LSTM to outperform physical process-based models. For this purpose, we utilize existing and calibrated hydrological models within the Wupper basin to simulate discharge for each subbasin. Finally, performance metrics of the calibrated model are used as benchmarks for assessing the performance of the EA-LSTM model.</p><p><strong>References</strong></p><p>Büttner, G. (2014). CORINE Land Cover and Land Cover Change Products. In: Manakos & M. Braun (Hrsg.), Land Use and Land Cover Mapping in Europe (Bd. 18, S. 55–74). Springer Netherlands. https://doi.org/10.1007/978-94-007-7969-3_5</p><p>Hu, C., Wu, Q., Li, H., Jian, S., Li, N., & Lou, Z. (2018). Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation. Water, 10(11), 1543. https://doi.org/10.3390/w10111543</p><p>Kratzert, F., Klotz, D., Shalev, G., Klambauer, G., Hochreiter, S., & Nearing, G. (2019). Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets. Hydrology and Earth System Sciences, 23(12), 5089–5110. https://doi.org/10.5194/hess-23-5089-2019</p><p>Rauthe, M, Steiner, H, Riediger, U, Mazurkiewicz, A &Gratzki, A (2013): A Central European precipitation climatology – Part I: Generation and validation of a high-resolution gridded daily data set (HYRAS), Meteorologische Zeitschrift, Vol 22, No 3, 235–256. https://doi.org/10.1127/0941-2948/2013/0436</p>

scholarly journals A Review on Recent Advancements in FOREX Currency Prediction

Algorithms ◽  
2020 ◽  
Vol 13 (8) ◽  
pp. 186 ◽  
Author(s):  
Md. Saiful Islam ◽  
Emam Hossain ◽  
Abdur Rahman ◽  
Mohammad Shahadat Hossain ◽  
Karl Andersson
Keyword(s):  
Selection Criteria ◽  
Short Term Memory ◽  
Time Series Prediction ◽  
Optimization Techniques ◽  
Selection Algorithm ◽  
Relevant Research ◽  
Short Term ◽  
Different Types ◽  
Long Short Term Memory ◽  
Gated Recurrent Unit

In recent years, the foreign exchange (FOREX) market has attracted quite a lot of scrutiny from researchers all over the world. Due to its vulnerable characteristics, different types of research have been conducted to accomplish the task of predicting future FOREX currency prices accurately. In this research, we present a comprehensive review of the recent advancements of FOREX currency prediction approaches. Besides, we provide some information about the FOREX market and cryptocurrency market. We wanted to analyze the most recent works in this field and therefore considered only those papers which were published from 2017 to 2019. We used a keyword-based searching technique to filter out popular and relevant research. Moreover, we have applied a selection algorithm to determine which papers to include in this review. Based on our selection criteria, we have reviewed 39 research articles that were published on “Elsevier”, “Springer”, and “IEEE Xplore” that predicted future FOREX prices within the stipulated time. Our research shows that in recent years, researchers have been interested mostly in neural networks models, pattern-based approaches, and optimization techniques. Our review also shows that many deep learning algorithms, such as gated recurrent unit (GRU) and long short term memory (LSTM), have been fully explored and show huge potential in time series prediction.

Sign in / Sign up

Export Citation Format

Share Document