Deep learning approaches for improving prediction of daily stream temperature in data-scarce, unmonitored, and dammed basins

Basin-centric long short-term memory (LSTM) network models have recently been shown to be an exceptionally powerful tool for simulating stream temperature (Ts, temperature measured in rivers), among other hydrological variables. However, spatial extrapolation is a well-known challenge to modeling Ts and it is uncertain how an LSTM-based daily Ts model will perform in unmonitored or dammed basins. Here we compiled a new benchmark dataset consisting of >400 basins for across the contiguous United States in different data availability groups (DAG, meaning the daily sampling frequency) with or without major dams and study how to assemble suitable training datasets for predictions in monitored or unmonitored situations. For temporal generalization, CONUS-median best root-mean-square error (RMSE) values for sites with extensive (99%), intermediate (60%), scarce (10%) and absent (0%, unmonitored) data for training were 0.75, 0.83, 0.88, and 1.59°C, representing the state of the art. For prediction in unmonitored basins (PUB), LSTM’s results surpassed those reported in the literature. Even for unmonitored basins with major reservoirs, we obtained a median RMSE of 1.492°C and an R2 of 0.966. The most suitable training set was the matching DAG that the basin could be grouped into, e.g., the 60% DAG for a basin with 61% data availability. However, for PUB, a training dataset including all basins with data is preferred. An input-selection ensemble moderately mitigated attribute overfitting. Our results suggest there are influential latent processes not sufficiently described by the inputs (e.g., geology, wetland covers), but temporal fluctuations are well predictable, and LSTM appears to be the more accurate Ts modeling tool when sufficient training data are available.

Download Full-text

Tourism Demand Forecasting Based on an LSTM Network and Its Variants

Algorithms ◽

10.3390/a14080243 ◽

2021 ◽

Vol 14 (8) ◽

pp. 243

Author(s):

Shun-Chieh Hsieh

Keyword(s):

Adaptive Learning ◽

Time Series Data ◽

Short Term Memory ◽

Demand Forecasting ◽

Series Data ◽

Learning Approaches ◽

Tourism Demand ◽

Long Short Term Memory ◽

Lstm Network ◽

Gated Recurrent Unit

The need for accurate tourism demand forecasting is widely recognized. The unreliability of traditional methods makes tourism demand forecasting still challenging. Using deep learning approaches, this study aims to adapt Long Short-Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM), and Gated Recurrent Unit networks (GRU), which are straightforward and efficient, to improve Taiwan’s tourism demand forecasting. The networks are able to seize the dependence of visitor arrival time series data. The Adam optimization algorithm with adaptive learning rate is used to optimize the basic setup of the models. The results show that the proposed models outperform previous studies undertaken during the Severe Acute Respiratory Syndrome (SARS) events of 2002–2003. This article also examines the effects of the current COVID-19 outbreak to tourist arrivals to Taiwan. The results show that the use of the LSTM network and its variants can perform satisfactorily for tourism demand forecasting.

Download Full-text

Application of CNN-LSTM in Gradual Changing Fault Diagnosis of Rod Pumping System

Mathematical Problems in Engineering ◽

10.1155/2019/4203821 ◽

2019 ◽

Vol 2019 ◽

pp. 1-9 ◽

Cited By ~ 2

Author(s):

Yanfeng He ◽

Yali Liu ◽

Shuai Shao ◽

Xuhang Zhao ◽

Guojun Liu ◽

...

Keyword(s):

Mathematical Model ◽

Short Term Memory ◽

Fault Classification ◽

Training Dataset ◽

Irreversible Damage ◽

Indicator Diagram ◽

Well Production ◽

Pumping System ◽

Lstm Network ◽

Model Diagnosis

Owing to the importance of rod pumping system fault detection using an indicator diagram, indicator diagram identification has been a challenging task in the computer-vision field. The gradual changing fault is a special type of fault because it is not clearly indicated in the indicator diagram at the onset of its occurrence and can only be identified when an irreversible damage in the well has been caused. In this paper, we proposed a new method that combines the convolutional neural network (CNN) and long short-term memory (LSTM) network to perform a gradual changing fault classification. In particular, we employed CNN to extract the indicator diagram multilevel abstraction features based on its hierarchical structure. We considered the change in the time series of indicator diagrams as a sequence and employed LSTM to perform recognition. Compared with traditional mathematical model diagnosis methods, CNN-LSTM overcame the limitations of the traditional mathematical model theoretical analysis such as unclear assumption conditions and improved the diagnosis accuracy. Finally, 1.3 million sets of well production were set as a training dataset and used to evaluate CNN-LSTM. The results demonstrated the effectiveness of utilizing CNN and LSTM to recognize a gradual changing fault using the indicator diagram and characteristic parameters. The accuracy reached 98.4%, and the loss was less than 0.9%.

Download Full-text

Gaze Tracking Based on Concatenating Spatial-Temporal Features

Sensors ◽

10.3390/s22020545 ◽

2022 ◽

Vol 22 (2) ◽

pp. 545

Author(s):

Bor-Jiunn Hwang ◽

Hui-Hui Chen ◽

Chaur-Heh Hsieh ◽

Deng-Yu Huang

Keyword(s):

Short Term Memory ◽

Image Data ◽

Training Model ◽

Point Estimation ◽

Training Data ◽

Gaze Tracking ◽

Spatial Features ◽

Temporal Features ◽

Lstm Network ◽

The Impact

Based on experimental observations, there is a correlation between time and consecutive gaze positions in visual behaviors. Previous studies on gaze point estimation usually use images as the input for model trainings without taking into account the sequence relationship between image data. In addition to the spatial features, the temporal features are considered to improve the accuracy in this paper by using videos instead of images as the input data. To be able to capture spatial and temporal features at the same time, the convolutional neural network (CNN) and long short-term memory (LSTM) network are introduced to build a training model. In this way, CNN is used to extract the spatial features, and LSTM correlates temporal features. This paper presents a CNN Concatenating LSTM network (CCLN) that concatenates spatial and temporal features to improve the performance of gaze estimation in the case of time-series videos as the input training data. In addition, the proposed model can be optimized by exploring the numbers of LSTM layers, the influence of batch normalization (BN) and global average pooling layer (GAP) on CCLN. It is generally believed that larger amounts of training data will lead to better models. To provide data for training and prediction, we propose a method for constructing datasets of video for gaze point estimation. The issues are studied, including the effectiveness of different commonly used general models and the impact of transfer learning. Through exhaustive evaluation, it has been proved that the proposed method achieves a better prediction accuracy than the existing CNN-based methods. Finally, 93.1% of the best model and 92.6% of the general model MobileNet are obtained.

Download Full-text

Forecasting variance of NiftyIT index with RNN and DNN

Journal of Physics Conference Series ◽

10.1088/1742-6596/2161/1/012005 ◽

2022 ◽

Vol 2161 (1) ◽

pp. 012005

Author(s):

C R Karthik ◽

Raghunandan ◽

B Ashwath Rao ◽

N V Subba Reddy

Keyword(s):

Neural Network ◽

Time Series ◽

Time Series Analysis ◽

Short Term Memory ◽

Stock Exchange ◽

Network Models ◽

Training Data ◽

Complex Data ◽

Neural Network Models ◽

Series Analysis

Abstract A time series is an order of observations engaged serially in time. The prime objective of time series analysis is to build mathematical models that provide reasonable descriptions from training data. The goal of time series analysis is to forecast the forthcoming values of a series based on the history of the same series. Forecasting of stock markets is a thought-provoking problem because of the number of possible variables as well as volatile noise that may contribute to the prices of the stock. However, the capability to analyze stock market leanings could be vital to investors, traders and researchers, hence has been of continued interest. Plentiful arithmetical and machine learning practices have been discovered for stock analysis and forecasting/prediction. In this paper, we perform a comparative study on two very capable artificial neural network models i) Deep Neural Network (DNN) and ii) Long Short-Term Memory (LSTM) a type of recurrent neural network (RNN) in predicting the daily variance of NIFTYIT in BSE (Bombay Stock Exchange) and NSE (National Stock Exchange) markets. DNN was chosen due to its capability to handle complex data with substantial performance and better generalization without being saturated. LSTM model was decided, as it contains intermediary memory which can hold the historic patterns and occurrence of the next prediction depends on the values that preceded it. With both networks, measures were taken to reduce overfitting. Daily predictions of the NIFTYIT index were made to test the generalizability of the models. Both networks performed well at making daily predictions, and both generalized admirably to make daily predictions of the NiftyIT data. The LSTM-RNN outpaced the DNN in terms of forecasting and thus, grips more potential for making longer-term estimates.

Download Full-text

On the Use of Generative Deep Learning Approaches for Generating Hidden Test Scripts

International Journal of Software Engineering and Knowledge Engineering ◽

10.1142/s0218194021500480 ◽

2021 ◽

Vol 31 (10) ◽

pp. 1447-1468

Author(s):

Mert Oz ◽

Caner Kaya ◽

Erdi Olmezogullari ◽

Mehmet S. Aktas

Keyword(s):

Web Application ◽

Web Applications ◽

Short Term Memory ◽

Learning Approaches ◽

Generation Task ◽

Clickstream Data ◽

Test Script ◽

Proposed Model ◽

Action Sequences ◽

Lstm Network

With the advent of web 2.0, web application architectures have been evolved, and their complexity has grown enormously. Due to the complexity, testing of web applications is getting time-consuming and intensive process. In today’s web applications, users can achieve the same goal by performing different actions. To ensure that the entire system is safe and robust, developers try to test all possible user action sequences in the testing phase. Since the space of all the possibilities is enormous, covering all user action sequences can be impossible. To automate the test script generation task and reduce the space of the possible user action sequences, we propose a novel method based on long short-term memory (LSTM) network for generating test scripts from user clickstream data. The experiment results clearly show that generated hidden test sequences are user-like sequences, and the process of generating test scripts with the proposed model is less time-consuming than writing them manually.

Download Full-text

Simulation-based Data Augmentation for the Quality Inspection of Structural Adhesive with Deep Learning

10.36227/techrxiv.14287334 ◽

2021 ◽

Author(s):

Ricardo Peres ◽

Magno Guedes ◽

Fábio Miranda ◽

José Barata

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Data Availability ◽

Training Data ◽

Learning Approaches ◽

Automotive Parts ◽

Novel Method ◽

The Cost ◽

Structural Adhesive ◽

Data Context

<div>The advent of Industry 4.0 has shown the tremendous transformative potential of combining artificial intelligence, cyber-physical systems and Internet of Things concepts in industrial settings. Despite this, data availability is still a major roadblock for the successful adoption of data-driven solutions, particularly concerning deep learning approaches in manufacturing. Specifically in the quality control domain, annotated defect data can often be costly, time-consuming and inefficient to obtain, potentially compromising the viability of deep learning approaches due to data scarcity. In this context, we propose a novel method for generating annotated synthetic training data for automated quality inspections of structural adhesive applications, validated in an industrial cell for automotive parts. Our approach greatly reduces the cost of training deep learning models for this task, while simultaneously improving their performance in a scarce manufacturing data context with imbalanced training sets by 3.1% ([email protected]). Additional results can be seen at https://git.io/Jtc4b.</div>

Download Full-text

An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos

Sensors ◽

10.3390/s21082811 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2811

Author(s):

Waseem Ullah ◽

Amin Ullah ◽

Tanveer Hussain ◽

Zulfiqar Ahmad Khan ◽

Sung Wook Baik

Keyword(s):

Time Complexity ◽

Short Term Memory ◽

Smart Cities ◽

Vital Role ◽

Training Data ◽

Surveillance Videos ◽

Video Frames ◽

Proposed Model ◽

Lstm Network ◽

Increase In Accuracy

Video anomaly recognition in smart cities is an important computer vision task that plays a vital role in smart surveillance and public safety but is challenging due to its diverse, complex, and infrequent occurrence in real-time surveillance environments. Various deep learning models use significant amounts of training data without generalization abilities and with huge time complexity. To overcome these problems, in the current work, we present an efficient light-weight convolutional neural network (CNN)-based anomaly recognition framework that is functional in a surveillance environment with reduced time complexity. We extract spatial CNN features from a series of video frames and feed them to the proposed residual attention-based long short-term memory (LSTM) network, which can precisely recognize anomalous activity in surveillance videos. The representative CNN features with the residual blocks concept in LSTM for sequence learning prove to be effective for anomaly detection and recognition, validating our model’s effective usage in smart cities video surveillance. Extensive experiments on the real-world benchmark UCF-Crime dataset validate the effectiveness of the proposed model within complex surveillance environments and demonstrate that our proposed model outperforms state-of-the-art models with a 1.77%, 0.76%, and 8.62% increase in accuracy on the UCF-Crime, UMN and Avenue datasets, respectively.

Download Full-text

Demand Forecasting in Supply Chain Management Using Different Deep Learning Methods

Advances in Logistics, Operations, and Management Science - Demand Forecasting and Order Planning in Supply Chains and Humanitarian Logistics ◽

10.4018/978-1-7998-3805-0.ch005 ◽

2021 ◽

pp. 140-170

Author(s):

Asma Husna ◽

Saman Hassanzadeh Amin ◽

Bharat Shah

Keyword(s):

Neural Network ◽

Supply Chain ◽

Deep Learning ◽

Supply Chain Management ◽

Short Term Memory ◽

Demand Forecasting ◽

Learning Approaches ◽

Chain Management ◽

Lstm Network ◽

Artificial Neural Network Ann

Supply chain management (SCM) is a fast growing and largely studied field of research. Forecasting of the required materials and parts is an important task in companies and can have a significant impact on the total cost. To have a reliable forecast, some advanced methods such as deep learning techniques are helpful. The main goal of this chapter is to forecast the unit sales of thousands of items sold at different chain stores located in Ecuador with holistic techniques. Three deep learning approaches including artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM) are adopted here for predictions from the Corporación Favorita grocery sales forecasting dataset collected from Kaggle website. Finally, the performances of the applied models are evaluated and compared. The results show that LSTM network tends to outperform the other two approaches in terms of performance. All experiments are conducted using Python's deep learning library and Keras and Tensorflow packages.

Download Full-text

A Dynamic Improvement of a Training Dataset for Source Code Classification Using Deep Learning approach

Journal of University of Shanghai for Science and Technology ◽

10.51201/jusst/21/06501 ◽

2021 ◽

Vol 23 (06) ◽

pp. 10-22

Author(s):

Ms. Anshika Shukla ◽

◽

Mr. Sanjeev Kumar Shukla ◽

Keyword(s):

Deep Learning ◽

Classification Accuracy ◽

Source Code ◽

Training Data ◽

Classification Model ◽

Training Dataset ◽

Learning Approaches ◽

Dynamic Learning ◽

Data Set ◽

Learning Data

In recent years, there are various methods for source code classification using deep learning approaches have been proposed. The classification accuracy of the method using deep learning is greatly influenced by the training data set. Therefore, it is possible to create a model with higher accuracy by improving the construction method of the training data set. In this study, we propose a dynamic learning data set improvement method for source code classification using deep learning. In the proposed method, we first train and verify the source code classification model using the training data set. Next, we reconstruct the training data set based on the verification result. We create a high-precision model by repeating this learning and reconstruction and improving the learning data set. In the evaluation experiment, the source code classification model was learned using the proposed method, and the classification accuracy was compared with the three baseline methods. As a result, it was found that the model learned using the proposed method has the highest classification accuracy. We also confirmed that the proposed method improves the classification accuracy of the model from 0.64 to 0.96

Download Full-text

A Broad-Area Method for Estimation of Upwelling Medium Wave Infrared for Fire Detection

10.20944/preprints201612.0079.v1 ◽

2016 ◽

Author(s):

Bryan Hally ◽

Luke Wallace ◽

Karin Reinke ◽

Simon Jones

Keyword(s):

Brightness Temperature ◽

Fire Detection ◽

Data Availability ◽

Training Data ◽

Radiation Budget ◽

Training Dataset ◽

Accurate Estimation ◽

Temperature Cycle ◽

Broad Area ◽

Diurnal Temperature

Fire detection from satellite sensors relies on an accurate estimation of the unperturbed state of a target pixel, from which an anomaly can be isolated. Methods for estimating the radiation budget of a pixel without fire depend upon training data derived from the location's recent history of brightness temperature variation over the diurnal cycle, which can be vulnerable to cloud contamination and the effects of weather. This study proposes a new method that utilises the common solar budget found at a given latitude in conjunction with an area's local solar time to aggregate a broad-area training dataset, which can be used to model the expected diurnal temperature cycle of a location. This training data is then used in a temperature fitting process with the measured brightness temperatures in a pixel, and compared to pixel-derived training data and contextual methods of background temperature determination. Results of this study show similar accuracy between clear-sky medium wave infrared upwelling radiation and the diurnal temperature cycle estimation compared to previous methods, with demonstrable improvements in processing time and training data availability. This method can be used in conjunction with brightness temperature thresholds to provide a baseline for upwelling radiation, from which positive thermal anomalies such as fire can be isolated.

Download Full-text