Causal Discovery with Attention-Based Convolutional Neural Networks

Having insight into the causal associations in a complex system facilitates decision making, e.g., for medical treatments, urban infrastructure improvements or financial investments. The amount of observational data grows, which enables the discovery of causal relationships between variables from observation of their behaviour in time. Existing methods for causal discovery from time series data do not yet exploit the representational power of deep learning. We therefore present the Temporal Causal Discovery Framework (TCDF), a deep learning framework that learns a causal graph structure by discovering causal relationships in observational time series data. TCDF uses attention-based convolutional neural networks combined with a causal validation step. By interpreting the internal parameters of the convolutional networks, TCDF can also discover the time delay between a cause and the occurrence of its effect. Our framework learns temporal causal graphs, which can include confounders and instantaneous effects. Experiments on financial and neuroscientific benchmarks show state-of-the-art performance of TCDF on discovering causal relationships in continuous time series data. Furthermore, we show that TCDF can circumstantially discover the presence of hidden confounders. Our broadly applicable framework can be used to gain novel insights into the causal dependencies in a complex system, which is important for reliable predictions, knowledge discovery and data-driven decision making.

Download Full-text

Ship-Collision Avoidance Decision-Making Learning of Unmanned Surface Vehicles with Automatic Identification System Data Based on Encoder—Decoder Automatic-Response Neural Networks

Journal of Marine Science and Engineering ◽

10.3390/jmse8100754 ◽

2020 ◽

Vol 8 (10) ◽

pp. 754

Author(s):

Miao Gao ◽

Guo-You Shi

Keyword(s):

Neural Networks ◽

Decision Making ◽

Time Series ◽

Collision Avoidance ◽

Time Series Data ◽

Series Data ◽

Automatic Identification ◽

Identification System ◽

Automatic Response ◽

Ship Collision

Intelligent unmanned surface vehicle (USV) collision avoidance is a complex inference problem based on current navigation status. This requires simultaneous processing of the input sequences and generation of the response sequences. The automatic identification system (AIS) encounter data mainly include the time-series data of two AIS sets, which exhibit a one-to-one mapping relation. Herein, an encoder–decoder automatic-response neural network is designed and implemented based on the sequence-to-sequence (Seq2Seq) structure to simultaneously process the two AIS encounter trajectory sequences. Furthermore, this model is combined with the bidirectional long short-term memory recurrent neural networks (Bi-LSTM RNN) to obtain a network framework for processing the time-series data to obtain ship-collision avoidance decisions based on big data. The encoder–decoder neural networks were trained based on the AIS data obtained in 2018 from Zhoushan Port to achieve ship collision avoidance decision-making learning. The results indicated that the encoder–decoder neural networks can be used to effectively formulate the sequence of the collision avoidance decision of the USV. Thus, this study significantly contributes to the increased efficiency and safety of maritime transportation. The proposed method can potentially be applied to the USV technology and intelligent collision-avoidance systems.

Download Full-text

A New Method of Mixed Gas Identification Based on a Convolutional Neural Network for Time Series Classification

Sensors ◽

10.3390/s19091960 ◽

2019 ◽

Vol 19 (9) ◽

pp. 1960 ◽

Cited By ~ 8

Author(s):

Lu Han ◽

Chongchong Yu ◽

Kaitai Xiao ◽

Xia Zhao

Keyword(s):

Neural Network ◽

Neural Networks ◽

Time Series ◽

Convolutional Neural Networks ◽

Time Series Data ◽

Series Data ◽

Time Series Classification ◽

Mixed Gas ◽

Gas Identification ◽

Mixed Gases

This paper proposes a new method of mixed gas identification based on a convolutional neural network for time series classification. In view of the superiority of convolutional neural networks in the field of computer vision, we applied the concept to the classification of five mixed gas time series data collected by an array of eight MOX gas sensors. Existing convolutional neural networks are mostly used for processing visual data, and are rarely used in gas data classification and have great limitations. Therefore, the idea of mapping time series data into an analogous-image matrix data is proposed. Then, five kinds of convolutional neural networks—VGG-16, VGG-19, ResNet18, ResNet34 and ResNet50—were used to classify and compare five kinds of mixed gases. By adjusting the parameters of the convolutional neural networks, the final gas recognition rate is 96.67%. The experimental results show that the method can classify the gas data quickly and effectively, and effectively combine the gas time series data with classical convolutional neural networks, which provides a new idea for the identification of mixed gases.

Download Full-text

Convolutional neural networks for modeling and forecasting nonlinear nonstationary processes

ScienceRise ◽

10.21303/2313-8416.2021.001924 ◽

2021 ◽

pp. 12-20

Author(s):

Andrii Belas ◽

Petro Bidyuk

Keyword(s):

Neural Networks ◽

Time Series ◽

Convolutional Neural Networks ◽

Recurrent Neural Networks ◽

Time Series Data ◽

Autoregressive Models ◽

Series Data ◽

Nonstationary Processes ◽

Modeling And Forecasting ◽

Technological Product

The object of research. The object of research is modeling and forecasting nonlinear nonstationary processes presented in the form of time-series data. Investigated problem. There are several popular approaches to solving the problems of adequate model constructing and forecasting nonlinear nonstationary processes, such as autoregressive models and recurrent neural networks. However, each of them has its advantages and drawbacks. Autoregressive models cannot deal with the nonlinear or combined influence of previous states or external factors. Recurrent neural networks are computationally expensive and cannot work with sequences of high length or frequency. The main scientific result. The model for forecasting nonlinear nonstationary processes presented in the form of the time series data was built using convolutional neural networks. The current study shows results in which convolutional networks are superior to recurrent ones in terms of both accuracy and complexity. It was possible to build a more accurate model with a much fewer number of parameters. It indicates that one-dimensional convolutional neural networks can be a quite reasonable choice for solving time series forecasting problems. The area of practical use of the research results. Forecasting dynamics of processes in economy, finances, ecology, healthcare, technical systems and other areas exhibiting the types of nonlinear nonstationary processes. Innovative technological product. Methodology of using convolutional neural networks for modeling and forecasting nonlinear nonstationary processes presented in the form of time-series data. Scope of the innovative technological product. Nonlinear nonstationary processes presented in the form of time-series data.

Download Full-text

Tactile texture recognition using convolutional neural networks for time-series data of pressure and 6-axis acceleration sensor

2017 IEEE International Conference on Industrial Technology (ICIT) ◽

10.1109/icit.2017.7915511 ◽

2017 ◽

Cited By ~ 5

Author(s):

Hideaki Orii ◽

Satoshi Tsuji ◽

Takaharu Kouda ◽

Teruhiko Kohama

Keyword(s):

Neural Networks ◽

Time Series ◽

Convolutional Neural Networks ◽

Time Series Data ◽

Series Data ◽

Acceleration Sensor ◽

Texture Recognition

Download Full-text

Deep Learning Based Superconducting Radio-Frequency Cavity Fault Classification at Jefferson Laboratory

Frontiers in Artificial Intelligence ◽

10.3389/frai.2021.718950 ◽

2022 ◽

Vol 4 ◽

Author(s):

Lasitha Vidyaratne ◽

Adam Carpenter ◽

Tom Powers ◽

Chris Tennant ◽

Khan M. Iftekharuddin ◽

...

Keyword(s):

Neural Networks ◽

Time Series ◽

Deep Learning ◽

Radio Frequency ◽

Large Scale ◽

Continuous Wave ◽

Time Series Data ◽

Fault Classification ◽

Series Data ◽

Superconducting Radio Frequency

This work investigates the efficacy of deep learning (DL) for classifying C100 superconducting radio-frequency (SRF) cavity faults in the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. CEBAF is a large, high-power continuous wave recirculating linac that utilizes 418 SRF cavities to accelerate electrons up to 12 GeV. Recent upgrades to CEBAF include installation of 11 new cryomodules (88 cavities) equipped with a low-level RF system that records RF time-series data from each cavity at the onset of an RF failure. Typically, subject matter experts (SME) analyze this data to determine the fault type and identify the cavity of origin. This information is subsequently utilized to identify failure trends and to implement corrective measures on the offending cavity. Manual inspection of large-scale, time-series data, generated by frequent system failures is tedious and time consuming, and thereby motivates the use of machine learning (ML) to automate the task. This study extends work on a previously developed system based on traditional ML methods (Tennant and Carpenter and Powers and Shabalina Solopova and Vidyaratne and Iftekharuddin, Phys. Rev. Accel. Beams, 2020, 23, 114601), and investigates the effectiveness of deep learning approaches. The transition to a DL model is driven by the goal of developing a system with sufficiently fast inference that it could be used to predict a fault event and take actionable information before the onset (on the order of a few hundred milliseconds). Because features are learned, rather than explicitly computed, DL offers a potential advantage over traditional ML. Specifically, two seminal DL architecture types are explored: deep recurrent neural networks (RNN) and deep convolutional neural networks (CNN). We provide a detailed analysis on the performance of individual models using an RF waveform dataset built from past operational runs of CEBAF. In particular, the performance of RNN models incorporating long short-term memory (LSTM) are analyzed along with the CNN performance. Furthermore, comparing these DL models with a state-of-the-art fault ML model shows that DL architectures obtain similar performance for cavity identification, do not perform quite as well for fault classification, but provide an advantage in inference speed.

Download Full-text

Financial Time-Series Data Analysis Using Deep Convolutional Neural Networks

2016 7th International Conference on Cloud Computing and Big Data (CCBD) ◽

10.1109/ccbd.2016.027 ◽

2016 ◽

Cited By ~ 21

Author(s):

Jou-Fan Chen ◽

Wei-Lun Chen ◽

Chun-Ping Huang ◽

Szu-Hao Huang ◽

An-Pin Chen

Keyword(s):

Neural Networks ◽

Time Series ◽

Data Analysis ◽

Convolutional Neural Networks ◽

Time Series Data ◽

Financial Time Series ◽

Series Data ◽

Deep Convolutional Neural Networks ◽

Financial Time ◽

Time Series Data Analysis

Download Full-text

Using deep convolutional neural networks to forecast spatial patterns of Amazonian deforestation

10.1101/2021.12.14.472442 ◽

2021 ◽

Author(s):

James George Clifford Ball ◽

Katerina Petrova ◽

David Coomes ◽

Seth Flaxman

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Tropical Forest ◽

Convolutional Neural Networks ◽

Time Series Data ◽

Short Term Memory ◽

Series Data ◽

Surface Model ◽

Observation Data ◽

Deep Convolutional Neural Networks

1. Tropical forests are subject to diverse deforestation pressures but their conservation is essential to achieve global climate goals. Predicting the location of deforestation is challenging due to the complexity of the natural and human systems involved but accurate and timely forecasts could enable effective planning and on-the-ground enforcement practices to curb deforestation rates. New computer vision technologies based on deep learning can be applied to the increasing volume of Earth observation data to generate novel insights and make predictions with unprecedented accuracy. 2. Here, we demonstrate the ability of deep convolutional neural networks to learn spatiotemporal patterns of deforestation from a limited set of freely available global data layers, including multispectral satellite imagery, the Hansen maps of historic deforestation (2001-2020) and the ALOS JAXA digital surface model, to forecast future deforestation (2021). We designed four original deep learning model architectures, based on 2D Convolutional Neural Networks (2DCNN), 3D Convolutional Neural Networks (3DCNN), and Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNN) to produce spatial maps that indicate the risk to each forested pixel (~30 m) in the landscape of becoming deforested within the next year.. They were trained and tested on data from two ~80,000 km2 tropical forest regions in the Southern Peruvian Amazon. 3. We found that the networks could predict the likely location of future deforestation to a high degree of accuracy. Our best performing model - a 3DCNN - had the highest pixel-wise accuracy (80-90%) when validated on 2020 deforestation based 2014-2019 training. Visual examination of the forecasts indicated that the 3DCNN network could automatically discern the drivers of forest loss from the input data. For example, pixels around new access routes (e.g. roads) were assigned high risk whereas this was not the case for recent, concentrated natural loss events (e.g. remote landslides). 4. CNNs can harness limited time-series data to predict near-future deforestation patterns, an important step in using the growing volume of satellite remote sensing data to curb global deforestation. The modelling framework can be readily applied to any tropical forest location and used by governments and conservation organisations to prevent deforestation and plan protected areas.

Download Full-text