Data-Driven Fault Detection in Large-Scale and Distributed Systems

2020 ◽  
pp. 351-424
Author(s):  
Steven X. Ding
Keyword(s):  
Distributed Systems ◽  
Fault Detection ◽  
Large Scale ◽  
Data Driven

Distributed data-driven optimal fault detection for large-scale systems

2020 ◽  
Vol 96 ◽  
pp. 94-103
Author(s):  
Linlin Li ◽  
Steven X. Ding ◽  
Xin Peng
Keyword(s):  
Fault Detection ◽  
Large Scale ◽  
Data Driven ◽  
Distributed Data ◽  
Large Scale Systems

A Comparative Assessment on Static and Dynamic PCA for Fault Detection in Natural Gas Transmission Systems

2017 ◽  
Author(s):  
Horacio Pinzón ◽  
Cinthia Audivet ◽  
Melitsa Torres ◽  
Javier Alexander ◽  
Marco Sanjuán
Keyword(s):  
Natural Gas ◽  
Fault Detection ◽  
Large Scale ◽  
Detection System ◽  
Principal Component ◽  
Data Driven ◽  
Transmission Systems ◽  
Alarm Management ◽  
Detection Algorithms ◽  
Natural Gas Transmission

Sustainability of natural gas transmission infrastructure is highly related to the system’s ability to decrease emissions due to ruptures or leaks. Although traditionally such detection relies in alarm management system and operator’s expertise, given the system’s nature as large-scale, complex, and with vast amount of information available, such alarm generation is better suited for a fault detection system based on data-driven techniques. This would allow operators and engineers to have a better framework to address the online data being gathered. This paper presents an assessment on multiple fault-case scenarios in critical infrastructure using two different data-driven based fault detection algorithms: Principal component analysis (PCA) and its dynamic variation (DPCA). Both strategies are assessed under fault scenarios related to natural gas transmission systems including pipeline leakage due to structural failure and flow interruption due to emergency valve shut down. Performance evaluation of fault detection algorithms is carried out based on false alarm rate, detection time and misdetection rate. The development of modern alarm management frameworks would have a significant contribution in natural gas transmission systems’ safety, reliability and sustainability.

scholarly journals Accelerating In-Transit Co-Processing for Scientific Simulations Using Region-Based Data-Driven Analysis

Algorithms ◽  
2021 ◽  
Vol 14 (5) ◽  
pp. 154
Author(s):  
Marcus Walldén ◽  
Masao Okita ◽  
Fumihiko Ino ◽  
Dimitris Drikakis ◽  
Ioannis Kokkinakis
Keyword(s):  
Large Scale ◽  
Data Driven ◽  
Data Sets ◽  
Output Constraints ◽  
Data Driven Approach ◽  
Scientific Simulations ◽  
Multiple Metrics ◽  
In Transit ◽  
Multiple Compression ◽  
Large Scale Simulations

Increasing processing capabilities and input/output constraints of supercomputers have increased the use of co-processing approaches, i.e., visualizing and analyzing data sets of simulations on the fly. We present a method that evaluates the importance of different regions of simulation data and a data-driven approach that uses the proposed method to accelerate in-transit co-processing of large-scale simulations. We use the importance metrics to simultaneously employ multiple compression methods on different data regions to accelerate the in-transit co-processing. Our approach strives to adaptively compress data on the fly and uses load balancing to counteract memory imbalances. We demonstrate the method’s efficiency through a fluid mechanics application, a Richtmyer–Meshkov instability simulation, showing how to accelerate the in-transit co-processing of simulations. The results show that the proposed method expeditiously can identify regions of interest, even when using multiple metrics. Our approach achieved a speedup of 1.29× in a lossless scenario. The data decompression time was sped up by 2× compared to using a single compression method uniformly.

scholarly journals Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

2021 ◽  
Author(s):  
Ekaterina Kochmar ◽  
Dung Do Vu ◽  
Robert Belfer ◽  
Varun Gupta ◽  
Iulian Vlad Serban ◽  
...  
Keyword(s):  
Machine Learning ◽  
Student Performance ◽  
Language Processing ◽  
Intelligent Tutoring Systems ◽  
Large Scale ◽  
Intelligent Tutoring ◽  
Performance Outcomes ◽  
Data Driven ◽  
Personalized Feedback ◽  
Tutoring Systems

AbstractIntelligent tutoring systems (ITS) have been shown to be highly effective at promoting learning as compared to other computer-based instructional approaches. However, many ITS rely heavily on expert design and hand-crafted rules. This makes them difficult to build and transfer across domains and limits their potential efficacy. In this paper, we investigate how feedback in a large-scale ITS can be automatically generated in a data-driven way, and more specifically how personalization of feedback can lead to improvements in student performance outcomes. First, in this paper we propose a machine learning approach to generate personalized feedback in an automated way, which takes individual needs of students into account, while alleviating the need of expert intervention and design of hand-crafted rules. We leverage state-of-the-art machine learning and natural language processing techniques to provide students with personalized feedback using hints and Wikipedia-based explanations. Second, we demonstrate that personalized feedback leads to improved success rates at solving exercises in practice: our personalized feedback model is used in , a large-scale dialogue-based ITS with around 20,000 students launched in 2019. We present the results of experiments with students and show that the automated, data-driven, personalized feedback leads to a significant overall improvement of 22.95% in student performance outcomes and substantial improvements in the subjective evaluation of the feedback.

Workshop on large-scale distributed systems for information retrieval

2007 ◽  
Vol 41 (2) ◽  
pp. 83-88
Author(s):  
Flavio P. Junqueira ◽  
Vassilis Plachouras ◽  
Fabrizio Silvestri ◽  
Ivana Podnar
Keyword(s):  
Information Retrieval ◽  
Distributed Systems ◽  
Large Scale
Sign in / Sign up

Export Citation Format

Share Document