An Efficient Stochastic Update Propagation Method in Data Warehousing

2018 ◽  
Vol 29 (2) ◽  
pp. 23-41
Author(s):  
Bijoy Bordoloi ◽  
Bhushan Kapoor ◽  
Tim Jacks
Keyword(s):  
Real Time ◽  
Data Storage ◽  
Data Warehousing ◽  
Time Lag ◽  
Time Data ◽  
Storage And Retrieval ◽  
Update Propagation ◽  
Data Store ◽  
Data Propagation ◽  
Source Data

This article develops a stochastic update propagation method for an operational data store (ODS) in data warehousing (DW) environments where data storage (and retrieval) is required as a sum of data at distributed source nodes. The authors' proposed method results in less network traffic (as compared with the real-time method) due to update propagation required because of changes in source data. More importantly, the method allows system users to place limits on the discrepancy between the source data and the ODS data that could result due to a time lag between source data changes and the update operation. Finally, the pre-specified limits on the discrepancy are maintained while accounting for two crucial factors in distributed systems: 1) some nodes are situated on more congested network links, and 2) some of the links on the network are less reliable. Real-time data propagation does not account for these frequently encountered networking concerns.

An Efficient Stochastic Update Propagation Method in Data Warehousing

2021 ◽  
pp. 1968-1987
Author(s):  
Bijoy Bordoloi ◽  
Bhushan Kapoor ◽  
Tim Jacks
Keyword(s):  
Real Time ◽  
Data Storage ◽  
Data Warehousing ◽  
Time Lag ◽  
Time Data ◽  
Storage And Retrieval ◽  
Update Propagation ◽  
Data Store ◽  
Data Propagation ◽  
Source Data

This article develops a stochastic update propagation method for an operational data store (ODS) in data warehousing (DW) environments where data storage (and retrieval) is required as a sum of data at distributed source nodes. The authors' proposed method results in less network traffic (as compared with the real-time method) due to update propagation required because of changes in source data. More importantly, the method allows system users to place limits on the discrepancy between the source data and the ODS data that could result due to a time lag between source data changes and the update operation. Finally, the pre-specified limits on the discrepancy are maintained while accounting for two crucial factors in distributed systems: 1) some nodes are situated on more congested network links, and 2) some of the links on the network are less reliable. Real-time data propagation does not account for these frequently encountered networking concerns.

scholarly journals Issues and Handy Solutions Addressed at Every Stage in Real Time Data Warehousing, I.E. ETL (Extraction, Transformation & Loading)

2019 ◽  
Vol 8 (5S3) ◽  
pp. 344-348
Keyword(s):  
Real Time ◽  
Data Warehouse ◽  
Data Warehousing ◽  
Time Data ◽  
Processing Load ◽  
Real Time Data ◽  
Data Source

In the standard ETL (Extract Processing Load), the data warehouse refreshment must be performed outside of peak hours. i It implies i that the i functioning and i analysis has stopped in their iall actions. iIt causes the iamount of icleanness of i data from the idata Warehouse which iisn't suggesting ithe latest i operational transections. This i issue is i known as i data i latency. The data warehousing is iemployed to ibe a iremedy for ithis iissue. It updates the idata warehouse iat a inear real-time iFashion, instantly after data found from the data source. Therefore, data i latency could i be reduced. Hence the near real time data warehousing was having issues which was not identified in traditional ETL. This paper claims to communicate the issues and accessible options at every point iin the i near real-time i data warehousing, i.e. i The i issues and Available alternatives iare based ion ia literature ireview by additional iStudy that ifocus ion near real-time data iwarehousing issue

scholarly journals Analisa Real-Time Data log honeypot menggunakan Algoritma K-Means pada serangan Distributed Denial of Service

Repositor ◽  
2020 ◽  
Vol 2 (5) ◽  
pp. 541
Author(s):  
Denni Septian Hermawan ◽  
Syaifuddin Syaifuddin ◽  
Diah Risqiwati
Keyword(s):  
Real Time ◽  
Data Storage ◽  
Denial Of Service ◽  
The Internet ◽  
Distributed Denial Of Service ◽  
Time Data ◽  
Local Networks ◽  
Real Time Data ◽  
Internet Networks ◽  
Data Log

AbstrakJaringan internet yang saat ini di gunakan untuk penyimpanan data atau halaman informasi pada website menjadi rentan terhadap serangan, untuk meninkatkan keamanan website dan jaringannya, di butuhkan honeypot yang mampu menangkap serangan yang di lakukan pada jaringan lokal dan internet. Untuk memudahkan administrator mengatasi serangan digunakanlah pengelompokan serangan dengan metode K-Means untuk mengambil ip penyerang. Pembagian kelompok pada titik cluster akan menghasilkan output ip penyerang.serangan di ambil sercara realtime dari log yang di miliki honeypot dengan memanfaatkan MHN.Abstract The number of internet networks used for data storage or information pages on the website is vulnerable to attacks, to secure the security of their websites and networks, requiring honeypots that are capable of capturing attacks on local networks and the internet. To make it easier for administrators to tackle attacks in the use of attacking groupings with the K-Means method to retrieve the attacker ip. Group divisions at the cluster point will generate the ip output of the attacker. The strike is taken as realtime from the logs that have honeypot by utilizing the MHN.

scholarly journals Drilling Efficiency Improvement and Rate of Penetration Optimization by Machine Learning and Data Analytics

2020 ◽  
Vol 5 (3) ◽  
pp. 381-394
Author(s):  
Sridharan Chandrasekaran ◽  
G. Suresh Kumar
Keyword(s):  
Machine Learning ◽  
Real Time ◽  
Data Storage ◽  
Predictive Accuracy ◽  
Empirical Models ◽  
Operational Parameters ◽  
Time Data ◽  
Bottom Line ◽  
Rate Of Penetration ◽  
Real Time Data

Rate of Penetration (ROP) is one of the important factors influencing the drilling efficiency. Since cost recovery is an important bottom line in the drilling industry, optimizing ROP is essential to minimize the drilling operational cost and capital cost. Traditional the empirical models are not adaptive to new lithology changes and hence the predictive accuracy is low and subjective. With advancement in big data technologies, real- time data storage cost is lowered, and the availability of real-time data is enhanced. In this study, it is shown that optimization methods together with data models has immense potential in predicting ROP based on real time measurements on the rig. A machine learning based data model is developed by utilizing the offset vertical wells’ real time operational parameters while drilling. Data pre-processing methods and feature engineering methods modify the raw data into a processed data so that the model learns effectively from the inputs. A multi – layer back propagation neural network is developed, cross-validated and compared with field measurements and empirical models.

Status Monitoring and Fault Diagnosis Network System Based on UPnP

2007 ◽  
Vol 353-358 ◽  
pp. 2632-2635
Author(s):  
Pei Yu Li ◽  
Da Peng Tan ◽  
Tao Qing Zhou ◽  
Bo Yu Lin
Keyword(s):  
Fault Diagnosis ◽  
Embedded System ◽  
Real Time ◽  
Data Storage ◽  
Network Architecture ◽  
Network System ◽  
Time Data ◽  
Status Monitoring ◽  
Self Recognition ◽  
Real Time Data

Aiming at some problems in the fields of industry monitoring technology (IMT) such as bad dynamic ability and poor versatility, this paper brought forward a kind of intelligent Status monitoring and Fault diagnosis Network System (SFNS) based on UPnP-Universal Plug and Play. The model for fault diagnosis network system was established according to characteristics and requirements of IMT network, and system network architecture was designed and realized by UPnP. Using embedded system technology, real-time data collection node, monitoring center node and data storage server were designed, and that supplies powerful real-time data support for SFNS. Industry fields experiments proved that this system can realize self recognition, seamless linkage and other self adapting ability, and can break through the limitation of real IP address to achieve real-time remote monitoring on line.

scholarly journals Software Design of a Railway Signal Monitoring System based on Optical Fiber Sensing

2018 ◽  
Vol 14 (08) ◽  
pp. 134
Author(s):  
Ma Chun-ying ◽  
Li Biqing
Keyword(s):  
Real Time ◽  
Monitoring System ◽  
Data Storage ◽  
Online Monitoring ◽  
Railway Track ◽  
Time Data ◽  
Railway Systems ◽  
Management Platform ◽  
Railway Signal ◽  
Optical Fiber Sensing

The current railway track circuit monitoring system is prone to disturbances that can result in accidents. Meanwhile, basic signaling equipment is slow and cannot achieve satisfactory real-time data acquisition speed. This study aims to solve the aforementioned problems by designing an online monitoring and management platform for railway signal infrastructure, which is based on the graphical programming language LabVIEW. Online monitoring and management of railways’ basic signaling equipment allow real-time collection and communication of various signal equipment data. These processes also enable signal processing, chart display, acousto-optic alarm, user authority management, data storage, data query analysis, and report printing. The test results show that the LabVIEW-based basic signaling equipment for monitoring and managing railway systems can transmit data correctly and steadily, thereby resulting in convenient and ideal operation.

Application of artificial intelligence in university sports risk recognition and identification

2020 ◽  
pp. 1-12
Author(s):  
Ju-An Wang ◽  
Shen Liu ◽  
Xiping Zhang
Keyword(s):  
Artificial Intelligence ◽  
Risk Factors ◽  
Real Time ◽  
Risk Mitigation ◽  
Time Data ◽  
Teaching Preparation ◽  
Data Store ◽  
Teacher Factors ◽  
Risk Sources ◽  
Artificial Intelligence Technology

This article is based on artificial intelligence technology to recognize and identify risks in college sport. The application of motion recognition technology first need to collect the source data, store the collected data in the server database, collect the learner’s real-time data and return it to the database to achieve the purpose of real-time monitoring. It is found that in the identification of risk sources of sports courses, there are a total of 4 first-level risk factors, namely teacher factors, student factors, environmental factors, and school management factors, and a total of 15 second-level risk factors, which are teaching preparation, teaching process, and teaching effect. When the frequency of teaching risks is low, the consequence loss is small. When the frequency of teaching risks is low, the consequences are very serious. Risk mitigation is the main measure to reduce the occurrence of teaching risks and reduce the consequences of losses.

Sign in / Sign up

Export Citation Format

Share Document