scholarly journals Issues and Handy Solutions Addressed at Every Stage in Real Time Data Warehousing, I.E. ETL (Extraction, Transformation & Loading)

2019 ◽  
Vol 8 (5S3) ◽  
pp. 344-348
Keyword(s):  
Real Time ◽  
Data Warehouse ◽  
Data Warehousing ◽  
Time Data ◽  
Processing Load ◽  
Real Time Data ◽  
Data Source

In the standard ETL (Extract Processing Load), the data warehouse refreshment must be performed outside of peak hours. i It implies i that the i functioning and i analysis has stopped in their iall actions. iIt causes the iamount of icleanness of i data from the idata Warehouse which iisn't suggesting ithe latest i operational transections. This i issue is i known as i data i latency. The data warehousing is iemployed to ibe a iremedy for ithis iissue. It updates the idata warehouse iat a inear real-time iFashion, instantly after data found from the data source. Therefore, data i latency could i be reduced. Hence the near real time data warehousing was having issues which was not identified in traditional ETL. This paper claims to communicate the issues and accessible options at every point iin the i near real-time i data warehousing, i.e. i The i issues and Available alternatives iare based ion ia literature ireview by additional iStudy that ifocus ion near real-time data iwarehousing issue

scholarly journals Kajian Pendekatan Binary Log dalam Change Data Capture

2017 ◽  
pp. 11
Author(s):  
Muhammad Febrian Rachmadhan Amri ◽  
I Made Sukarsa ◽  
I Ketut Adi Purnawan
Keyword(s):  
Data Management ◽  
Real Time ◽  
Data Warehouse ◽  
Data Capture ◽  
Problem Solution ◽  
Time Data ◽  
Quality Of Data ◽  
Data Manipulation ◽  
Change Data Capture ◽  
Real Time Data

The online business era causes the form of transactions to occur so quickly that the information stored in the data warehouse becomes invalid. Companies are required to have a strong system, which is a system that is real time in order to be able to perform data loading into the media repository that resides on different hosts in the near-real time. Data Warehouse is used as a media repository of data that has the nature of subject-oriented, integrated, time-variant, and is fixed. Data Warehouse can be built into real time management with the advantages possessed and utilize Change Data Capture. Change Data Capture (CDC) is a technique that can be used as problem solution to build real time data warehousing (RTDW). The binary log approach in change data capture is made to record any data manipulation activity that occurs at the OLTP level and is managed back before being stored into the Data Warehouse (loading process). This can improve the quality of data management so that the creation of the right information, because the information available is always updated. Testing shows that Binary Log approach in Change Data Capture (BinlogCDC) is able to generate real time data management, valid current information, dynamic communication between systems, and data management without losing any information from data manipulation.

Sign in / Sign up

Export Citation Format

Share Document