Real-time data warehousing for business intelligence

Author(s):  
Farrah Farooq ◽  
Syed Mansoor Sarwar

In the standard ETL (Extract Processing Load), the data warehouse refreshment must be performed outside of peak hours. i It implies i that the i functioning and i analysis has stopped in their iall actions. iIt causes the iamount of icleanness of i data from the idata Warehouse which iisn't suggesting ithe latest i operational transections. This i issue is i known as i data i latency. The data warehousing is iemployed to ibe a iremedy for ithis iissue. It updates the idata warehouse iat a inear real-time iFashion, instantly after data found from the data source. Therefore, data i latency could i be reduced. Hence the near real time data warehousing was having issues which was not identified in traditional ETL. This paper claims to communicate the issues and accessible options at every point iin the i near real-time i data warehousing, i.e. i The i issues and Available alternatives iare based ion ia literature ireview by additional iStudy that ifocus ion near real-time data iwarehousing issue


2020 ◽  
Vol 30 (Supplement_5) ◽  
Author(s):  
G M Sechi ◽  
M Migliori ◽  
G Dassi ◽  
A Pagliosa ◽  
R Bonora ◽  
...  

Abstract Background In Italy on the 20th of February, the first Italian patient was tested positive for Coronavirus Disease 2019 (COVID-19) in the Lombardy region. The Regional Emergency Medical Services (EMS) Trust (Azienda Regionale Emergenza Urgenza, AREU) of the Lombardy region decided to apply a Business Intelligence (BI) System to take timely decisions on the management of EMS and to monitor the spread of the disease in the region in order to better respond to the outbreak. Methods Since the beginning of the COVID-19 outbreak, AREU developed a BI System to track the daily number of first aid requests received from 1.1.2. (Public Safety Answering Point 1). BI evaluates the number of requests that have been classified as respiratory and/or infectious episodes during the telephone dispatch interview. Moreover, BI analyses the pattern of the epidemic, identifying the numerical trend of episodes in each municipality (increasing, stable, decreasing). Currently, AREU is still implementing the BI as the epidemic is still ongoing. Results In the Lombardy region on the 20th of February the number of the first aid requests for respiratory and/or infectious episodes were 314. This figure increased sharply during the month of February and March reaching its peak on the 16th of March with 1537 episodes. In the area around Bergamo, this number experienced a greater rise compared to the rest of the Lombardy territory, going from 74 episodes on the 20th of February to 694 on the 13th of March. Therefore, AREU decided to reallocate in the territory the resources (ambulances and human resources) based on the real-time data elaborated by the BI system. Conclusions The BI System has been of paramount importance in taking timely decisions on the management of EMS during the COVID-19 outbreak in the Lombardy region. Indeed, BI can be usefully applied to promptly identify the trend of the COVID-19 epidemic and, consequently, make informed decisions to improve the response to the outbreak. Key messages The Emergency Medical Services Trust of the Lombardy region applied a Business Intelligence System to promptly respond to the outbreak of COVID-19 and reallocate the resources based on real-time data. AREU used a Business Intelligence System to track the daily number of first aid requests that have been classified as respiratory and/or infectious episodes during the telephone dispatch interview.


2021 ◽  
Author(s):  
Flavio de Assis Vilela ◽  
Ricardo Rodrigues Ciferri

ETL (Extract, Transform, and Load) is an essential process required to perform data extraction in knowledge discovery in databases and in data warehousing environments. The ETL process aims to gather data that is available from operational sources, process and store them into an integrated data repository. Also, the ETL process can be performed in a real-time data warehousing environment and store data into a data warehouse. This paper presents a new and innovative method named Data Extraction Magnet (DEM) to perform the extraction phase of ETL process in a real-time data warehousing environment based on non-intrusive, tag and parallelism concepts. DEM has been validated on a dairy farming domain using synthetic data. The results showed a great performance gain in comparison to the traditional trigger technique and the attendance of real-time requirements.


Sign in / Sign up

Export Citation Format

Share Document