HTAP With Reactive Streaming ETL

In database management systems (DBMSs), query workloads can be classified as online transactional processing (OLTP) or online analytical processing (OLAP). These often run within separate DBMSs. In hybrid transactional and analytical processing (HTAP), both workloads may execute within the same DBMS. This article shows that it is possible to run separate OLTP and OLAP DBMSs, and still support timely business decisions from analytical queries running off fresh transactional data. Several setups to manage OLTP and OLAP workloads are analysed. Then, benchmarks on two industry standard DBMSs empirically show that, under an OLTP workload, a row-store DBMS sustains a 1000 times higher throughput than a columnar DBMS, whilst OLAP queries are more than 4 times faster on a columnar DBMS. Finally, a reactive streaming ETL pipeline is implemented which connects these two DBMSs. Separate benchmarks show that OLTP events can be streamed to an OLAP database within a few seconds.

Download Full-text

HTAP With Reactive Streaming ETL

Journal of Cases on Information Technology ◽

10.4018/jcit.20211001.oa10 ◽

2021 ◽

Vol 23 (4) ◽

pp. 1-19

Author(s):

Carl Camilleri ◽

Joseph G. Vella ◽

Vitezslav Nezval

Keyword(s):

Database Management ◽

Database Management Systems ◽

Online Analytical Processing ◽

Management Systems ◽

Business Decisions ◽

Industry Standard ◽

Analytical Processing ◽

Transactional Data

Download Full-text

A Survey of Open Source Tools for Business Intelligence

Integrations of Data Warehousing, Data Mining and Database Technologies ◽

10.4018/978-1-60960-537-7.ch010 ◽

2011 ◽

pp. 237-257

Keyword(s):

Open Source ◽

Business Intelligence ◽

Database Management ◽

Database Management Systems ◽

Management Systems ◽

Survey Article ◽

On Line ◽

Analytical Processing ◽

Industrial Use ◽

On Line Analytical Processing

The industrial use of open source Business Intelligence (BI) tools is becoming more common, but is still not as widespread as for other types of software. It is therefore of interest to explore which possibilities are available for open source BI and compare the tools. In this survey article, we consider the capabilities of a number of open source tools for BI. In the article, we consider a number of Extract- Transform-Load (ETL) tools, database management systems (DBMSs), On-Line Analytical Processing (OLAP) servers, and OLAP clients. We find that, unlike the situation a few years ago, there now exist mature and powerful tools in all these categories. However, the functionality still falls somewhat short of that found in commercial tools.

Download Full-text

A Survey of Open Source Tools for Business Intelligence

Information Resources Management ◽

10.4018/978-1-61520-965-1.ch309 ◽

2010 ◽

pp. 624-644

Author(s):

Christian Thomsen ◽

Torben Bach Pedersen

Keyword(s):

Open Source ◽

Business Intelligence ◽

Database Management ◽

Database Management Systems ◽

Management Systems ◽

Survey Article ◽

On Line ◽

Analytical Processing ◽

Industrial Use ◽

On Line Analytical Processing

The industrial use of open source Business Intelligence (BI) tools is becoming more common, but is still not as widespread as for other types of software. It is therefore of interest to explore which possibilities are available for open source BI and compare the tools. In this survey article, we consider the capabilities of a number of open source tools for BI. In the article, we consider a number of Extract-Transform-Load (ETL) tools, database management systems (DBMSs), On-Line Analytical Processing (OLAP) servers, and OLAP clients. We find that, unlike the situation a few years ago, there now exist mature and powerful tools in all these categories. However, the functionality still falls somewhat short of that found in commercial tools.

Download Full-text