Data Warehouse Software

2011 ◽  
pp. 179-184
Author(s):  
Huanyu Ouyang ◽  
John Wang
Keyword(s):  
Data Warehouse ◽  
Data Storage ◽  
Data Warehousing ◽  
Information Analysis ◽  
Distribution Solution ◽  
The World ◽  
Information Center ◽  
Operational Systems ◽  
Data Source ◽  
Flow Of Information

A data warehouse (DW) is a complete intelligent data storage and information delivery or distribution solution enabling users to customize the flow of information through their organization (Inmon & Hackathorn, 2002). It provides all authorized members of users’ organization with flexible, secure, and rapid access to critical information and intelligent reporting. DW can extract information from sources anywhere in the world and then delivers intelligence anywhere in the world. It connects to any platform, database, data source, and it will also scale to businesses and applications of any size. As early as the 1970’s, data warehousing software (DWS) was recognized when the earliest systems were first developed. The database designs of operational systems were not effective enough for the information analysis and reporting (The Data Warehousing Information Center, 2006).

Optimizing ETL by a Two-Level Data Staging Method

2016 ◽  
Vol 12 (3) ◽  
pp. 32-50
Author(s):  
Xiufeng Liu ◽  
Nadeem Iftikhar ◽  
Huan Huo ◽  
Per Sieverts Nielsen
Keyword(s):  
Data Warehouse ◽  
Data Warehousing ◽  
Data Extraction ◽  
Data Staging ◽  
Staging Area ◽  
Level Data ◽  
Different Types ◽  
Operational Systems ◽  
Early Late ◽  
Central Data

In data warehousing, the data from source systems are populated into a central data warehouse (DW) through extraction, transformation and loading (ETL). The standard ETL approach usually uses sequential jobs to process the data with dependencies, such as dimension and fact data. It is a non-trivial task to process the so-called early-/late-arriving data, which arrive out of order. This paper proposes a two-level data staging area method to optimize ETL. The proposed method is an all-in-one solution that supports processing different types of data from operational systems, including early-/late-arriving data, and fast-/slowly-changing data. The introduced additional staging area decouples loading process from data extraction and transformation, which improves ETL flexibility and minimizes intervention to the data warehouse. This paper evaluates the proposed method empirically, which shows that it is more efficient and less intrusive than the standard ETL method.

scholarly journals Issues and Handy Solutions Addressed at Every Stage in Real Time Data Warehousing, I.E. ETL (Extraction, Transformation & Loading)

2019 ◽  
Vol 8 (5S3) ◽  
pp. 344-348
Keyword(s):  
Real Time ◽  
Data Warehouse ◽  
Data Warehousing ◽  
Time Data ◽  
Processing Load ◽  
Real Time Data ◽  
Data Source

In the standard ETL (Extract Processing Load), the data warehouse refreshment must be performed outside of peak hours. i It implies i that the i functioning and i analysis has stopped in their iall actions. iIt causes the iamount of icleanness of i data from the idata Warehouse which iisn't suggesting ithe latest i operational transections. This i issue is i known as i data i latency. The data warehousing is iemployed to ibe a iremedy for ithis iissue. It updates the idata warehouse iat a inear real-time iFashion, instantly after data found from the data source. Therefore, data i latency could i be reduced. Hence the near real time data warehousing was having issues which was not identified in traditional ETL. This paper claims to communicate the issues and accessible options at every point iin the i near real-time i data warehousing, i.e. i The i issues and Available alternatives iare based ion ia literature ireview by additional iStudy that ifocus ion near real-time data iwarehousing issue

scholarly journals A Two-Tiered Segmentation Approach for Transaction Data Warehousing

2019 ◽  
pp. 1-27
Author(s):  
Xiufeng Liu ◽  
Huan Huo ◽  
Nadeem Iftikhar ◽  
Per Sieverts Nielsen
Keyword(s):  
Data Warehouse ◽  
High Velocity ◽  
Data Warehousing ◽  
Transaction Data ◽  
Data Process ◽  
Data Store ◽  
Segmentation Approach ◽  
Operational Systems ◽  
Early Late ◽  
Central Data

Data warehousing populates data from different source systems into a central data warehouse (DW) through extraction, transformation, and loading (ETL). Massive transaction data are routinely recorded in a variety of applications such as retail commerce, bank systems, and website management. Transaction data record the timestamp and relevant reference data needed for a particular transaction record. It is a non-trivial task for a standard ETL to process transaction data with dependencies and high velocity. This chapter presents a two-tiered segmentation approach for transaction data warehousing. The approach uses a so-called two-staging ETL method to process detailed records from operational systems, followed by a dimensional data process to populate the data store with a star or snowflake schema. The proposed approach is an all-in-one solution capable of processing fast/slowly changing data and early/late-arriving data. This chapter evaluates the proposed method, and the results have validated the effectiveness of the proposed approach for processing transaction data.

scholarly journals Digital Preservation of Photo Journalism (Case Study of the Kompas Daily)

2017 ◽  
Vol 1 (2) ◽  
pp. 183
Author(s):  
Yuli Rahmawati
Keyword(s):  
Data Storage ◽  
Storage Systems ◽  
Reference Source ◽  
Digital Object ◽  
Retrieval Systems ◽  
Digital Format ◽  
The World ◽  
Information Center ◽  
Information Retrieval Systems

Newspaper provides many information from politics, economics, cultures, and sports for society. Newspaper reports important events in the world. Besides the news, there also some pictures portraying those events. So readers will easilly to see what happen visually. According to the journalistic concept, photos has comply as a reference source. Kompas daily, firstly published on 1965, and published many historical photos both paper and digital format. The digitize process is done by kompas information center, through scanning, indexing, and syncronizing. Two most important issues are standardization of metadata and integrated retrieval systems. Standarization of metadata was design based on publishing standard,  IIM and was modified with IPTC news codes. Information retrieval systems was built by inserting information about publication. This system connecting photo creating data, storage systems, and retrieval systems. For kompas daily, the availability if digital object  such as photo can trigger the innitiatives of re-publishing historical momments in thematical rubrics. Digitize will give benefits for information disemination.

Search Engines and their Impact on Data Warehouses

2011 ◽  
pp. 1727-1734
Author(s):  
Hadrian Peter
Keyword(s):  
Data Warehouse ◽  
Search Engine ◽  
Data Storage ◽  
Large Scale ◽  
Data Warehousing ◽  
New Technology ◽  
The Internet ◽  
Data Warehouses ◽  
External Data ◽  
New Generation

Over the past ten years or so data warehousing has emerged as a new technology in the database environment. “A data warehouse is a global repository that stores pre-processed queries on data which resides in multiple, possibly heterogeneous, operational or legacy sources” (Samtani et al, 2004). Data warehousing as a specialized field is continuing to grow and mature. Despite the phenomenal upgrades in terms of data storage capability there has been a flood of new streams of data entering the warehouse. During the last decade there has been an increase from 1 terabyte to 100 terabyte and, soon to be 1 petabyte, environments. Therefore, the ability to search, mine and analyze data of such immense proportions remains a significant issue even as analytical capabilities increase. The data warehouse is an environment which is readily tuned to maximize the efficiency of making useful decisions. However the advent of commercial uses of the Internet on a large scale has opened new possibilities for data capture and integration into the warehouse. While most of the data necessary for a data warehouse originates from the organization’s internal (operational) data sources, additional data is available externally that can add significant value to the data warehouse. One of the major reasons why organizations implement data warehousing is to make it easier, on a regular basis, to query and report data from multiple transaction processing systems and/or from external sources. One important source of this external data is the Internet. A few researchers (Walters, 1997; Strand & Olsson, 2004; Strand & Wangler, 2004) have investigated the possibility of incorporating external data in data warehouses, however, there is little literature detailing research in which the Internet is the source of the external data. In (Peter & Greenidge, 2005) a high-level model, the Data Warehousing Search Engine (DWSE), was presented. However, in this article we examine in some detail the issues in search engine technology that make the Internet a plausible and reliable source for external data. As John Ladley (Ladley, 2005) states “There is a new generation of Data Warehousing on the horizon that reflects maturing technology and attitudes”. Our long-term goal is to design this new generation Data Warehouse.

A Multidimensional Data Warehouse Development Methodology

2011 ◽  
pp. 188-201 ◽  
Author(s):  
Jose Maria Cavero ◽  
Carmen Costilla ◽  
Esperanza Marcos ◽  
Mario G. Piattini ◽  
Adolfo Sanchez
Keyword(s):  
Software Development ◽  
Data Warehouse ◽  
Data Warehousing ◽  
Conceptual Modeling ◽  
Physical Design ◽  
Multidimensional Data ◽  
Development Methodology ◽  
Development Cycle ◽  
Operational Systems ◽  
Analytical Processing

Data warehousing and online analytical processing (OLAP) technologies have become growing interest areas in recent years. Specific issues such as conceptual modeling, schemes translation from operational systems, physical design, etc. have been widely treated. A few methodologies covering the entire development cycle have also been proposed, but there is still not a general, accepted, complete methodology for data warehouse design. In this work we present a multidimensional data warehouse development methodology integrated within a traditional software development methodology.

The Benefits of Data Warehousing at Whirlpool

1999 ◽  
pp. 14-25 ◽  
Author(s):  
Barbara J. Haley ◽  
Hugh J. Watson ◽  
Dale L. Goodhue
Keyword(s):  
Data Warehouse ◽  
Data Warehousing ◽  
Lessons Learned ◽  
Approval Process ◽  
Critical Areas ◽  
The World ◽  
Purchasing Managers

At Whirlpool, data warehousing is providing important support in all of these critical areas (see Table 1). To illustrate, Whirlpool’s data warehouse enables quality engineers to easily track the performance of component parts. This allows the engineers to assess new components that are being field tested, to quickly detect problems with particular parts, and to identify the high and low quality suppliers. From a different perspective, suppliers can check on the performance of the parts they supply and, consequently, can manage proactively the quality provided to Whirlpool. Purchasing managers have parts information from around the world so that they can find the lowest-cost, highest quality part available on a global basis. This case study briefly describes Whirlpool, the business need that suggested a data warehouse, the approval process, and the data warehouse that was built. It describes how the data warehouse is accessed, how users are trained and supported, and the major applications and benefits. The lessons learned also are described to benefit those companies that are implementing or thinking about implementing data warehousing.

Mapping the Topics and Evolutions of Chinese Children’s Bestsellers

Libri ◽  
2020 ◽  
Vol 70 (4) ◽  
pp. 305-317
Author(s):  
Jiming Hu ◽  
Xiang Zheng ◽  
Peng Wen ◽  
Jie Xu
Keyword(s):  
Social Network ◽  
Quantitative Analysis ◽  
Network Analysis ◽  
Intellectual Structure ◽  
Structure Evolution ◽  
Children's Books ◽  
Children’S Books ◽  
Correlation Structures ◽  
The World ◽  
Data Source

AbstractChildren’s books involve a large number of topics. Research on them has been paid much attention to by both scholars and practitioners. However, the existing achievements do not focus on China, which is the fastest growing market for children’s books in the world. Studies using quantitative analysis are low in number, especially on the intellectual structure, evolution patterns, and development trends of topics of children’s bestsellers in China. Dangdang.com, the biggest Chinese online bookstore, was chosen as a data source to obtain children’s bestsellers, and topic words in them were extracted from brief introductions. With the aid of co-occurrence theory and tools of social network analysis and visualization, the distribution, correlation structures, and evolution patterns of topics were revealed and visualized. This study shows that topics of Chinese children’s bestsellers are broad and relatively concentrated, but their distribution is unbalanced. There are four distinguished topic communities (Living, Animal, World, and Child) in terms of centrality and maturity, and they all establish their individual systems and tend to be mature. The evolution of these communities tends to be stable with powerful continuity.

scholarly journals A Large-Scale COVID-19 Twitter Chatter Dataset for Open Scientific Research—An International Collaboration

Epidemiologia ◽  
2021 ◽  
Vol 2 (3) ◽  
pp. 315-324
Author(s):  
Juan M. Banda ◽  
Ramya Tekumalla ◽  
Guanyu Wang ◽  
Jingyuan Yu ◽  
Tuo Liu ◽  
...  
Keyword(s):  
Large Scale ◽  
Social Dynamics ◽  
Additional Data ◽  
Open Data ◽  
Data Sources ◽  
Research Projects ◽  
Research Groups ◽  
The World ◽  
Data Source

As the COVID-19 pandemic continues to spread worldwide, an unprecedented amount of open data is being generated for medical, genetics, and epidemiological research. The unparalleled rate at which many research groups around the world are releasing data and publications on the ongoing pandemic is allowing other scientists to learn from local experiences and data generated on the front lines of the COVID-19 pandemic. However, there is a need to integrate additional data sources that map and measure the role of social dynamics of such a unique worldwide event in biomedical, biological, and epidemiological analyses. For this purpose, we present a large-scale curated dataset of over 1.12 billion tweets, growing daily, related to COVID-19 chatter generated from 1 January 2020 to 27 June 2021 at the time of writing. This data source provides a freely available additional data source for researchers worldwide to conduct a wide and diverse number of research projects, such as epidemiological analyses, emotional and mental responses to social distancing measures, the identification of sources of misinformation, stratified measurement of sentiment towards the pandemic in near real time, among many others.

Sign in / Sign up

Export Citation Format

Share Document