scholarly journals ASSOCIATION RULES MINING IN BIG DATA

2018 ◽  
pp. 25-32 ◽  
Author(s):  
Nataliya Shakhovska ◽  
Roman Kaminskyy ◽  
Eugen Zasoba ◽  
Mykola Tsiutsiura
Keyword(s):  
Big Data ◽  
Association Rules ◽  
Distributed Databases ◽  
Data Sources ◽  
Data Types ◽  
Association Rules Mining ◽  
Mining Algorithm ◽  
Data Definition ◽  
Subject Areas ◽  
Asymptotic Complexity

The paper proposes a method for Big data analyzing in the presence of different data sources and different methods of processing these data. The Big data definition is given, the main problems of data mining process are described. The concept of association rules is introduced and the method of association rules searching for working with Big Data is modified. The method of finding dependencies is developed, efficiency and possibility of its parallelization are determined. The developed algorithm makes it possible to assert that the task of detecting association dependencies in distributed databases belongs to the class of P-tasks. The algorithm for finding association dependencies is well-solved with MapReduce. The low asymptotic complexity of the developed association rules mining algorithm and a wide set of data types supported for analysis allow to apply the proposed algorithm in practically all subject areas working with association dependencies in the data domain.

Big Data Architecture Components

2018 ◽  
pp. 10-31
Keyword(s):  
Big Data ◽  
Missing Values ◽  
Noise Removal ◽  
Data Sources ◽  
Data Types ◽  
Layer Analysis ◽  
External Sources ◽  
Internal Sources ◽  
High Level

The previous chapter overviewed big data including its types, sources, analytic techniques, and applications. This chapter briefly discusses the architecture components dealing with the huge volume of data. The complexity of big data types defines a logical architecture with layers and high-level components to obtain a big data solution that includes data sources with the relation to atomic patterns. The dimensions of the approach include volume, variety, velocity, veracity, and governance. The diverse layers of the architecture are big data sources, data massaging and store layer, analysis layer, and consumption layer. Big data sources are data collected from various sources to perform analytics by data scientists. Data can be from internal and external sources. Internal sources comprise transactional data, device sensors, business documents, internal files, etc. External sources can be from social network profiles, geographical data, data stores, etc. Data massage is the process of extracting data by preprocessing like removal of missing values, dimensionality reduction, and noise removal to attain a useful format to be stored. Analysis layer is to provide insight with preferred analytics techniques and tools. The analytics methods, issues to be considered, requirements, and tools are widely mentioned. Consumption layer being the result of business insight can be outsourced to sources like retail marketing, public sector, financial body, and media. Finally, a case study of architectural drivers is applied on a retail industry application and its challenges and usecases are discussed.

Sign in / Sign up

Export Citation Format

Share Document