Double Deep Autoencoder for Heterogeneous Distributed Clustering

Given the issues relating to big data and privacy-preserving challenges, distributed data mining (DDM) has received much attention recently. Here, we focus on the clustering problem of distributed environments. Several distributed clustering algorithms have been proposed to solve this problem, however, previous studies have mainly considered homogeneous data. In this paper, we develop a double deep autoencoder structure for clustering in distributed and heterogeneous datasets. Three datasets are used to demonstrate the proposed algorithms, and show their usefulness according to the consistent accuracy index.

Download Full-text

Distributed Data Mining for Multiple Sourced Heterogeneous Datasets: A Survey

DEStech Transactions on Computer Science and Engineering ◽

10.12783/dtcse/cmsam2018/26563 ◽

2018 ◽

Author(s):

Xing-ying LI ◽

Shan-zi LI ◽

Yi-xuan WU ◽

Ai-jia HE ◽

Xiao-ya HUANG ◽

...

Keyword(s):

Data Mining ◽

Distributed Data Mining ◽

Distributed Data ◽

Heterogeneous Datasets

Download Full-text

Distributed Data Mining

Encyclopedia of Data Warehousing and Mining, Second Edition ◽

10.4018/978-1-60566-010-3.ch110 ◽

2011 ◽

pp. 709-715 ◽

Cited By ~ 4

Author(s):

Grigorios Tsoumakas

Keyword(s):

Data Mining ◽

Data Warehouse ◽

Computational Cost ◽

Distributed Data Mining ◽

The Internet ◽

Distributed Data ◽

Distributed Environments ◽

Sensitive Data ◽

Huge Data ◽

Central Storage

The continuous developments in information and communication technology have recently led to the appearance of distributed computing environments, which comprise several, and different sources of large volumes of data and several computing units. The most prominent example of a distributed environment is the Internet, where increasingly more databases and data streams appear that deal with several areas, such as meteorology, oceanography, economy and others. In addition the Internet constitutes the communication medium for geographically distributed information systems, as for example the earth observing system of NASA (eos. gsfc.nasa.gov). Other examples of distributed environments that have been developed in the last few years are sensor networks for process monitoring and grids where a large number of computing and storage units are interconnected over a high-speed network. The application of the classical knowledge discovery process in distributed environments requires the collection of distributed data in a data warehouse for central processing. However, this is usually either ineffective or infeasible for the following reasons: (1) Storage cost. It is obvious that the requirements of a central storage system are enormous. A classical example concerns data from the astronomy science, and especially images from earth and space telescopes. The size of such databases is reaching the scale of exabytes (1018 bytes) and is increasing at a high pace. The central storage of the data of all telescopes of the planet would require a huge data warehouse of enormous cost. (2) Communication cost. The transfer of huge data volumes over network might take extremely much time and also require an unbearable financial cost. Even a small volume of data might create problems in wireless network environments with limited bandwidth. Note also that communication may be a continuous overhead, as distributed databases are not always constant and unchangeable. On the contrary, it is common to have databases that are frequently updated with new data or data streams that constantly record information (e.g remote sensing, sports statistics, etc.). (3) Computational cost. The computational cost of mining a central data warehouse is much bigger than the sum of the cost of analyzing smaller parts of the data that could also be done in parallel. In a grid, for example, it is easier to gather the data at a central location. However, a distributed mining approach would make a better exploitation of the available resources. (4) Private and sensitive data. There are many popular data mining applications that deal with sensitive data, such as people’s medical and financial records. The central collection of such data is not desirable as it puts their privacy into risk. In certain cases (e.g. banking, telecommunication) the data might belong to different, perhaps competing, organizations that want to exchange knowledge without the exchange of raw private data. This article is concerned with Distributed Data Mining algorithms, methods and systems that deal with the above issues in order to discover knowledge from distributed data in an effective and efficient way.

Download Full-text

A Survey of Data Mining Activities in Distributed Systems

Asian Journal of Research in Computer Science ◽

10.9734/ajrcos/2021/v11i430267 ◽

2021 ◽

pp. 1-18

Author(s):

Waleed A. Mohammad ◽

Hajar Maseeh Yasin ◽

Azar Abid Salih ◽

Adel AL-Zebari ◽

Naaman Omar ◽

...

Keyword(s):

Data Mining ◽

Distributed Systems ◽

Distributed Data Mining ◽

Distributed Data ◽

Sequence Mining ◽

Distributed Clustering ◽

Usable Information ◽

Single Location ◽

Processing Ability ◽

Mining Methods

Distributed systems, which may be utilized to do computations, are being developed as a result of the fast growth of sharing resources. Data mining, which has a huge range of real applications, provides significant techniques for extracting meaningful and usable information from massive amounts of data. Traditional data mining methods, on the other hand, suppose that the data is gathered centrally, stored in memory, and is static. Managing massive amounts of data and processing them with limited resources is difficult. Large volumes of data, for instance, are swiftly generated and stored in many locations. This becomes increasingly costly to centralize them at a single location. Furthermore, traditional data mining methods typically have several issues and limitations, such as memory restrictions, limited processing ability, and insufficient hard drive space, among others. To overcome the following issues, distributed data mining's have emerged as a beneficial option in several applications According to several authors, this research provides a study of state-of-the-art distributed data mining methods, such as distributed common item-set mining, distributed frequent sequence mining, technical difficulties with distributed systems, distributed clustering, as well as privacy-protection distributed data mining. Furthermore, each work is evaluated and compared to the others.

Download Full-text