Urban green economic development indicators based on spatial clustering algorithm and blockchain

2020 ◽  
pp. 1-12
Author(s):  
Xiaoguang Gao
Keyword(s):  
Development Strategy ◽  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
Large Data ◽  
Experimental Comparison ◽  
High Dimensional ◽  
Density Peak ◽  
Data Set

The unbalanced development strategy makes the regional development unbalanced. Therefore, in the development process, resources must be effectively utilized according to the level and characteristics of each region. Considering the resource and environmental constraints, this paper measures and analyzes China’s green economic efficiency and green total factor productivity. Moreover, by expounding the characteristics of high-dimensional data, this paper points out the problems of traditional clustering algorithms in high-dimensional data clustering. This paper proposes a density peak clustering algorithm based on sampling and residual squares, which is suitable for high-dimensional large data sets. The algorithm finds abnormal points and boundary points by identifying halo points, and finally determines clusters. In addition, from the experimental comparison on the data set, it can be seen that the improved algorithm is better than the DPC algorithm in both time complexity and clustering results. Finally, this article analyzes data based on actual cases. The research results show that the method proposed in this paper is effective.

scholarly journals Subspace Clustering of High-Dimensional Data: An Evolutionary Approach

2013 ◽  
Vol 2013 ◽  
pp. 1-12 ◽  
Author(s):  
Singh Vijendra ◽  
Sahoo Laxman
Keyword(s):  
Clustering Algorithm ◽  
Dimensional Space ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
Subspace Clustering ◽  
High Dimensional ◽  
Data Sets ◽  
Real World Data ◽  
Data Set ◽  
Data Points

Clustering high-dimensional data has been a major challenge due to the inherent sparsity of the points. Most existing clustering algorithms become substantially inefficient if the required similarity measure is computed between data points in the full-dimensional space. In this paper, we have presented a robust multi objective subspace clustering (MOSCL) algorithm for the challenging problem of high-dimensional clustering. The first phase of MOSCL performs subspace relevance analysis by detecting dense and sparse regions with their locations in data set. After detection of dense regions it eliminates outliers. MOSCL discovers subspaces in dense regions of data set and produces subspace clusters. In thorough experiments on synthetic and real-world data sets, we demonstrate that MOSCL for subspace clustering is superior to PROCLUS clustering algorithm. Additionally we investigate the effects of first phase for detecting dense regions on the results of subspace clustering. Our results indicate that removing outliers improves the accuracy of subspace clustering. The clustering results are validated by clustering error (CE) distance on various data sets. MOSCL can discover the clusters in all subspaces with high quality, and the efficiency of MOSCL outperforms PROCLUS.

scholarly journals Data segmentation based on the local intrinsic dimension

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Michele Allegra ◽  
Elena Facco ◽  
Francesco Denti ◽  
Alessandro Laio ◽  
Antonietta Mira
Keyword(s):  
High Dimensional Data ◽  
Large Data ◽  
Large Data Sets ◽  
High Dimensional ◽  
Data Sets ◽  
Imaging Data ◽  
Unsupervised Segmentation ◽  
Real World Data ◽  
Data Set ◽  
Intrinsic Dimension

Abstract One of the founding paradigms of machine learning is that a small number of variables is often sufficient to describe high-dimensional data. The minimum number of variables required is called the intrinsic dimension (ID) of the data. Contrary to common intuition, there are cases where the ID varies within the same data set. This fact has been highlighted in technical discussions, but seldom exploited to analyze large data sets and obtain insight into their structure. Here we develop a robust approach to discriminate regions with different local IDs and segment the points accordingly. Our approach is computationally efficient and can be proficiently used even on large data sets. We find that many real-world data sets contain regions with widely heterogeneous dimensions. These regions host points differing in core properties: folded versus unfolded configurations in a protein molecular dynamics trajectory, active versus non-active regions in brain imaging data, and firms with different financial risk in company balance sheets. A simple topological feature, the local ID, is thus sufficient to achieve an unsupervised segmentation of high-dimensional data, complementary to the one given by clustering algorithms.

scholarly journals An Improved Algorithm Based on Fast Search and Find of Density Peak Clustering for High-Dimensional Data

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Hui Du ◽  
Yiyang Ni ◽  
Zhihe Wang
Keyword(s):  
Random Forest ◽  
Clustering Algorithm ◽  
High Dimensional Data ◽  
Mean Value ◽  
High Dimensional ◽  
Importance Value ◽  
Density Peak ◽  
The Mean ◽  
Density Peak Clustering ◽  
Improved Algorithm

The find of density peak clustering algorithm (FDP) has poor performance on high-dimensional data. This problem occurs because the clustering algorithm ignores the feature selection. All features are evaluated and calculated under the same weight, without distinguishing. This will lead to the final clustering effect which cannot achieve the expected. Aiming at this problem, we propose a new method to solve it. We calculate the importance value of all features of high-dimensional data and calculate the mean value by constructing random forest. The features whose importance value is less than 10% of the mean value are removed. At this time, we extract the important features to form a new dataset. At this time, improved t-SNE is used for dimension reduction, and better performance will be obtained. This method uses t-SNE that is improved by the idea of random forest to reduce the dimension of the original data and combines with improved FDP to compose the new clustering method. Through experiments, we find that the evaluation index NMI of the improved algorithm proposed in this paper is 23% higher than that of the original FDP algorithm, and 9.1% higher than that of other clustering algorithms ( K -means, DBSCAN, and spectral clustering). It has good performance in high-dimensional datasets that are verified by experiments on UCI datasets and wireless sensor networks.

scholarly journals Massively scalable density based clustering (DBSCAN) on the HPCC systems big data platform

2021 ◽  
Vol 10 (1) ◽  
pp. 207
Author(s):  
Yatish H. R. ◽  
Shubham Milind Phal ◽  
Tanmay Sanjay Hukkeri ◽  
Lili Xu ◽  
Shobha G ◽  
...  
Keyword(s):  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Computation Time ◽  
Large Data ◽  
Single Node ◽  
Data Set ◽  
Traffic Pattern ◽  
Density Based Clustering ◽  
Data Points ◽  
Hpcc Systems

<span id="docs-internal-guid-919b015d-7fff-56da-f81d-8f032097bce2"><span>Dealing with large samples of unlabeled data is a key challenge in today’s world, especially in applications such as traffic pattern analysis and disaster management. DBSCAN, or density based spatial clustering of applications with noise, is a well-known density-based clustering algorithm. Its key strengths lie in its capability to detect outliers and handle arbitrarily shaped clusters. However, the algorithm, being fundamentally sequential in nature, proves expensive and time consuming when operated on extensively large data chunks. This paper thus presents a novel implementation of a parallel and distributed DBSCAN algorithm on the HPCC Systems platform. The algorithm seeks to fully parallelize the algorithm implementation by making use of HPCC Systems optimal distributed architecture and performing a tree-based union to merge local clusters. The proposed approach* was tested both on synthetic as well as standard datasets (MFCCs Data Set) and found to be completely accurate. Additionally, when compared against a single node setup, a significant decrease in computation time was observed with no impact to accuracy. The parallelized algorithm performed eight times better for higher number of data points and takes exponentially lesser time as the number of data points increases.</span></span>

scholarly journals Visualization of High-Dimensional Data by Pairwise Fusion Matrices Using t-SNE

Symmetry ◽  
2019 ◽  
Vol 11 (1) ◽  
pp. 107 ◽  
Author(s):  
Mujtaba Husnain ◽  
Malik Missen ◽  
Shahzad Mumtaz ◽  
Muhammad Luqman ◽  
Mickaël Coustaty ◽  
...  
Keyword(s):  
Local Structure ◽  
High Dimensional Data ◽  
Three Dimensional ◽  
Principal Component ◽  
Large Data ◽  
High Dimensional ◽  
Data Set ◽  
Novel Approach ◽  
Critical Issues ◽  
Low Dimensional

We applied t-distributed stochastic neighbor embedding (t-SNE) to visualize Urdu handwritten numerals (or digits). The data set used consists of 28 × 28 images of handwritten Urdu numerals. The data set was created by inviting authors from different categories of native Urdu speakers. One of the challenging and critical issues for the correct visualization of Urdu numerals is shape similarity between some of the digits. This issue was resolved using t-SNE, by exploiting local and global structures of the large data set at different scales. The global structure consists of geometrical features and local structure is the pixel-based information for each class of Urdu digits. We introduce a novel approach that allows the fusion of these two independent spaces using Euclidean pairwise distances in a highly organized and principled way. The fusion matrix embedded with t-SNE helps to locate each data point in a two (or three-) dimensional map in a very different way. Furthermore, our proposed approach focuses on preserving the local structure of the high-dimensional data while mapping to a low-dimensional plane. The visualizations produced by t-SNE outperformed other classical techniques like principal component analysis (PCA) and auto-encoders (AE) on our handwritten Urdu numeral dataset.

scholarly journals Cross Breed Clustering Algorithm for High Dimensional Data

2019 ◽  
Vol 9 (1) ◽  
pp. 5049-5052
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Data Clustering ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
High Dimensional ◽  
Growing Domain ◽  
Present World

Clustering plays a major role in machine learning and also in data mining. Deep learning is fast growing domain in present world. Improving the quality of the clustering results by adopting the deep learning algorithms. Many clustering algorithm process various datasets to get the better results. But for the high dimensional data clustering is still an issue to process and get the quality clustering results with the existing clustering algorithms. In this paper, the cross breed clustering algorithm for high dimensional data is utilized. Various datasets are used to get the results.

scholarly journals Visualizing balances of compositional data: A new alternative to balance dendrograms

F1000Research ◽  
2018 ◽  
Vol 7 ◽  
pp. 1278 ◽  
Author(s):  
Thomas P. Quinn
Keyword(s):  
Data Analysis ◽  
Compositional Data ◽  
High Dimensional Data ◽  
Large Data ◽  
High Dimensional ◽  
Compositional Data Analysis ◽  
Data Set ◽  
R Programming Language ◽  
Common Scale ◽  
R Programming

Balances have become a cornerstone of compositional data analysis. However, conceptualizing balances is difficult, especially for high-dimensional data. Most often, investigators visualize balances with the balance dendrogram, but this technique is not necessarily intuitive and does not scale well for large data. This manuscript introduces the 'balance' package for the R programming language. This package visualizes balances of compositional data using an alternative to the balance dendrogram. This alternative contains the same information coded by the balance dendrogram, but projects data on a common scale that facilitates direct comparisons and accommodates high-dimensional data. By stripping the branches from the tree, 'balance' can cleanly visualize any subset of balances without disrupting the interpretation of the remaining balances. As an example, this package is applied to a publicly available meta-genomics data set measuring the relative abundance of 500 microbe taxa.

scholarly journals Density Peaks Clustering Based on Feature Reduction and Quasi-Monte Carlo

2022 ◽  
Vol 2022 ◽  
pp. 1-17
Author(s):  
Zhihui Hu ◽  
Xiaoran Wei ◽  
Xiaoxu Han ◽  
Guang Kou ◽  
Haoyu Zhang ◽  
...  
Keyword(s):  
Clustering Algorithm ◽  
High Dimensional Data ◽  
Original Data ◽  
Feature Reduction ◽  
High Dimensional ◽  
Data Sampling ◽  
Data Set ◽  
Quasi Monte Carlo ◽  
Density Peaks ◽  
Density Peaks Clustering

Density peaks clustering (DPC) is a well-known density-based clustering algorithm that can deal with nonspherical clusters well. However, DPC has high computational complexity and space complexity in calculating local density ρ and distance δ , which makes it suitable only for small-scale data sets. In addition, for clustering high-dimensional data, the performance of DPC still needs to be improved. High-dimensional data not only make the data distribution more complex but also lead to more computational overheads. To address the above issues, we propose an improved density peaks clustering algorithm, which combines feature reduction and data sampling strategy. Specifically, features of the high-dimensional data are automatically extracted by principal component analysis (PCA), auto-encoder (AE), and t-distributed stochastic neighbor embedding (t-SNE). Next, in order to reduce the computational overhead, we propose a novel data sampling method for the low-dimensional feature data. Firstly, the data distribution in the low-dimensional feature space is estimated by the Quasi-Monte Carlo (QMC) sequence with low-discrepancy characteristics. Then, the representative QMC points are selected according to their cell densities. Next, the selected QMC points are used to calculate ρ and δ instead of the original data points. In general, the number of the selected QMC points is much smaller than that of the initial data set. Finally, a two-stage classification strategy based on the QMC points clustering results is proposed to classify the original data set. Compared with current works, our proposed algorithm can reduce the computational complexity from O n 2 to O N n , where N denotes the number of selected QMC points and n is the size of original data set, typically N ≪ n . Experimental results demonstrate that the proposed algorithm can effectively reduce the computational overhead and improve the model performance.

scholarly journals An improved Kohonen self-organizing map clustering algorithm for high-dimensional data sets

2021 ◽  
Vol 24 (1) ◽  
pp. 600
Author(s):  
Momotaz Begum ◽  
Bimal Chandra Das ◽  
Md. Zakir Hossain ◽  
Antu Saha ◽  
Khaleda Akther Papry
Keyword(s):  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
High Dimensional Data ◽  
Predictive Performance ◽  
High Dimensional ◽  
Data Sets ◽  
Self Organizing Map ◽  
Distance Measurements ◽  
Cancer Data ◽  
Self Organizing

<p>Manipulating high-dimensional data is a major research challenge in the field of computer science in recent years. To classify this data, a lot of clustering algorithms have already been proposed. Kohonen self-organizing map (KSOM) is one of them. However, this algorithm has some drawbacks like overlapping clusters and non-linear separability problems. Therefore, in this paper, we propose an improved KSOM (I-KSOM) to reduce the problems that measures distances among objects using EISEN Cosine correlation formula. So far as we know, no previous work has used EISEN Cosine correlation distance measurements to classify high-dimensional data sets. To the robustness of the proposed KSOM, we carry out the experiments on several popular datasets like Iris, Seeds, Glass, Vertebral column, and Wisconsin breast cancer data sets. Our proposed algorithm shows better result compared to the existing original KSOM and another modified KSOM in terms of predictive performance with topographic and quantization error.</p>

Sign in / Sign up

Export Citation Format

Share Document