Rapid Prototyping of Hierarchical Agglomerative Clustering Algorithms for Distributed Systems

Correctly defining and grouping electrical feeders is of great importance for electrical system operators. In this paper, we compare two different clustering techniques, K-means and hierarchical agglomerative clustering, applied to real data from the east region of Paraguay. The raw data were pre-processed, resulting in four data sets, namely, (i) a weekly feeder demand, (ii) a monthly feeder demand, (iii) a statistical feature set extracted from the original data and (iv) a seasonal and daily consumption feature set obtained considering the characteristics of the Paraguayan load curve. Considering the four data sets, two clustering algorithms, two distance metrics and five linkage criteria a total of 36 models with the Silhouette, Davies–Bouldin and Calinski–Harabasz index scores was assessed. The K-means algorithms with the seasonal feature data sets showed the best performance considering the Silhouette, Calinski–Harabasz and Davies–Bouldin validation index scores with a configuration of six clusters.

Download Full-text

Efficient Hierarchical Agglomerative Clustering Algorithms on GPU Using Data Partitioning

2011 12th International Conference on Parallel and Distributed Computing, Applications and Technologies ◽

10.1109/pdcat.2011.38 ◽

2011 ◽

Cited By ~ 2

Author(s):

S.A. Arul Shalom ◽

Manoranjan Dash

Keyword(s):

Clustering Algorithms ◽

Data Partitioning ◽

Agglomerative Clustering ◽

Hierarchical Agglomerative Clustering ◽

Using Data

Download Full-text

Clustering of Micro-Messages Using Similarity Upper Approximation

International Journal of Uncertainty Fuzziness and Knowledge-Based Systems ◽

10.1142/s0218488517500039 ◽

2017 ◽

Vol 25 (01) ◽

pp. 53-79 ◽

Cited By ~ 1

Author(s):

Mukul Gupta ◽

Pradeep Kumar ◽

Bharat Bhasker

Keyword(s):

Clustering Algorithm ◽

State Of The Art ◽

Clustering Algorithms ◽

Mining Operation ◽

Agglomerative Clustering ◽

Upper Approximation ◽

Affinity Propagation Clustering ◽

Hierarchical Agglomerative Clustering ◽

Text Content ◽

Extract Information

Microblogging platforms like Twitter, Tumblr and Plurk have radically changed our lives. The presence of millions of people has made these platforms a preferred channel for communication. A large amount of User Generated Content, on these platforms, has attracted researchers and practitioners to mine and extract information nuggets. For information extraction, clustering is an important and widely used mining operation. This paper addresses the issue of clustering of micro-messages and corresponding users based on the text content of micro-messages that reflect their primitive interest. In this paper, we performed modification of the Similarity Upper Approximation based clustering algorithm for clustering of micro-messages. We compared the performance of the modified Similarity Upper Approximation based clustering algorithm with state-of-the-art clustering algorithms such as Partition Around Medoids, Hierarchical Agglomerative Clustering, Affinity Propagation Clustering and DBSCAN. Experiments were performed on micro-messages collected from Twitter. Experimental results show the effectiveness of the proposed algorithm.

Download Full-text

Clustering of a Health Dataset Using Diagnosis Co-Occurrences

Applied Sciences ◽

10.3390/app11052373 ◽

2021 ◽

Vol 11 (5) ◽

pp. 2373

Author(s):

Adrien Wartelle ◽

Farah Mourad-Chehade ◽

Farouk Yalaoui ◽

Jan Chrusciel ◽

David Laplanche ◽

...

Keyword(s):

Quality Indicators ◽

Clustering Algorithm ◽

Clustering Algorithms ◽

Agglomerative Clustering ◽

Cluster Membership ◽

Healthcare Needs ◽

Similar Patient ◽

Hierarchical Agglomerative Clustering ◽

Patient Profiles ◽

Cluster Compactness

Assessing the health profiles of populations is a crucial task to create a coherent healthcare offer. Emergency Departments (EDs) are at the core of the healthcare system and could benefit from this evaluation via an improved understanding of the healthcare needs of their population. This paper proposes a novel hierarchical agglomerative clustering algorithm based on multimorbidity analysis. The proposed approach constructs the clustering dendrogram by introducing new quality indicators based on the relative risk of co-occurrences of patient diagnoses. This algorithm enables the detection of multimorbidity patterns by merging similar patient profiles according to their common diagnoses. The multimorbidity approach has been applied to the data of the largest ED of the Aube Department (Eastern France) to cluster its patient visits. Among the 120,718 visits identified during a 24-month period, 16 clusters were identified, accounting for 94.8% of the visits, with the five most prevalent clusters representing 63.0% of them. The new quality indicators show a coherent and good clustering solution with a cluster membership of 1.81 based on a cluster compactness of 1.40 and a cluster separation of 0.77. Compared to the literature, the proposed approach is appropriate for the discovery of multimorbidity patterns and could help to develop better clustering algorithms for more diverse healthcare datasets.

Download Full-text

A Hierarchical Clustering Algorithm Based on Silhouette Index for Cancer Subtype Discovery from Omics Data

10.1101/309716 ◽

2018 ◽

Cited By ~ 1

Author(s):

N. Nidheesh ◽

K.A. Abdul Nazeer ◽

P.M. Ameer

Keyword(s):

Clustering Algorithm ◽

Clustering Algorithms ◽

Agglomerative Clustering ◽

Cluster Validity ◽

Cluster Validity Index ◽

Number Of Clusters ◽

Silhouette Index ◽

Cancer Subtype ◽

Hierarchical Agglomerative Clustering ◽

Hierarchical Clustering Algorithm

AbstractCancer subtype discovery fromomicsdata requires techniques to estimate the number of natural clusters in the data. Automatically estimating the number of clusters has been a challenging problem in Machine Learning. Using clustering algorithms together with internal cluster validity indexes have been a popular method of estimating the number of clusters in biomolecular data. We propose a Hierarchical Agglomerative Clustering algorithm, namedSilHAC, which can automatically estimate the number of natural clusters and can find the associated clustering solution.SilHACis parameterless. We also present two hybrids ofSilHACwithSpectral ClusteringandK-Meansrespectively as components.SilHACand the hybrids could find reasonable estimates for the number of clusters and the associated clustering solution when applied to a collection of cancer gene expression datasets. The proposed methods are better alternatives to the ‘clustering algorithm - internal cluster validity index’ pipelines for estimating the number of natural clusters.

Download Full-text

Application of Hierarchical Agglomerative Clustering (HAC) for Systemic Classification of Pop-Up Housing (PUH) Environments

Applied Sciences ◽

10.3390/app112311122 ◽

2021 ◽

Vol 11 (23) ◽

pp. 11122

Author(s):

Thomas Märzinger ◽

Jan Kotík ◽

Christoph Pfeifer

Keyword(s):

Urban Planning ◽

Hierarchical Clustering ◽

Clustering Algorithms ◽

Field Studies ◽

Assessment Tools ◽

Agglomerative Clustering ◽

Research Project ◽

Minkowski Metric ◽

Hierarchical Agglomerative Clustering

This paper is the result of the first-phase, inter-disciplinary work of a multi-disciplinary research project (“Urban pop-up housing environments and their potential as local innovation systems”) consisting of energy engineers and waste managers, landscape architects and spatial planners, innovation researchers and technology assessors. The project is aiming at globally analyzing and describing existing pop-up housings (PUH), developing modeling and assessment tools for sustainable, energy-efficient and socially innovative temporary housing solutions (THS), especially for sustainable and resilient urban structures. The present paper presents an effective application of hierarchical agglomerative clustering (HAC) for analyses of large datasets typically derived from field studies. As can be shown, the method, although well-known and successfully established in (soft) computing science, can also be used very constructively as a potential urban planning tool. The main aim of the underlying multi-disciplinary research project was to deeply analyze and structure THS and PUE. Multiple aspects are to be considered when it comes to the characterization and classification of such environments. A thorough (global) web survey of PUH and analysis of scientific literature concerning descriptive work of PUH and THS has been performed. Moreover, out of several tested different approaches and methods for classifying PUH, hierarchical clustering algorithms functioned well when properly selected metrics and cut-off criteria were applied. To be specific, the ‘Minkowski’-metric and the ‘Calinski-Harabasz’-criteria, as clustering indices, have shown the best overall results in clustering the inhomogeneous data concerning PUH. Several additional algorithms/functions derived from the field of hierarchical clustering have also been tested to exploit their potential in interpreting and graphically analyzing particular structures and dependencies in the resulting clusters. Hereby, (math.) the significance ‘S’ and (math.) proportion ‘P’ have been concluded to yield the best interpretable and comprehensible results when it comes to analyzing the given set (objects n = 85) of researched PUH-objects together with their properties (n > 190). The resulting easily readable graphs clearly demonstrate the applicability and usability of hierarchical clustering- and their derivative algorithms for scientifically profound building classification tasks in Urban Planning by effectively managing huge inhomogeneous building datasets.

Download Full-text

Robustness of fish assemblages derived from three hierarchical agglomerative clustering algorithms performed on Icelandic groundfish survey data

ICES Journal of Marine Science ◽

10.1093/icesjms/fsq144 ◽

2010 ◽

Vol 68 (1) ◽

pp. 189-200 ◽

Cited By ~ 12

Author(s):

Warsha Singh ◽

Einar Hjorleifsson ◽

Gunnar Stefansson

Keyword(s):

Survey Data ◽

Fish Assemblages ◽

Clustering Algorithms ◽

Agglomerative Clustering ◽

Cluster Validity Indices ◽

Validity Indices ◽

Hierarchical Agglomerative Clustering ◽

Species Area ◽

Species Groups ◽

The Stability

Abstract Singh, W., Hjorleifsson, E., and Stefansson, G. 2011. Robustness of fish assemblages derived from three hierarchical agglomerative clustering algorithms performed on Icelandic groundfish survey data. – ICES Journal of Marine Science, 68: 189–200. Heatmaps are used to identify species–area assemblages based on Icelandic groundfish survey data. Hierarchical agglomerative clustering algorithms are widely applied for species assemblage studies and form the basis for heatmaps. First, the robustness of fish assemblages derived from three clustering algorithms, Average, Complete, and Ward's linkage, was examined. For statistical reliability, the use of a bootstrap resampling technique to generate the confidence values for the clusters is emphasized. Two cluster validity indices were used to measure the efficiency and the quality of the clusters. To examine the stability of the results, clustering was carried out across different sample sizes and levels of data smoothing. Second, cluster analysis was carried out using a different combination of data standardization and dissimilarity measure. Ward's linkage gave the most robust fish assemblages for both modes of data analyses. Four fish assemblages were identified which could be characterized according to the depth and the geographic distribution. This algorithm was then used to generate a heatmap to determine the species–area relationships. Specific areas were characterized by the identified species groups.

Download Full-text

Clustering Techniques for Secondary Substations Siting

Energies ◽

10.3390/en14041028 ◽

2021 ◽

Vol 14 (4) ◽

pp. 1028

Author(s):

Silvia Corigliano ◽

Federico Rosato ◽

Carla Ortiz Dominguez ◽

Marco Merlo

Keyword(s):

Rural Areas ◽

Urban Areas ◽

Universal Access ◽

Distribution Networks ◽

Industrialized Countries ◽

Agglomerative Clustering ◽

Clustering Techniques ◽

Hierarchical Agglomerative Clustering ◽

Efficient Planning ◽

Target Set

The scientific community is active in developing new models and methods to help reach the ambitious target set by UN SDGs7: universal access to electricity by 2030. Efficient planning of distribution networks is a complex and multivariate task, which is usually split into multiple subproblems to reduce the number of variables. The present work addresses the problem of optimal secondary substation siting, by means of different clustering techniques. In contrast with the majority of approaches found in the literature, which are devoted to the planning of MV grids in already electrified urban areas, this work focuses on greenfield planning in rural areas. K-means algorithm, hierarchical agglomerative clustering, and a method based on optimal weighted tree partitioning are adapted to the problem and run on two real case studies, with different population densities. The algorithms are compared in terms of different indicators useful to assess the feasibility of the solutions found. The algorithms have proven to be effective in addressing some of the crucial aspects of substations siting and to constitute relevant improvements to the classic K-means approach found in the literature. However, it is found that it is very challenging to conjugate an acceptable geographical span of the area served by a single substation with a substation power high enough to justify the installation when the load density is very low. In other words, well known standards adopted in industrialized countries do not fit with developing countries’ requirements.

Download Full-text

Identifying organ dysfunction trajectory-based subphenotypes in critically ill patients with COVID-19

Scientific Reports ◽

10.1038/s41598-021-95431-7 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Chang Su ◽

Zhenxing Xu ◽

Katherine Hoffman ◽

Parag Goyal ◽

Monika M. Safford ◽

...

Keyword(s):

New York ◽

Respiratory Failure ◽

Sofa Score ◽

Severity Of Illness ◽

Agglomerative Clustering ◽

Baseline Severity ◽

Organ Systems ◽

Hierarchical Agglomerative Clustering ◽

Dynamic Time ◽

Post Intubation

AbstractCOVID-19-associated respiratory failure offers the unprecedented opportunity to evaluate the differential host response to a uniform pathogenic insult. Understanding whether there are distinct subphenotypes of severe COVID-19 may offer insight into its pathophysiology. Sequential Organ Failure Assessment (SOFA) score is an objective and comprehensive measurement that measures dysfunction severity of six organ systems, i.e., cardiovascular, central nervous system, coagulation, liver, renal, and respiration. Our aim was to identify and characterize distinct subphenotypes of COVID-19 critical illness defined by the post-intubation trajectory of SOFA score. Intubated COVID-19 patients at two hospitals in New York city were leveraged as development and validation cohorts. Patients were grouped into mild, intermediate, and severe strata by their baseline post-intubation SOFA. Hierarchical agglomerative clustering was performed within each stratum to detect subphenotypes based on similarities amongst SOFA score trajectories evaluated by Dynamic Time Warping. Distinct worsening and recovering subphenotypes were identified within each stratum, which had distinct 7-day post-intubation SOFA progression trends. Patients in the worsening suphenotypes had a higher mortality than those in the recovering subphenotypes within each stratum (mild stratum, 29.7% vs. 10.3%, p = 0.033; intermediate stratum, 29.3% vs. 8.0%, p = 0.002; severe stratum, 53.7% vs. 22.2%, p < 0.001). Pathophysiologic biomarkers associated with progression were distinct at each stratum, including findings suggestive of inflammation in low baseline severity of illness versus hemophagocytic lymphohistiocytosis in higher baseline severity of illness. The findings suggest that there are clear worsening and recovering subphenotypes of COVID-19 respiratory failure after intubation, which are more predictive of outcomes than baseline severity of illness. Distinct progression biomarkers at differential baseline severity of illness suggests a heterogeneous pathobiology in the progression of COVID-19 respiratory failure.

Download Full-text

Hierarchical Agglomerative Clustering

Encyclopedia of Systems Biology ◽

10.1007/978-1-4419-9863-7_1371 ◽

2013 ◽

pp. 886-887 ◽

Cited By ~ 28

Author(s):

Marie Lisandra Zepeda-Mendoza ◽

Osbaldo Resendis-Antonio

Keyword(s):

Agglomerative Clustering ◽

Hierarchical Agglomerative Clustering

Download Full-text