GMM with parameters initialization based on SVD for network threat detection

2021 ◽  
Vol 40 (1) ◽  
pp. 477-490
Author(s):  
Yanping Xu ◽  
Tingcong Ye ◽  
Xin Wang ◽  
Yuping Lai ◽  
Jian Qiu ◽  
...  
Keyword(s):  
Gaussian Mixture Models ◽  
Singular Values ◽  
Gaussian Mixture ◽  
Singular Value ◽  
Optimal Parameters ◽  
Clustering Methods ◽  
Data Set ◽  
Detection Model ◽  
Clustering Model ◽  
Threat Behavior

In the field of security, the data labels are unknown or the labels are too expensive to label, so that clustering methods are used to detect the threat behavior contained in the big data. The most widely used probabilistic clustering model is Gaussian Mixture Models(GMM), which is flexible and powerful to apply prior knowledge for modelling the uncertainty of the data. Therefore, in this paper, we use GMM to build the threat behavior detection model. Commonly, Expectation Maximization (EM) and Variational Inference (VI) are used to estimate the optimal parameters of GMM. However, both EM and VI are quite sensitive to the initial values of the parameters. Therefore, we propose to use Singular Value Decomposition (SVD) to initialize the parameters. Firstly, SVD is used to factorize the data set matrix to get the singular value matrix and singular matrices. Then we calculate the number of the components of GMM by the first two singular values in the singular value matrix and the dimension of the data. Next, other parameters of GMM, such as the mixing coefficients, the mean and the covariance, are calculated based on the number of the components. After that, the initialization values of the parameters are input into EM and VI to estimate the optimal parameters of GMM. The experiment results indicate that our proposed method performs well on the parameters initialization of GMM clustering using EM and VI for estimating parameters.

scholarly journals Comparison of dimensionality reduction and clustering methods for SARS-CoV-2 genome

2021 ◽  
Vol 10 (4) ◽  
pp. 2170-2180
Author(s):  
Untari N. Wisesty ◽  
Tati Rajab Mengko
Keyword(s):  
Dimensionality Reduction ◽  
Dimensional Reduction ◽  
Clustering Algorithm ◽  
Sequence Data ◽  
Clustering Algorithms ◽  
Gaussian Mixture Models ◽  
Reduction Process ◽  
Principal Component ◽  
Gaussian Mixture ◽  
Clustering Methods

This paper aims to conduct an analysis of the SARS-CoV-2 genome variation was carried out by comparing the results of genome clustering using several clustering algorithms and distribution of sequence in each cluster. The clustering algorithms used are K-means, Gaussian mixture models, agglomerative hierarchical clustering, mean-shift clustering, and DBSCAN. However, the clustering algorithm has a weakness in grouping data that has very high dimensions such as genome data, so that a dimensional reduction process is needed. In this research, dimensionality reduction was carried out using principal component analysis (PCA) and autoencoder method with three models that produce 2, 10, and 50 features. The main contributions achieved were the dimensional reduction and clustering scheme of SARS-CoV-2 sequence data and the performance analysis of each experiment on each scheme and hyper parameters for each method. Based on the results of experiments conducted, PCA and DBSCAN algorithm achieve the highest silhouette score of 0.8770 with three clusters when using two features. However, dimensionality reduction using autoencoder need more iterations to converge. On the testing process with Indonesian sequence data, more than half of them enter one cluster and the rest are distributed in the other two clusters.

Modeling with Gaussian mixture regression for lactationmilk yield in Anatolian buffaloes

2016 ◽  
Author(s):  
Abdullah Yesilova ◽  
Ayhan Yilmaz ◽  
Gazel Ser ◽  
Baris Kaki
Keyword(s):  
Milk Yield ◽  
Environmental Effects ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Lactation Period ◽  
Multivariate Statistical ◽  
Yield Data ◽  
Data Set ◽  
Mixture Regression ◽  
Lactation Duration

The purpose of this study was to classify Anatolian buffalo using Gaussian mixture regression model according to discrete and continuous environmental effects. Gaussian mixture model performs separately regression analysis both within and between groups. This is an important property of Gaussian mixture models which makes it different from other multivariate statistical methods. The data were obtained from 1455 Anatolian buffalo lactation milk yield records reared in seven different locations in Bitlis province, Turkey. Age of dam, lactation duration and locations were considered as environmental effects on lactation milk yield. Data set was divided into three homogenous subgroups with respect to AIC and BIC in the Gaussian mixture regression, based on environmental effects on lactation milk yield. Estimated mean for lactation milk yields and mixing probabilities for the first, second and third subgroups were determined as 1494.33 kg (16.9%), 540.33 kg (45.2%) and 847.61 (37.9%), respectively. The numbers of buffalo in each subgroup according to mixing probability were obtained as 159, 756, and 540 for the first, second, and third groups, respectively. The effects of lactation period, age of dam and villages were found statistically significant on lactation milk yield in subgroup 1 that was highest mean for lactation milk yield (p less than 0.01). In conclusion, results showed that Gaussian mixture regression was an important tool for classifying quantitative traits considering environmental effects in animal breeding.

scholarly journals The Information Bottleneck and Geometric Clustering

2019 ◽  
Vol 31 (3) ◽  
pp. 596-612 ◽  
Author(s):  
DJ Strouse ◽  
David J. Schwab
Keyword(s):  
Clustering Algorithms ◽  
Gaussian Mixture Models ◽  
Selection Procedure ◽  
Gaussian Mixture ◽  
Model Parameters ◽  
Data Set ◽  
Information Bottleneck ◽  
Geometric Clustering ◽  
Data Points ◽  
Intuitive Method

The information bottleneck (IB) approach to clustering takes a joint distribution [Formula: see text] and maps the data [Formula: see text] to cluster labels [Formula: see text], which retain maximal information about [Formula: see text] (Tishby, Pereira, & Bialek, 1999 ). This objective results in an algorithm that clusters data points based on the similarity of their conditional distributions [Formula: see text]. This is in contrast to classic geometric clustering algorithms such as [Formula: see text]-means and gaussian mixture models (GMMs), which take a set of observed data points [Formula: see text] and cluster them based on their geometric (typically Euclidean) distance from one another. Here, we show how to use the deterministic information bottleneck (DIB) (Strouse & Schwab, 2017 ), a variant of IB, to perform geometric clustering by choosing cluster labels that preserve information about data point location on a smoothed data set. We also introduce a novel intuitive method to choose the number of clusters via kinks in the information curve. We apply this approach to a variety of simple clustering problems, showing that DIB with our model selection procedure recovers the generative cluster labels. We also show that, in particular limits of our model parameters, clustering with DIB and IB is equivalent to [Formula: see text]-means and EM fitting of a GMM with hard and soft assignments, respectively. Thus, clustering with (D)IB generalizes and provides an information-theoretic perspective on these classic algorithms.

scholarly journals Rapid and Accurate Analysis of an X-Ray Fluorescence Microscopy Data Set through Gaussian Mixture-Based Soft Clustering Methods

2013 ◽  
Vol 19 (5) ◽  
pp. 1281-1289 ◽  
Author(s):  
Jesse Ward ◽  
Rebecca Marvin ◽  
Thomas O'Halloran ◽  
Chris Jacobsen ◽  
Stefan Vogt
Keyword(s):  
Simultaneous Detection ◽  
Large Data ◽  
Gaussian Mixture ◽  
Subcellular Fractionation ◽  
Food Vacuole ◽  
Clustering Methods ◽  
Accurate Analysis ◽  
Data Set ◽  
X Ray ◽  
Soft Clustering

AbstractX-ray fluorescence (XRF) microscopy is an important tool for studying trace metals in biology, enabling simultaneous detection of multiple elements of interest and allowing quantification of metals in organelles without the need for subcellular fractionation. Currently, analysis of XRF images is often done using manually defined regions of interest (ROIs). However, since advances in synchrotron instrumentation have enabled the collection of very large data sets encompassing hundreds of cells, manual approaches are becoming increasingly impractical. We describe here the use of soft clustering to identify cell ROIs based on elemental contents, using data collected over a sample of the malaria parasite Plasmodium falciparum as a test case. Soft clustering was able to successfully classify regions in infected erythrocytes as “parasite,” “food vacuole,” “host,” or “background.” In contrast, hard clustering using the k-means algorithm was found to have difficulty in distinguishing cells from background. While initial tests showed convergence on two or three distinct solutions in 60% of the cells studied, subsequent modifications to the clustering routine improved results to yield 100% consistency in image segmentation. Data extracted using soft cluster ROIs were found to be as accurate as data extracted using manually defined ROIs, and analysis time was considerably improved.

Gaussian mixture copulas for high-dimensional clustering and dependency-based subtyping

2019 ◽  
Author(s):  
Siva Rajesh Kasa ◽  
Sakyajit Bhattacharya ◽  
Vaibhav Rajan
Keyword(s):  
Survival Rates ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Supplementary Information ◽  
High Dimensional ◽  
Clustering Methods ◽  
Mortality And Morbidity ◽  
Model Based Clustering ◽  
Cancer Data ◽  
Model Based

Abstract Motivation The identification of sub-populations of patients with similar characteristics, called patient subtyping, is important for realizing the goals of precision medicine. Accurate subtyping is crucial for tailoring therapeutic strategies that can potentially lead to reduced mortality and morbidity. Model-based clustering, such as Gaussian mixture models, provides a principled and interpretable methodology that is widely used to identify subtypes. However, they impose identical marginal distributions on each variable; such assumptions restrict their modeling flexibility and deteriorates clustering performance. Results In this paper, we use the statistical framework of copulas to decouple the modeling of marginals from the dependencies between them. Current copula-based methods cannot scale to high dimensions due to challenges in parameter inference. We develop HD-GMCM, that addresses these challenges and, to our knowledge, is the first copula-based clustering method that can fit high-dimensional data. Our experiments on real high-dimensional gene-expression and clinical datasets show that HD-GMCM outperforms state-of-the-art model-based clustering methods, by virtue of modeling non-Gaussian data and being robust to outliers through the use of Gaussian mixture copulas. We present a case study on lung cancer data from TCGA. Clusters obtained from HD-GMCM can be interpreted based on the dependencies they model, that offers a new way of characterizing subtypes. Empirically, such modeling not only uncovers latent structure that leads to better clustering but also meaningful clinical subtypes in terms of survival rates of patients. Availability and implementation An implementation of HD-GMCM in R is available at: https://bitbucket.org/cdal/hdgmcm/. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Mixture-of-Experts Variational Autoencoder for clustering and generating from similarity-based representations on single cell data

2021 ◽  
Vol 17 (6) ◽  
pp. e1009086
Author(s):  
Andreas Kopf ◽  
Vincent Fortuin ◽  
Vignesh Ram Somnath ◽  
Manfred Claassen
Keyword(s):  
Single Cell ◽  
High Dimensional Data ◽  
Mixture Distribution ◽  
Gaussian Mixture ◽  
High Dimensional ◽  
Complex Data ◽  
Mixture Of Experts ◽  
Data Set ◽  
Clustering Model ◽  
Variational Autoencoder

Clustering high-dimensional data, such as images or biological measurements, is a long-standing problem and has been studied extensively. Recently, Deep Clustering has gained popularity due to its flexibility in fitting the specific peculiarities of complex data. Here we introduce the Mixture-of-Experts Similarity Variational Autoencoder (MoE-Sim-VAE), a novel generative clustering model. The model can learn multi-modal distributions of high-dimensional data and use these to generate realistic data with high efficacy and efficiency. MoE-Sim-VAE is based on a Variational Autoencoder (VAE), where the decoder consists of a Mixture-of-Experts (MoE) architecture. This specific architecture allows for various modes of the data to be automatically learned by means of the experts. Additionally, we encourage the lower dimensional latent representation of our model to follow a Gaussian mixture distribution and to accurately represent the similarities between the data points. We assess the performance of our model on the MNIST benchmark data set and challenging real-world tasks of clustering mouse organs from single-cell RNA-sequencing measurements and defining cell subpopulations from mass cytometry (CyTOF) measurements on hundreds of different datasets. MoE-Sim-VAE exhibits superior clustering performance on all these tasks in comparison to the baselines as well as competitor methods.

scholarly journals Systematic exploration of unsupervised methods for mapping behavior

2016 ◽  
Author(s):  
Jeremy G. Todd ◽  
Jamey S. Kain ◽  
Benjamin L. de Bivort
Keyword(s):  
Gaussian Mixture ◽  
Data Sets ◽  
Clustering Methods ◽  
Cluster Assignment ◽  
Data Set ◽  
Behavioral Transitions ◽  
Assignment Algorithm ◽  
Higher Dimensional ◽  
Systematic Exploration ◽  
Components Analysis

AbstractTo fully understand the mechanisms giving rise to behavior, we need to be able to precisely measure it. When coupled with large behavioral data sets, unsupervised clustering methods offer the potential of unbiased mapping of behavioral spaces. However, unsupervised techniques to map behavioral spaces are in their infancy, and there have been few systematic considerations of all the methodological options. We compared the performance of seven distinct mapping methods in clustering a data set consisting of the x-and y-positions of the six legs of individual flies. Legs were automatically tracked by small pieces of fluorescent dye, while the fly was tethered and walking on an air-suspended ball. We find that there is considerable variation in the performance of these mapping methods, and that better performance is attained when clustering is done in higher dimensional spaces (which are otherwise less preferable because they are hard to visualize). High dimensionality means that some algorithms, including the non-parametric watershed cluster assignment algorithm, cannot be used. We developed an alternative watershed algorithm which can be used in high-dimensional spaces when the probability density estimate can be computed directly. With these tools in hand, we examined the behavioral space of fly leg postural dynamics and locomotion. We find a striking division of behavior into modes involving the fore legs and modes involving the hind legs, with few direct transitions between them. By computing behavioral clusters using the data from all flies simultaneously, we show that this division appears to be common to all flies. We also identify individual-to-individual differences in behavior and behavioral transitions. Lastly, we suggest a computational pipeline that can achieve satisfactory levels of performance without the taxing computational demands of a systematic combinatorial approach.AbbreviationsGMM: Gaussian mixture model; PCA: principal components analysis; SW: sparse watershed; t-SNE: t-distributed stochastic neighbor embedding

scholarly journals Incorporating Clustering Techniques into GAMLSS

Stats ◽  
2021 ◽  
Vol 4 (4) ◽  
pp. 916-930
Author(s):  
Thiago G. Ramires ◽  
Luiz R. Nakamura ◽  
Ana J. Righetto ◽  
Andréa C. Konrath ◽  
Carlos A. B. Pereira
Keyword(s):  
Mixture Models ◽  
Latent Variable ◽  
Generalized Additive Models ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Additive Models ◽  
Clustering Methods ◽  
Response Variable ◽  
Explanatory Variables ◽  
Variable Cluster

A method for statistical analysis of multimodal and/or highly distorted data is presented. The new methodology combines different clustering methods with the GAMLSS (generalized additive models for location, scale, and shape) framework, and is therefore called c-GAMLSS, for “clustering GAMLSS. ” In this new extended structure, a latent variable (cluster) is created to explain the response-variable (target). Any and all parameters of the distribution for the response variable can also be modeled by functions of the new covariate added to other available resources (features). The method of selecting resources to be used is carried out in stages, a step-based method. A simulation study considering multiple scenarios is presented to compare the c-GAMLSS method with existing Gaussian mixture models. We show by means of four different data applications that in cases where other authentic explanatory variables are or are not available, the c-GAMLSS structure outperforms mixture models, some recently developed complex distributions, cluster-weighted models, and a mixture-of-experts model. Even though we use simple distributions in our examples, other more sophisticated distributions can be used to explain the response variable.

Text Independent Amharic Language Speaker Identification in Noisy Environments using Speech Processing Techniques

2017 ◽  
Vol 5 (1) ◽  
pp. 109 ◽  
Author(s):  
Abrham Debasu Mengistu ◽  
Dagnachew Melesew Alemayehu
Keyword(s):  
Speech Processing ◽  
Speaker Identification ◽  
Hybrid Approach ◽  
Gaussian Mixture Models ◽  
Back Propagation ◽  
Gaussian Mixture ◽  
Back Propagation Neural Network ◽  
Identification System ◽  
Noisy Environments ◽  
Data Set

<p>In Ethiopia, the largest ethnic and linguistic groups are the Oromos, Amharas and Tigrayans. This paper presents the performance analysis of text-independent speaker identification system for the Amharic language in noisy environments. VQ (Vector Quantization), GMM (Gaussian Mixture Models), BPNN (Back propagation neural network), MFCC (Mel-frequency cepstrum coefficients), GFCC (Gammatone Frequency Cepstral Coefficients), and a hybrid approach had been use as techniques for identifying speakers of Amharic language in noisy environments. For the identification process, speech signals are collected from different speakers including both sexes; for our data set, a total of 90 speakers’ speech samples were collected, and each speech have 10 seconds duration from each individual. From these speakers, 59.2%, 70.9% and 84.7% accuracy are achieved when VQ, GMM and BPNN are used on the combined feature vector of MFCC and GFCC. </p>

Sign in / Sign up

Export Citation Format

Share Document