Nonmetric ANOVA: a generic framework for analysis of variance on dissimilarity measures

Classic analysis of variance (ANOVA; cA) tests the explanatory power of a partitioning on a set of objects. Nonparametric ANOVA (npA) extends to a case where instead of the object values themselves, their mutual distances are available. While considerably widening the applicability of the cA, the npA does not provide a statistical framework for the cases where the mutual dissimilarity measurements between objects are nonmetric. Based on the central limit theorem (CLT), we introduce nonmetric ANOVA (nmA) as an extension of the cA and npA models where metric properties (identity, symmetry, and subadditivity) are relaxed. Our model allows any dissimilarity measures to be defined between objects where a distinctiveness of a specific partitioning imposed on those are of interest. This derivation accommodates an ANOVA-like framework of judgment, indicative of significant dispersion of the partitioned outputs in nonmetric space. We present a statistic which under the null hypothesis of no differences between the mean of the imposed partitioning, follows an exact F-distribution allowing to obtain the consequential p-value. Three biological examples are provided and the performance of our method in relation to the cA and npA is discussed.

Time Series Clustering for Robust Mean-Variance Portfolio Selection: Comparison of Several Dissimilarity Measures

This paper shows how to create a robust portfolio selection with time series clustering by using some dissimilarity measure. Based on such dissimilarity measures, stocks are initially sorted into multiple clusters using the Partitioning Around Medoids (PAM) time series clustering approach. Following clustering, a portfolio is constructed by selecting one stock from each cluster. Stocks having the greatest Sharpe ratio are selected from each cluster. The optimum portfolio is then constructed using the robust Fast Minimum Covariance Determinant (FMCD) and robust S MV portfolio model. When there are a big number of stocks accessible for the portfolio formation process, we can use this approach to quickly generate the optimum portfolio. This approach is also resistant to the presence of any outliers in the data. The Sharpe ratio was used to evaluate the performance of the portfolios that were created. The daily closing price of stocks listed on the Indonesia Stock Exchange, which are included in the LQ-45 indexed from August 2017 to July 2018, was utilized as a case study. Empirical study revealed that portfolios constructed using PAM time series clustering with autocorrelation dissimilarity and a robust FMCD MV portfolio model outperformed portfolios created using other approaches.

Hyperspectral Data Clustering Using Hellinger Divergence

Clustering is an important task in hyperspectral image processing. Despite the existence of a large number of clustering algorithms, little attention has been paid to the use of non-Euclidean dissimilarity measures in the clustering of hyperspectral data. This paper proposes a clustering technique based on the Hellinger divergence as a dissimilarity measure. The proposed technique uses Lloyd’s ideas of the k-means algorithm and gradient descent-based procedure to update clusters centroids. The proposed technique is compared with an alternative fast k-medoid algorithm implemented using the same metric from the viewpoint of clustering error and runtime. Experiments carried out using an open hyperspectral scene have shown the advantages of the proposed technique.

An Integrated Process for Verifying Deep Learning Classifiers Using Dataset Dissimilarity Measures

The specification and verification of algorithms is vital for safety-critical autonomous systems which incorporate deep learning elements. We propose an integrated process for verifying artificial neural network (ANN) classifiers. This process consists of an off-line verification and an on-line performance prediction phase. The process is intended to verify ANN classifier generalisation performance, and to this end makes use of dataset dissimilarity measures. We introduce a novel measure for quantifying the dissimilarity between the dataset used to train a classification algorithm, and the test dataset used to evaluate and verify classifier performance. A system-level requirement could specify the permitted form of the functional relationship between classifier performance and a dissimilarity measure; such a requirement could be verified by dynamic testing. Experimental results, obtained using publicly available datasets, suggest that the measures have relevance to real-world practice for both quantifying dataset dissimilarity, and specifying and verifying classifier performance.

A STEP TOWARDS THE MAJORITY-BASED CLUSTERING VALIDATION DECISION FUSION METHOD

A variety of clustering validation indices (CVIs) aimed at validating the results of clustering analysis and determining which clustering algorithm performs best. Different validation indices may be appropriate for different clustering algorithms or partition dissimilarity measures; however, the best suitable index to use in practice remains unknown. A single CVI is generally unable to handle the wide variability and scalability of the data and cope successfully with all the contexts. Therefore, one of the popular approaches is to use a combination of multiple CVIs and fuse their votes into the final decision. The aim of this work is to analyze the majority-based decision fusion method. Thus, the experimental work consisted of designing and implementing the NbClust majority-based decision fusion method and then evaluating the CVIs performance with different clustering algorithms and dissimilarity measures in order to discover the best validation configuration. Moreover, the author proposed to enhance the standard majority-based decision fusion method with straightforward rules for the maximum efficiency of the validation procedure. The result showed that the designed enhanced method with an invasive validation configuration could cope with almost all data sets (99%) with different experimental factors (density, dimensionality, number of clusters, etc.).

Parental selection based on molecular information under a population genetics approach

The correct choice of parents that will compose optimal segregating populations is the key to success for breeding programs. It was postulated the hypothesis that this choice of these parents could be made based on information of molecular markers analyzed in the context of population structure. Ten parental populations were simulated and 45 hybrid combinations were obtained from the dialel crosses. Each population consisted of 200 individuals with 50 independent loci. The populations were evaluated for the Hardy-Weinberg Equilibrium (HWE), Coefficient of Inbreeding (F), Heterozygosity (H), and the Polymorphic Information Content (PIC). Genetic diversity between pairs of parental populations was evaluated using five dissimilarity measures. Values of Mantel correlation were obtained for the pairs of the dissimilarity matrices, and the PIC, H, and F values ​​were obtained in the hybrid combinations. All parental populations were under HWE, and the combination that emerged from this condition was the hybrid 3x5, with only 26% of the loci manifesting HWE. This same hybrid was among those with lower F estimates and higher values ​​of H, which indicated the existence of greater divergence between their parentals. There was agreement on the indication of the more and less divergent hybrid combinations for the dissimilarity measures. This fact is important because the variability, associated with the good average potential, are important criteria for the formation of an initial population in breeding programs of any kind, involving sexual processes.