An Algebraic Implicitization and Specialization of Minimum KL-Divergence Models

2010 ◽  
pp. 85-96
Author(s):  
Ambedkar Dukkipati ◽  
Joel George Manathara
Keyword(s):  
Kl Divergence

A Hard C-Means Clustering Algorithm Incorporating Membership KL Divergence and Local Data Information for Noisy Image Segmentation

2017 ◽  
Vol 32 (04) ◽  
pp. 1850012 ◽  
Author(s):  
R. R. Gharieb ◽  
G. Gendy ◽  
H. Selim
Keyword(s):  
Image Segmentation ◽  
Membership Function ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Cluster Center ◽  
Local Data ◽  
Cluster Membership ◽  
Kl Divergence ◽  
Clustering Approach ◽  
Center Distance

In this paper, the standard hard C-means (HCM) clustering approach to image segmentation is modified by incorporating weighted membership Kullback–Leibler (KL) divergence and local data information into the HCM objective function. The membership KL divergence, used for fuzzification, measures the proximity between each cluster membership function of a pixel and the locally-smoothed value of the membership in the pixel vicinity. The fuzzification weight is a function of the pixel to cluster-centers distances. The used pixel to a cluster-center distance is composed of the original pixel data distance plus a fraction of the distance generated from the locally-smoothed pixel data. It is shown that the obtained membership function of a pixel is proportional to the locally-smoothed membership function of this pixel multiplied by an exponentially distributed function of the minus pixel distance relative to the minimum distance provided by the nearest cluster-center to the pixel. Therefore, since incorporating the locally-smoothed membership and data information in addition to the relative distance, which is more tolerant to additive noise than the absolute distance, the proposed algorithm has a threefold noise-handling process. The presented algorithm, named local data and membership KL divergence based fuzzy C-means (LDMKLFCM), is tested by synthetic and real-world noisy images and its results are compared with those of several FCM-based clustering algorithms.

scholarly journals Research on the Application of User Recommendation Based on the Fusion Method of Spatially Complex Location Similarity

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Lili Wang ◽  
Ting Shi ◽  
Shijin Li
Keyword(s):  
Collaborative Filtering ◽  
Similarity Measure ◽  
Recall Rate ◽  
Cosine Similarity ◽  
Fusion Method ◽  
Complex Matrix ◽  
Recommendation Algorithm ◽  
Kl Divergence ◽  
Experimental Effect ◽  
User Recommendation

Since the user recommendation complex matrix is characterized by strong sparsity, it is difficult to correctly recommend relevant services for users by using the recommendation method based on location and collaborative filtering. The similarity measure between users is low. This paper proposes a fusion method based on KL divergence and cosine similarity. KL divergence and cosine similarity have advantages by comparing three similar metrics at different K values. Using the fusion method of the two, the user’s similarity with the preference is reused. By comparing the location-based collaborative filtering (LCF) algorithm, user-based collaborative filtering (UCF) algorithm, and user recommendation algorithm (F2F), the proposed method has the preparation rate, recall rate, and experimental effect advantage. In different median values, the proposed method also has an advantage in experimental results.

Concentration inequalities for the empirical distribution of discrete distributions: beyond the method of types

2019 ◽  
Vol 9 (4) ◽  
pp. 813-850 ◽  
Author(s):  
Jay Mardia ◽  
Jiantao Jiao ◽  
Ervin Tánczos ◽  
Robert D Nowak ◽  
Tsachy Weissman
Keyword(s):  
Sample Size ◽  
Empirical Distribution ◽  
Discrete Distributions ◽  
Concentration Inequalities ◽  
Sample Sizes ◽  
Alphabet Size ◽  
Large Sample ◽  
Kl Divergence ◽  
The Difference ◽  
True Distribution

Abstract We study concentration inequalities for the Kullback–Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes of sample size $n$ and alphabet size $k$, and the improvement becomes more significant when $k$ is large. We discuss the applications of our results in obtaining tighter concentration inequalities for $L_1$ deviations of the empirical distribution from the true distribution, and the difference between concentration around the expectation or zero. We also obtain asymptotically tight bounds on the variance of the KL divergence between the empirical and true distribution, and demonstrate their quantitatively different behaviours between small and large sample sizes compared to the alphabet size.

Sign in / Sign up

Export Citation Format

Share Document