A Study on GBW-KNN Using Statistical Testing

In the 4th industrial revolution, big data and artificial intelligence are becoming more and more important. This is because the value can be four by applying artificial intelligence techniques to data generated and accumulated in real-time. Various industries utilize them to provide a variety of services and products to customers and enhance their competitiveness. The KNN algorithm is one of such analysis methods, which predicts the class of an unlabeled instance by using the classes of nearby neighbors. It is used a lot because it is simpler and easier to understand than other methods. In this study, we proposed a GBW-KNN algorithm that finds KNN after assigning weights to each individual based on the KNN graph. In addition, a statistical test was conducted to see if there was a significant difference in the performance difference between the KNN and GBW-KNN methods. As a result of the experiment, it was confirmed that the performance of GBW-KNN was excellent overall, and the difference in performance between the two methods was significant.

Scalable Semi-Supervised SVM via Triply Stochastic Gradients

Semi-supervised learning (SSL) plays an increasingly important role in the big data era because a large number of unlabeled samples can be used effectively to improve the performance of the classifier. Semi-supervised support vector machine (S3VM) is one of the most appealing methods for SSL, but scaling up S3VM for kernel learning is still an open problem. Recently, a doubly stochastic gradient (DSG) algorithm has been proposed to achieve efficient and scalable training for kernel methods. However, the algorithm and theoretical analysis of DSG are developed based on the convexity assumption which makes them incompetent for non-convex problems such as S3VM. To address this problem, in this paper, we propose a triply stochastic gradient algorithm for S3VM, called TSGS3VM. Specifically, to handle two types of data instances involved in S3VM, TSGS3VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient. We use the approximated gradient to update the solution. More importantly, we establish new theoretic analysis for TSGS3VM which guarantees that TSGS3VM can converge to a stationary point. Extensive experimental results on a variety of datasets demonstrate that TSGS3VM is much more efficient and scalable than existing S3VM algorithms.

Discriminatory Target Learning: Mining Significant Dependence Relationships from Labeled and Unlabeled Data

Machine learning techniques have shown superior predictive power, among which Bayesian network classifiers (BNCs) have remained of great interest due to its capacity to demonstrate complex dependence relationships. Most traditional BNCs tend to build only one model to fit training instances by analyzing independence between attributes using conditional mutual information. However, for different class labels, the conditional dependence relationships may be different rather than invariant when attributes take different values, which may result in classification bias. To address this issue, we propose a novel framework, called discriminatory target learning, which can be regarded as a tradeoff between probabilistic model learned from unlabeled instance at the uncertain end and that learned from labeled training data at the certain end. The final model can discriminately represent the dependence relationships hidden in unlabeled instance with respect to different possible class labels. Taking k-dependence Bayesian classifier as an example, experimental comparison on 42 publicly available datasets indicated that the final model achieved competitive classification performance compared to state-of-the-art learners such as Random forest and averaged one-dependence estimators.