Weighting features by the value displacement rebound

Learning from examples draws on similarity, a concept which formalisation leads to the notion of instance space. Continuous spaces are easier to embrace since, unlike discrete, they often can be seen as hyper-constructs of 3D. Unsurprisingly, the instance-based learning methods are more developed for continuous domains than for discrete ones. The value difference metric (VDM) is one of the few examples of metrics for discrete spaces. Mixed reports about utility of VDM exist. In this paper VDM is compared with another approach where data features are weighted by the Information Gain. Some vulnerabilities of VDM are identified. A weighting method, nothing like VDM, although inspired by the former, is proposed. The results are in favour of the new weighting scheme with illustration of utility for health diagnostics.

A NOVEL DISTANCE FUNCTION: FREQUENCY DIFFERENCE METRIC

A high quality distance function that measures the difference between instances is essential in many real-world applications and research fields. For example, in instance-based learning, the distance function plays the most important role. A large number of distance functions have been proposed. For nominal attributes, Value Difference Metric (VDM) is one of the state-of-the-art and widely used distance functions. However, it needs to estimate the conditional probabilities, which drops its efficiency in computing the distance between instances. Besides, a practical issue that arises in estimating the conditional probabilities is that the denominators can be zero or very small. This makes them either undefined or very large. Therefore, an efficient distance function that can measure the difference between two instances but without the practical issue confronting VDM is desirable. In this paper, we propose a novel distance function: Frequency Difference Metric (FDM). FDM is just based on the joint frequencies of class labels and attribute values, instead of the conditional probabilities. Extensive empirical studies show that FDM performs almost as well as VDM in terms of accuracy, but significantly outperforms VDM in terms of efficiency. This work provides a very simple, efficient, and effective distance function that can be widely used in many real-world applications and research fields.