MATHURA (MBI) - A NOVEL IMPUTATION MEASURE FOR IMPUTATION OF MISSING VALUES IN MEDICAL DATASETS

2019 ◽  
Vol 13 ◽  
Author(s):  
B. Mathura Bai ◽  
N. Mangathayaru ◽  
B. Padmaja Rani ◽  
Shadi Aljawarneh
Keyword(s):  
Similarity Measure ◽  
Medical Records ◽  
Missing Values ◽  
Similarity Measures ◽  
Common Problems ◽  
Experiment Analysis

: Missing attribute values in medical datasets are one of the most common problems faced when mining medical datasets. Estimation of missing values is a major challenging task in pre-processing of datasets. Any wrong estimate of missing attribute values can lead to inefficient and improper classification thus resulting in lower classifier accuracies. Similarity measures play a key role during the imputation process. The use of an appropriate and better similarity measure can help to achieve better imputation and improved classification accuracies. This paper proposes a novel imputation measure for finding similarity between missing and non-missing instances in medical datasets. Experiments are carried by applying both the proposed imputation technique and popular benchmark existing imputation techniques. Classification is carried using KNN, J48, SMO and RBFN classifiers. Experiment analysis proved that after imputation of medical records using proposed imputation technique, the resulting classification accuracies reported by the classifiers KNN, J48 and SMO have improved when compared to other existing benchmark imputation techniques.

scholarly journals A New Fuzzy Logic-Based Similarity Measure Applied to Large Gap Imputation for Uncorrelated Multivariate Time Series

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Thi-Thu-Hong Phan ◽  
André Bigand ◽  
Émilie Poisson Caillault
Keyword(s):  
Time Series ◽  
Fuzzy Logic ◽  
Similarity Measure ◽  
Missing Values ◽  
Multivariate Time Series ◽  
Similarity Measures ◽  
New Approach ◽  
The One ◽  
High Level ◽  
Basic Similarity

The completion of missing values is a prevalent problem in many domains of pattern recognition and signal processing. Analyzing data with incompleteness may lead to a loss of power and unreliable results, especially for large missing subsequence(s). Therefore, this paper aims to introduce a new approach for filling successive missing values in low/uncorrelated multivariate time series which allows managing a high level of uncertainty. In this way, we propose using a novel fuzzy weighting-based similarity measure. The proposed method involves three main steps. Firstly, for each incomplete signal, the data before a gap and the data after this gap are considered as two separated reference time series with their respective query windowsQbandQa. We then find the most similar subsequence (Qbs) to the subsequence before this gapQband the most similar one (Qas) to the subsequence after the gapQa. To find these similar windows, we build a new similarity measure based on fuzzy grades of basic similarity measures and on fuzzy logic rules. Finally, we fill in the gap with average values of the window followingQbsand the one precedingQas. The experimental results have demonstrated that the proposed approach outperforms the state-of-the-art methods in case of multivariate time series having low/noncorrelated data but effective information on each signal.

scholarly journals A set theory based similarity measure for text clustering and classification

2020 ◽  
Vol 7 (1) ◽  
Author(s):  
Ali A. Amer ◽  
Hassan I. Abdalla
Keyword(s):  
Set Theory ◽  
Similarity Measure ◽  
Similarity Measures ◽  
Text Clustering ◽  
Plagiarism Detection ◽  
K Nearest Neighbor ◽  
Single Measure ◽  
Highly Effective ◽  
Clustering And Classification ◽  
Effectiveness And Efficiency

Abstract Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures is that, until recently, there has never been one single measure recorded to be highly effective and efficient at the same time. Thus, the quest for an efficient and effective similarity measure is still an open-ended challenge. This study, in consequence, introduces a new highly-effective and time-efficient similarity measure for text clustering and classification. Furthermore, the study aims to provide a comprehensive scrutinization for seven of the most widely used similarity measures, mainly concerning their effectiveness and efficiency. Using the K-nearest neighbor algorithm (KNN) for classification, the K-means algorithm for clustering, and the bag of word (BoW) model for feature selection, all similarity measures are carefully examined in detail. The experimental evaluation has been made on two of the most popular datasets, namely, Reuters-21 and Web-KB. The obtained results confirm that the proposed set theory-based similarity measure (STB-SM), as a pre-eminent measure, outweighs all state-of-art measures significantly with regards to both effectiveness and efficiency.

scholarly journals A New Approach to Measuring the Similarity of Indoor Semantic Trajectories

2021 ◽  
Vol 10 (2) ◽  
pp. 90
Author(s):  
Jin Zhu ◽  
Dayu Cheng ◽  
Weiwei Zhang ◽  
Ci Song ◽  
Jie Chen ◽  
...  
Keyword(s):  
Similarity Measure ◽  
Semantic Information ◽  
Edit Distance ◽  
Similarity Measures ◽  
Indoor Positioning ◽  
Synthetic Dataset ◽  
Shopping Mall ◽  
Indoor Space ◽  
Trajectory Similarity ◽  
Indoor Spaces

People spend more than 80% of their time in indoor spaces, such as shopping malls and office buildings. Indoor trajectories collected by indoor positioning devices, such as WiFi and Bluetooth devices, can reflect human movement behaviors in indoor spaces. Insightful indoor movement patterns can be discovered from indoor trajectories using various clustering methods. These methods are based on a measure that reflects the degree of similarity between indoor trajectories. Researchers have proposed many trajectory similarity measures. However, existing trajectory similarity measures ignore the indoor movement constraints imposed by the indoor space and the characteristics of indoor positioning sensors, which leads to an inaccurate measure of indoor trajectory similarity. Additionally, most of these works focus on the spatial and temporal dimensions of trajectories and pay less attention to indoor semantic information. Integrating indoor semantic information such as the indoor point of interest into the indoor trajectory similarity measurement is beneficial to discovering pedestrians having similar intentions. In this paper, we propose an accurate and reasonable indoor trajectory similarity measure called the indoor semantic trajectory similarity measure (ISTSM), which considers the features of indoor trajectories and indoor semantic information simultaneously. The ISTSM is modified from the edit distance that is a measure of the distance between string sequences. The key component of the ISTSM is an indoor navigation graph that is transformed from an indoor floor plan representing the indoor space for computing accurate indoor walking distances. The indoor walking distances and indoor semantic information are fused into the edit distance seamlessly. The ISTSM is evaluated using a synthetic dataset and real dataset for a shopping mall. The experiment with the synthetic dataset reveals that the ISTSM is more accurate and reasonable than three other popular trajectory similarities, namely the longest common subsequence (LCSS), edit distance on real sequence (EDR), and the multidimensional similarity measure (MSM). The case study of a shopping mall shows that the ISTSM effectively reveals customer movement patterns of indoor customers.

Knowledge Transfer for Entity Resolution with Siamese Neural Networks

2021 ◽  
Vol 13 (1) ◽  
pp. 1-25
Author(s):  
Michael Loster ◽  
Ioannis Koumarelas ◽  
Felix Naumann
Keyword(s):  
Knowledge Transfer ◽  
Similarity Measure ◽  
State Of The Art ◽  
Similarity Measures ◽  
Engineering Process ◽  
Domain Experts ◽  
Multiple Datasets ◽  
Multiple Data ◽  
Domain Expertise ◽  
F Measure

The integration of multiple data sources is a common problem in a large variety of applications. Traditionally, handcrafted similarity measures are used to discover, merge, and integrate multiple representations of the same entity—duplicates—into a large homogeneous collection of data. Often, these similarity measures do not cope well with the heterogeneity of the underlying dataset. In addition, domain experts are needed to manually design and configure such measures, which is both time-consuming and requires extensive domain expertise. We propose a deep Siamese neural network, capable of learning a similarity measure that is tailored to the characteristics of a particular dataset. With the properties of deep learning methods, we are able to eliminate the manual feature engineering process and thus considerably reduce the effort required for model construction. In addition, we show that it is possible to transfer knowledge acquired during the deduplication of one dataset to another, and thus significantly reduce the amount of data required to train a similarity measure. We evaluated our method on multiple datasets and compare our approach to state-of-the-art deduplication methods. Our approach outperforms competitors by up to +26 percent F-measure, depending on task and dataset. In addition, we show that knowledge transfer is not only feasible, but in our experiments led to an improvement in F-measure of up to +4.7 percent.

scholarly journals A Novel Cosine Similarity Like Data Clustering Method for Effective Data Classification in Data Mining

2020 ◽  
Vol 9 (8) ◽  
pp. 340-346
Keyword(s):  
Data Mining ◽  
Similarity Measure ◽  
Categorical Data ◽  
Data Clustering ◽  
Similarity Measures ◽  
Numerical Data ◽  
Data Classification ◽  
Fundamental Goal ◽  
Learning Technique ◽  
Categorical Data Clustering

In data mining ample techniques use distance based measures for data clustering. Improving clustering performance is the fundamental goal in cluster domain related tasks. Many techniques are available for clustering numerical data as well as categorical data. Clustering is an unsupervised learning technique and objects are grouped or clustered based on similarity among the objects. A new cluster similarity finding measure, which is cosine like cluster similarity measure (CLCSM), is proposed in this paper. The proposed cluster similarity measure is used for data classification. Extensive experiments are conducted by taking UCI machine learning datasets. The experimental results have shown that the proposed cosinelike cluster similarity measure is superior to many of the existing cluster similarity measures for data classification.

SeqPAM

2008 ◽  
pp. 17-38 ◽  
Author(s):  
Pradeep Kumar Kumar ◽  
Raju S. Bapi ◽  
P. Radha Krishna
Keyword(s):  
Similarity Measure ◽  
Clustering Algorithm ◽  
Clustering Algorithms ◽  
Similarity Measures ◽  
Sequential Data ◽  
Cluster Validation ◽  
Web Personalization ◽  
Similarity Preserving ◽  
Validation Technique ◽  
The Web

With the growth in the number of web users and necessity for making information available on the web, the problem of web personalization has become very critical and popular. Developers are trying to customize a web site to the needs of specific users with the help of knowledge acquired from user navigational behavior. Since user page visits are intrinsically sequential in nature, efficient clustering algorithms for sequential data are needed. In this paper, we introduce a similarity preserving function called sequence and set similarity measure S3M that captures both the order of occurrence of page visits as well as the content of pages. We conducted pilot experiments comparing the results of PAM, a standard clustering algorithm, with two similarity measures: Cosine and S3M. The goodness of the clusters resulting from both the measures was computed using a cluster validation technique based on average levensthein distance. Results on pilot dataset established the effectiveness of S3M for sequential data. Based on these results, we proposed a new clustering algorithm, SeqPAM for clustering sequential data. We tested the new algorithm on two datasets namely, cti and msnbc datasets. We provided recommendations for web personalization based on the clusters obtained from SeqPAM for msnbc dataset.

scholarly journals Neutrosophic Modeling of Talcott Parsons’s Action and Decision-Making Applications for It

Symmetry ◽  
2020 ◽  
Vol 12 (7) ◽  
pp. 1166
Author(s):  
Cahit Aslan ◽  
Abdullah Kargın ◽  
Memet Şahin
Keyword(s):  
Decision Making ◽  
Similarity Measure ◽  
Similarity Measures ◽  
Theory Of Action ◽  
Grand Theory ◽  
Important Place ◽  
Social Theories ◽  
Measure Of Similarity ◽  
First Time ◽  
The Ideal

The grand theory of action of Parsons has an important place in social theories. Furthermore, there are many uncertainties in the theory of Parsons. Classical math logic is often insufficient to explain these uncertainties. In this study, we explain the grand theory of action of Parsons in neutrosociology for the first time. Thus, we achieve a more effective way of dealing with the uncertainties in the theory of Parsons as in all social theories. We obtain a similarity measure for single-valued neutrosophic numbers. In addition, we show that this measure of similarity satisfies the similarity measure conditions. By making use of this similarity measure, we obtain applications that allow finding the ideal society in the theory of Parsons within the theory of neutrosociology. In addition, we compare the results we obtained with the data in this study with the results of the similarity measures previously defined. Thus, we have checked the appropriateness of the decision-making application that we obtained.

scholarly journals Fuzzy Set-Valued Information Systems and the Algorithm of Filling Missing Values for Incomplete Information Systems

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-17
Author(s):  
Zhaohao Wang ◽  
Xiaoping Zhang
Keyword(s):  
Information System ◽  
Information Systems ◽  
Incomplete Information ◽  
Rough Set ◽  
Fuzzy Set ◽  
Missing Values ◽  
Rough Set Theory ◽  
Similarity Measures ◽  
New Information ◽  
Incomplete Information Systems

How to effectively deal with missing values in incomplete information systems (IISs) according to the research target is still a key issue for investigating IISs. If the missing values in IISs are not handled properly, they will destroy the internal connection of data and reduce the efficiency of data usage. In this paper, in order to establish effective methods for filling missing values, we propose a new information system, namely, a fuzzy set-valued information system (FSvIS). By means of the similarity measures of fuzzy sets, we obtain several binary relations in FSvISs, and we investigate the relationship among them. This is a foundation for the researches on FSvISs in terms of rough set approach. Then, we provide an algorithm to fill the missing values in IISs with fuzzy set values. In fact, this algorithm can transform an IIS into an FSvIS. Furthermore, we also construct an algorithm to fill the missing values in IISs with set values (or real values). The effectiveness of these algorithms is analyzed. The results showed that the proposed algorithms achieve higher correct rate than traditional algorithms, and they have good stability. Finally, we discuss the importance of these algorithms for investigating IISs from the viewpoint of rough set theory.

scholarly journals Two-Dimensional Spatial Frequency Content and Confusions among Dot Matrix Characters

1979 ◽  
Vol 23 (1) ◽  
pp. 384-388
Author(s):  
Michael E. Maddox
Keyword(s):  
Spatial Frequency ◽  
Similarity Measure ◽  
Similarity Measures ◽  
Final Analysis ◽  
Second Phase ◽  
Two Dimensional ◽  
Two Phase ◽  
Frequency Spectra ◽  
Character Size ◽  
Point Fast Fourier Transform

A two-phase study was conducted which related the confusions among dot matrix characters to the two-dimensional spatial frequency similarity of these characters. During the first phase of the study, subjects were shown single alphanumeric characters from four different dot matrix fonts and five matrix size/character sub-tense combinations. Data from this phase of the research were analyzed in terms of both correctness and character confusion frequencies. The second phase of the study consisted of digitizing and analyzing all characters from two of the fonts used in the first phase. The fonts chosen represent the most and least confusable of the four, based on the performance data obtained. These characters were scanned photometrically using a computer-controlled X-Y stage and subjected to a 512 × 512 point fast Fourier transform (FFT). The Fourier coefficients were correlated for all possible character pairs within each font-matrix/character size cell. These correlations provided an objective similarity measure among characters based upon their 2-D spatial frequency spectra. In addition to the spatial frequency similarity measure, a simple digital Phi coefficient was calculated for each character pair. The final analysis performed in this study was the correlation of observed performance (confusions) with objective similarity measures (2-D spectra and Phi coefficients).

Sign in / Sign up

Export Citation Format

Share Document