scholarly journals Pendeteksi Kesamaan Dokumen pada Sistem Informasi Pendaftaran Proposal Skripsi dengan Pendekatan Algoritma Rabin-Karp

2019 ◽  
Vol 2 (1) ◽  
pp. 40-47
Author(s):  
Pandu Pratama Putra ◽  
Afriansyah Afriansyah ◽  
Muhammad Syaifullah
Keyword(s):  
Search Algorithm ◽  
Significant Problem ◽  
Digital Content ◽  
Text Documents ◽  
Testing Phase ◽  
Text Document

Plagiarism is a significant problem in many areas, including universities. plagiarism is usually performed on digital content is to copy-paste of the original document. To anticipate, we need a way to analyze the technique of plagiarism. There are several approaches that can be taken, for example by using a search algorithm Rabin-Karp string because these algorithms can be used to detect plagiarism in a text document. In the testing phase, the test documents used were three documents with similarity level categories of low, medium and high. From some of the testing that has been done, this approach can find the longest quote the same between the two text documents and measure the similarity of text documents. With this system will help prevent acts of plagiarism in the thesis proposal registration so that the absence of the same thesis. Keywords: Plagiarism, Rabin-Karp algorithm, Similarity, Document

An Improved B-hill Climbing Optimization Technique for Solving the Text Documents Clustering Problem

2020 ◽  
Vol 16 (4) ◽  
pp. 296-306 ◽  
Author(s):  
Laith Mohammad Abualigah ◽  
Essam Said Hanandeh ◽  
Ahamad Tajudin Khader ◽  
Mohammed Abdallh Otair ◽  
Shishir Kumar Shandilya
Keyword(s):  
Optimization Technique ◽  
Document Clustering ◽  
Text Clustering ◽  
Hill Climbing ◽  
Text Documents ◽  
Clustering Problem ◽  
Text Document ◽  
Text Information ◽  
Amount Of Knowledge ◽  
The Hill

Background: Considering the increasing volume of text document information on Internet pages, dealing with such a tremendous amount of knowledge becomes totally complex due to its large size. Text clustering is a common optimization problem used to manage a large amount of text information into a subset of comparable and coherent clusters. Aims: This paper presents a novel local clustering technique, namely, β-hill climbing, to solve the problem of the text document clustering through modeling the β-hill climbing technique for partitioning the similar documents into the same cluster. Methods: The β parameter is the primary innovation in β-hill climbing technique. It has been introduced in order to perform a balance between local and global search. Local search methods are successfully applied to solve the problem of the text document clustering such as; k-medoid and kmean techniques. Results: Experiments were conducted on eight benchmark standard text datasets with different characteristics taken from the Laboratory of Computational Intelligence (LABIC). The results proved that the proposed β-hill climbing achieved better results in comparison with the original hill climbing technique in solving the text clustering problem. Conclusion: The performance of the text clustering is useful by adding the β operator to the hill climbing.

scholarly journals Applying the Bell’s Test to Chinese Texts

Entropy ◽  
2020 ◽  
Vol 22 (3) ◽  
pp. 275
Author(s):  
Igor A. Bessmertny ◽  
Xiaoxi Huang ◽  
Aleksei V. Platonov ◽  
Chuqiao Yu ◽  
Julia A. Koroleva
Keyword(s):  
Quantum Entanglement ◽  
Chinese Text ◽  
Search Engines ◽  
Text Processing ◽  
Word Segmentation ◽  
Significant Problem ◽  
Text Segmentation ◽  
Text Documents ◽  
Segmentation Algorithms ◽  
Chinese Texts

Search engines are able to find documents containing patterns from a query. This approach can be used for alphabetic languages such as English. However, Chinese is highly dependent on context. The significant problem of Chinese text processing is the missing blanks between words, so it is necessary to segment the text to words before any other action. Algorithms for Chinese text segmentation should consider context; that is, the word segmentation process depends on other ideograms. As the existing segmentation algorithms are imperfect, we have considered an approach to build the context from all possible n-grams surrounding the query words. This paper proposes a quantum-inspired approach to rank Chinese text documents by their relevancy to the query. Particularly, this approach uses Bell’s test, which measures the quantum entanglement of two words within the context. The contexts of words are built using the hyperspace analogue to language (HAL) algorithm. Experiments fulfilled in three domains demonstrated that the proposed approach provides acceptable results.

DIGITAL CONTENT OF PERIODICALS IN THE ELECTRONIC CATALOGUE OF THE CENTRAL SCIENCE LIBRARY OF THE NAS OF BELARUS

2020 ◽  
Author(s):  
I. P. Komenda
Keyword(s):  
Full Text ◽  
Digital Content ◽  
Text Documents ◽  
Bibliographic Records

The publication deals with the initial stages of inclusion into the electronic catalogue of bibliographic records of electronic periodicals from eLIBRARY.RU platform and electronic serials which have been subscribed by the Central Science Library of the NAS of Belarus. The activities on addition of full text documents and tables of contents of periodicals into bibliographic records have been considered.

scholarly journals Language Semantics Interpretation with an Interaction-Based Recurrent Neural Network

2021 ◽  
Vol 3 (4) ◽  
pp. 922-945
Author(s):  
Shaw-Hwa Lo ◽  
Yiqiao Yin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
Search Algorithm ◽  
Greedy Search ◽  
Text Documents ◽  
Engineering Technique ◽  
Language Semantics ◽  
Sequential Models

Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models are capable of making good predictions, yet there is a lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), a greedy search algorithm, called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the “dagger technique”. First, the paper proposes to use the novel influence score (I-score) to detect and search for the important language semantics in text documents that are useful for making good predictions in text classification tasks. Next, a greedy search algorithm, called the Backward Dropping Algorithm, is proposed to handle long-term dependencies in the dataset. Moreover, the paper proposes a novel engineering technique called the “dagger technique” that fully preserves the relationship between the explanatory variable and the response variable. The proposed techniques can be further generalized into any feed-forward Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and any neural network. A real-world application on the Internet Movie Database (IMDB) is used and the proposed methods are applied to improve prediction performance with an 81% error reduction compared to other popular peers if I-score and “dagger technique” are not implemented.

scholarly journals Development of the documents comparison module for an electronic document management system

2019 ◽  
pp. 527-533
Author(s):  
M A Mikheev ◽  
P Y Yakimov
Keyword(s):  
Character Recognition ◽  
Optical Character Recognition ◽  
Document Management ◽  
Electronic Document ◽  
Text Documents ◽  
Text Document ◽  
Document Management System ◽  
Optical Character ◽  
Electronic Document Management ◽  
Scanned Image

The article is devoted to solving the problem of document versions comparison in electronic document management systems. Systems-analogues were considered, the process of comparing text documents was studied. In order to recognize the text on the scanned image, the technology of optical character recognition and its implementation — Tesseract library were chosen. The Myers algorithm is applied to compare received texts. The software implementation of the text document comparison module was implemented using the solutions described above.

scholarly journals The Evaluation of Accuracy Performance in an Enhanced Embedded Feature Selection for Unstructured Text Classification

2020 ◽  
pp. 3397-3407
Author(s):  
Nur Syafiqah Mohd Nafis ◽  
Suryanti Awang
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
Training Dataset ◽  
Recursive Feature Elimination ◽  
High Dimensional ◽  
Significant Feature ◽  
Support Vector ◽  
Svm Classifier ◽  
Text Documents ◽  
Text Document

Text documents are unstructured and high dimensional. Effective feature selection is required to select the most important and significant feature from the sparse feature space. Thus, this paper proposed an embedded feature selection technique based on Term Frequency-Inverse Document Frequency (TF-IDF) and Support Vector Machine-Recursive Feature Elimination (SVM-RFE) for unstructured and high dimensional text classificationhis technique has the ability to measure the feature’s importance in a high-dimensional text document. In addition, it aims to increase the efficiency of the feature selection. Hence, obtaining a promising text classification accuracy. TF-IDF act as a filter approach which measures features importance of the text documents at the first stage. SVM-RFE utilized a backward feature elimination scheme to recursively remove insignificant features from the filtered feature subsets at the second stage. This research executes sets of experiments using a text document retrieved from a benchmark repository comprising a collection of Twitter posts. Pre-processing processes are applied to extract relevant features. After that, the pre-processed features are divided into training and testing datasets. Next, feature selection is implemented on the training dataset by calculating the TF-IDF score for each feature. SVM-RFE is applied for feature ranking as the next feature selection step. Only top-rank features will be selected for text classification using the SVM classifier. Based on the experiments, it shows that the proposed technique able to achieve 98% accuracy that outperformed other existing techniques. In conclusion, the proposed technique able to select the significant features in the unstructured and high dimensional text document.

Assessment of Twitter Data Clusters with Cosine-Based Validation Metrics Using Hybrid Topic Models

2020 ◽  
Vol 25 (6) ◽  
pp. 755-769
Author(s):  
Noorullah R. Mohammed ◽  
Moulana Mohammed
Keyword(s):  
Data Clustering ◽  
Topic Models ◽  
Cluster Validity ◽  
Text Documents ◽  
Text Data ◽  
Validity Assessment ◽  
Text Document ◽  
Cluster Validity Indices ◽  
Validity Indices ◽  
Data Clusters

Text data clustering is performed for organizing the set of text documents into the desired number of coherent and meaningful sub-clusters. Modeling the text documents in terms of topics derivations is a vital task in text data clustering. Each tweet is considered as a text document, and various topic models perform modeling of tweets. In existing topic models, the clustering tendency of tweets is assessed initially based on Euclidean dissimilarity features. Cosine metric is more suitable for more informative assessment, especially of text clustering. Thus, this paper develops a novel cosine based external and interval validity assessment of cluster tendency for improving the computational efficiency of tweets data clustering. In the experimental, tweets data clustering results are evaluated using cluster validity indices measures. Experimentally proved that cosine based internal and external validity metrics outperforms the other using benchmarked and Twitter-based datasets.

Text Documents Plagiarism Detection using Rabin-Karp and Jaro-Winkler Distance Algorithms

2017 ◽  
Vol 5 (2) ◽  
pp. 462 ◽  
Author(s):  
Brinardi Leonardo ◽  
Seng Hansun
Keyword(s):  
Detection System ◽  
String Matching ◽  
Experimental Results ◽  
Plagiarism Detection ◽  
Text Documents ◽  
Matching Algorithm ◽  
Text Document ◽  
Different Types ◽  
The University

Plagiarism is an act that is considered by the university as a fraud by taking someone ideas or writings without mentioning the references and claimed as his own. Plagiarism detection system is generally implement string matching algorithm in a text document to search for common words between documents. There are some algorithms used for string matching, two of them are Rabin-Karp and Jaro-Winkler Distance algorithms. Rabin-Karp algorithm is one of compatible algorithms to solve the problem of multiple string patterns, while, Jaro-Winkler Distance algorithm has advantages in terms of time. A plagiarism detection application is developed and tested on different types of documents, i.e. doc, docx, pdf and txt. From the experimental results, we obtained that both of these algorithms can be used to perform plagiarism detection of those documents, but in terms of their effectiveness, Rabin-Karp algorithm is much more effective and faster in the process of detecting the document with the size more than 1000 KB.

Dual Scaling in Data Mining from Text Databases

2006 ◽  
Vol 10 (4) ◽  
pp. 451-457 ◽  
Author(s):  
Junzo Watada ◽  
◽  
Keisuke Aoki ◽  
Masahiro Kawano ◽  
Muhammad Suzuri Hitam ◽  
...  
Keyword(s):  
Multivariate Analysis ◽  
Text Mining ◽  
Kansei Engineering ◽  
Semantic Meaning ◽  
Dual Scaling ◽  
Text Documents ◽  
Text Data ◽  
Text Document ◽  
Text Information ◽  
Quantification Model

The availability of multimedia text document information has disseminated text mining among researchers. Text documents, integrate numerical and linguistic data, making text mining interesting and challenging. We propose text mining based on a fuzzy quantification model and fuzzy thesaurus. In text mining, we focus on: 1) Sentences included in Japanese text that are broken down into words. 2) Fuzzy thesaurus for finding words matching keywords in text. 3) Fuzzy multivariate analysis to analyze semantic meaning in predefined case studies. We use a fuzzy thesaurus to translate words using Chinese and Japanese characters into keywords. This speeds up processing without requiring a dictionary to separate words. Fuzzy multivariate analysis is used to analyze such processed data and to extract latent mutual related structures in text data, i.e., to extract otherwise obscured knowledge. We apply dual scaling to mining library and Web page text information, and propose integrating the result in Kansei engineering for possible application in sales, marketing, and production.

Sign in / Sign up

Export Citation Format

Share Document