Using word semantic concepts for plagiarism detection in text documents

2021 ◽  
Author(s):  
Chia-Yang Chang ◽  
Shie-Jue Lee ◽  
Chih-Hung Wu ◽  
Chih-Feng Liu ◽  
Ching-Kuan Liu
Keyword(s):  
Plagiarism Detection ◽  
Text Documents ◽  
Semantic Concepts

Latent Dirichlet Allocation and POS Tags Based Method for External Plagiarism Detection

2019 ◽  
pp. 319-336
Author(s):  
Ali Daud ◽  
Jamal Ahmad Khan ◽  
Jamal Abdul Nasir ◽  
Rabeeh Ayaz Abbasi ◽  
Naif Radi Aljohani ◽  
...  
Keyword(s):  
Latent Dirichlet Allocation ◽  
Plagiarism Detection ◽  
Text Documents ◽  
Parts Of Speech ◽  
Stop Word ◽  
Processing Step ◽  
Syntactic Information ◽  
N Gram ◽  
Basic Hypothesis ◽  
Dirichlet Allocation

In this article we present a new semantic and syntactic-based method for external plagiarism detection. In the proposed approach, latent dirichlet allocation (LDA) and parts of speech (POS) tags are used together to detect plagiarism between the sample and a number of source documents. The basic hypothesis is that considering semantic and syntactic information between two text documents may improve the performance of the plagiarism detection task. Our method is based on two steps, naming, which is a pre-processing where we detect the topics from the sentences in documents using the LDA and convert each sentence in POS tags array; then a post processing step where the suspicious cases are verified purely on the basis of semantic rules. For two types of external plagiarism (copy and random obfuscation), we empirically compare our approach to the state-of-the-art N-gram based and stop-word N-gram based methods and observe significant improvements.

Text Documents Plagiarism Detection using Rabin-Karp and Jaro-Winkler Distance Algorithms

2017 ◽  
Vol 5 (2) ◽  
pp. 462 ◽  
Author(s):  
Brinardi Leonardo ◽  
Seng Hansun
Keyword(s):  
Detection System ◽  
String Matching ◽  
Experimental Results ◽  
Plagiarism Detection ◽  
Text Documents ◽  
Matching Algorithm ◽  
Text Document ◽  
Different Types ◽  
The University

Plagiarism is an act that is considered by the university as a fraud by taking someone ideas or writings without mentioning the references and claimed as his own. Plagiarism detection system is generally implement string matching algorithm in a text document to search for common words between documents. There are some algorithms used for string matching, two of them are Rabin-Karp and Jaro-Winkler Distance algorithms. Rabin-Karp algorithm is one of compatible algorithms to solve the problem of multiple string patterns, while, Jaro-Winkler Distance algorithm has advantages in terms of time. A plagiarism detection application is developed and tested on different types of documents, i.e. doc, docx, pdf and txt. From the experimental results, we obtained that both of these algorithms can be used to perform plagiarism detection of those documents, but in terms of their effectiveness, Rabin-Karp algorithm is much more effective and faster in the process of detecting the document with the size more than 1000 KB.

Unsupervised Multi-Document Abstractive Summarization Using Recursive Neural Network with Attention Mechanism

2020 ◽  
Vol 17 (9) ◽  
pp. 3867-3872
Author(s):  
Aniv Chakravarty ◽  
Jagadish S. Kallimani
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Attention Mechanism ◽  
Text Summarization ◽  
Text Generation ◽  
Text Documents ◽  
Current State ◽  
Semantic Concepts ◽  
Text Information ◽  
Abstractive Summarization

Text summarization is an active field of research with a goal to provide short and meaningful gists from large amount of text documents. Extractive text summarization methods have been extensively studied where text is extracted from the documents to build summaries. There are various type of multi document ranging from different formats to domains and topics. With the recent advancement in technology and use of neural networks for text generation, interest for research in abstractive text summarization has increased significantly. The use of graph based methods which handle semantic information has shown significant results. When given a set of documents of English text files, we make use of abstractive method and predicate argument structures to retrieve necessary text information and pass it through a neural network for text generation. Recurrent neural networks are a subtype of recursive neural networks which try to predict the next sequence based on the current state and considering the information from previous states. The use of neural networks allows generation of summaries for long text sentences as well. This paper implements a semantic based filtering approach using a similarity matrix while keeping all stop-words. The similarity is calculated using semantic concepts and Jiang–Conrath similarity and making use of a recurrent neural network with an attention mechanism to generate summary. ROUGE score is used for measuring accuracy, precision and recall scores.

scholarly journals Multi-Document Abstractive Summarization using Recursive Neural Network

2020 ◽  
Vol 9 (7) ◽  
pp. 364-370
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Text Summarization ◽  
Similarity Matrix ◽  
Text Documents ◽  
Short Text ◽  
Current State ◽  
Recursive Neural Networks ◽  
Semantic Concepts ◽  
Abstractive Summarization

Text summarization is an area of research with a goal to provide short text from huge text documents. Extractive text summarization methods have been extensively studied by many researchers. There are various type of multi document ranging from different formats to domains and topic specific. With the application of neural networks for text generation, interest for research in abstractive text summarization has increased significantly. This approach has been attempted for English and Telugu languages in this article. Recurrent neural networks are a subtype of recursive neural networks which try to predict the next sequence based on the current state and considering the information from previous states. The use of neural networks allows generation of summaries for long text sentences as well. The work implements semantic based filtering using a similarity matrix while keeping all stop-words. The similarity is calculated using semantic concepts and Jiang Similarity and making use of a Recurrent Neural Network (RNN) with an attention mechanism to generate summary. ROUGE score is used for measuring the performance of the applied method on Telugu and English langauges .

scholarly journals PERANCANGAN DAN PENERAPAN ALGORITMA RIZKI TANJUNG 24 (RTG24) UNTUK KOMPARASI KATA PADA FILE TEXT

Compiler ◽  
2014 ◽  
Vol 3 (1) ◽  
Author(s):  
Rizki Tanjung ◽  
Haruno Sajati ◽  
Dwi Nugraheny
Keyword(s):  
String Matching ◽  
Plagiarism Detection ◽  
Text Documents ◽  
Text Document ◽  
Basic Word ◽  
Root Word

Plagiarism is the act of taking essay or work of others, and recognize it as his own work. Plagiarism of the text is very common and difficult to avoid. Therefore, many created a system that can assist in plagiarism detection text document. To make the detection of plagiarism of text documents at its core is to perform string matching. This makes the emergence of the idea to build an algorithm that will be implemented in RTG24 Comparison file.txt applications. Document to be compared must be a file. Txt or plaintext, and every word contained in the document must be in the dictionary of Indonesian. RTG24 algorithm works by determining the number of same or similar words in any text between the two documents. In the process RTG24 algorithm has several stages: parsing, filtering, stemming and comparison. Parsing stage is the stage where every sentence in the document will be broken down into basic words, filtering step is cleaning the particles are not important. The next stage, stemming is the stage where every word searchable basic word or root word, this is done to simplify and facilitate comparison between the two documents. Right after through the process of parsing, filtering, and stemming, then the document should be inserted into the array for the comparison or the comparison between the two documents. So it can be determined the percentage of similarity between the two documents.

Performance Analysis and Evaluation of Feature Selection Techniques to Find Coherence Using Cosine Similarity

2020 ◽  
Vol 17 (9) ◽  
pp. 4106-4110
Author(s):  
Mausumi Goswami ◽  
B. S. Purkayastha
Keyword(s):  
Feature Selection ◽  
Vital Role ◽  
Unstructured Data ◽  
Sensor Data ◽  
Computational Techniques ◽  
Plagiarism Detection ◽  
Feature Selection Technique ◽  
Text Documents ◽  
Selection Technique ◽  
Feature Selection Techniques

Unstructured Data is utilized in many major applications. It seems 80% of the data generated by various business applications are unstructured. Unstructured data can not be directly processed to generate information. Few major applications which uses AI are Recommendation systems, Sentiment Analysis of customer’s emotions, finding duplicate content through plagiarism detection, document organization based on the requirements etc. Different forms of origin of such data can be categorized as unstructured text on the World Wide Web, sensor data, digital images, videos, sound, result of scientific experiments and user profiles for marketing. Information retrieval from huge text datasets is quite challenging. This is caused by the various characteristics associated with natural languages and a major concern in text mining. Before we apply computational techniques on documents it is important to make the documents ready for processing. Document Preprocessing is one such method applied for text documents. Document Preprocessing plays a vital role in document grouping. In this paper, four feature selection techniques are implemented and empirical investigation results are included. The evaluation of the grouping outcomes are used to evaluate the effectiveness of each feature selection technique. The evaluation of the grouping outcomes are done to evaluate the effectiveness of each feature selection technique.

scholarly journals Taxonomy of academic plagiarism methods

2021 ◽  
Vol 9 (1) ◽  
pp. 283-300
Author(s):  
Tedo Vrbanec ◽  
Ana Meštrović
Keyword(s):  
Academic Community ◽  
Plagiarism Detection ◽  
New Classification ◽  
Software Developers ◽  
Text Documents ◽  
Comprehensive Classification ◽  
Academic Plagiarism

The article gives an overview of the plagiarism domain, with focus on academic plagiarism. The article defines plagiarism, explains the origin of the term, as well as plagiarism related terms. It identifies the extent of the plagiarism domain and then focuses on the plagiarism subdomain of text documents, for which it gives an overview of current classifications and taxonomies and then proposes a more comprehensive classification according to several criteria: their origin and purpose, technical implementation, consequence, complexity of detection and according to the number of linguistic sources. The article suggests the new classification of academic plagiarism, describes sorts and methods of plagiarism, types and categories, approaches and phases of plagiarism detection, the classification of methods and algorithms for plagiarism detection. The title of the article explicitly targets the academic community, but it is sufficiently general and interdisciplinary, so it can be useful for many other professionals like software developers, linguists and librarians.

Sign in / Sign up

Export Citation Format

Share Document