scholarly journals Sistem Deteksi Struktur Kalimat Bahasa Arab Menggunakan Algoritma Light Stemming

2019 ◽  
Vol 19 (1) ◽  
pp. 109-118
Author(s):  
Mudafiq Riyan Pratama ◽  
Muhammad Yunus
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Failure Rate ◽  
Language Processing ◽  
Syntactic Analysis ◽  
Human Language ◽  
Accuracy Rate

To understand Arabic, it is necessary to study nahwu theory. Nahwu theory is the study of structure of Arabic sentences. By learning nahwu, being able to distinguish subjects, predicates and objects in sentences. One of the fields in the computer that studies about human language processing is NLP (Natural Language Processing), which is natural human language processing through syntactic analysis of sentences structure. One method to analyzing syntactic of sentences is stemming. One variant of the stemming algorithm is Light Stemming algorithm. Light Stemming algorithm is an algorithm that only removes prefix and sufix. Based on the tests conducted, the light stemming algorithm is able to detect nahwu with an accuracy rate of 82.22%. The 17.78% failure rate occurs because words that do not have an affix will be detected automatically the type is fi’il (verb), even though the fact may be that the type is isim (noun), and the failure of the detection results is also due to not being able to stemming the words that have affix in the middle (infix), because indeed the process of light stemming only eliminates the sufix and prefix.

PRINCIPAL PROBLEMS OF NATURAL LANGUAGE PROCESSING SYSTEMS

2018 ◽  
pp. 35-38
Author(s):  
O. Hyryn
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Syntactic Analysis ◽  
Syntactic Ambiguity ◽  
Grammatical Structure ◽  
English Sentence ◽  
Analysis Methods ◽  
The Way

The article deals with natural language processing, namely that of an English sentence. The article describes the problems, which might arise during the process and which are connected with graphic, semantic, and syntactic ambiguity. The article provides the description of how the problems had been solved before the automatic syntactic analysis was applied and the way, such analysis methods could be helpful in developing new analysis algorithms. The analysis focuses on the issues, blocking the basis for the natural language processing — parsing — the process of sentence analysis according to their structure, content and meaning, which aims to analyze the grammatical structure of the sentence, the division of sentences into constituent components and defining links between them.

scholarly journals Identificação de Pragas e Doenças na Cultura da Soja por meio de um Sistema Computacional em Linguagem Natural

2021 ◽  
Author(s):  
Carolinne Roque e Faria ◽  
Cinthyan Renata Sachs Camerlengo de Barb
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Computer System ◽  
Language Processing ◽  
Agricultural Area ◽  
Syntactic Analysis ◽  
Dependency Parsing ◽  
Named Entities ◽  
Pests And Diseases ◽  
Improve Production

Technology is becoming expressively popular among agribusiness producers and is progressing in all agricultural area. One of the difficulties in this context is to handle data in natural language to solve problems in the field of agriculture. In order to build up dialogs and provide rich researchers, the present work uses Natural Language Processing (NLP) techniques to develop an automatic and effective computer system to interact with the user and assist in the identification of pests and diseases in the soybean farming, stored in a database repository to provide accurate diagnoses to simplify the work of the agricultural professional and also for those who deal with a lot of information in this area. Information on 108 pests and 19 diseases that damage Brazilian soybean was collected from Brazilian bibliographic manuals with the purpose to optimize the data and improve production, using the spaCy library for syntactic analysis of NLP, which allowed the pre-process the texts, recognize the named entities, calculate the similarity between the words, verify dependency parsing and also provided the support for the development requirements of the CAROLINA tool (Robotized Agronomic Conversation in Natural Language) using the language belonging to the agricultural area.

scholarly journals SEQ2SEQ VS SKETCH FILLING STRUCTURE FOR NATURAL LANGUAGE TO SQL TRANSLATION

2020 ◽  
Vol XLIV-4/W3-2020 ◽  
pp. 7-11
Author(s):  
K. Ahkouk ◽  
M. Machkour ◽  
K. Majhadi ◽  
R. Mama
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Human Language ◽  
Test Results ◽  
Exact Matching ◽  
Cross Domain ◽  
Pros And Cons ◽  
Time Required

Abstract. Sequence to sequence models have been widely used in the recent years in the different tasks of Natural Language processing. In particular, the concept has been deeply adopted to treat the problem of translating human language questions to SQL. In this context, many studies suggest the use of sequence to sequence approaches for predicting the target SQL queries using the different available datasets. In this paper, we put the light on another way to resolve natural language processing tasks, especially the Natural Language to SQL one using the method of sketch-based decoding which is based on a sketch with holes that the model incrementally tries to fill. We present the pros and cons of each approach and how a sketch-based model can outperform the already existing solutions in order to predict the wanted SQL queries and to generate to unseen input pairs in different contexts and cross-domain datasets, and finally we discuss the test results of the already proposed models using the exact matching scores and the errors propagation and the time required for the training as metrics.

scholarly journals Text: An R-package for Analyzing and Visualizing Human Language Using Natural Language Processing and Deep Learning

2021 ◽  
Author(s):  
Oscar Nils Erik Kjell ◽  
H. Andrew Schwartz ◽  
Salvatore Giorgi
Keyword(s):  
Deep Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Rating Scale ◽  
State Of The Art ◽  
R Package ◽  
Language Models ◽  
Categorical Variables ◽  
Human Language

The language that individuals use for expressing themselves contains rich psychological information. Recent significant advances in Natural Language Processing (NLP) and Deep Learning (DL), namely transformers, have resulted in large performance gains in tasks related to understanding natural language such as machine translation. However, these state-of-the-art methods have not yet been made easily accessible for psychology researchers, nor designed to be optimal for human-level analyses. This tutorial introduces text (www.r-text.org), a new R-package for analyzing and visualizing human language using transformers, the latest techniques from NLP and DL. Text is both a modular solution for accessing state-of-the-art language models and an end-to-end solution catered for human-level analyses. Hence, text provides user-friendly functions tailored to test hypotheses in social sciences for both relatively small and large datasets. This tutorial describes useful methods for analyzing text, providing functions with reliable defaults that can be used off-the-shelf as well as providing a framework for the advanced users to build on for novel techniques and analysis pipelines. The reader learns about six methods: 1) textEmbed: to transform text to traditional or modern transformer-based word embeddings (i.e., numeric representations of words); 2) textTrain: to examine the relationships between text and numeric/categorical variables; 3) textSimilarity and 4) textSimilarityTest: to computing semantic similarity scores between texts and significance test the difference in meaning between two sets of texts; and 5) textProjection and 6) textProjectionPlot: to examine and visualize text within the embedding space according to latent or specified construct dimensions (e.g., low to high rating scale scores).

scholarly journals Basic challenges in natural language processing systems

2020 ◽  
pp. 41-45
Author(s):  
O. Hyryn
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Information Search ◽  
Question Answering ◽  
Syntactic Analysis ◽  
Anaphora Resolution ◽  
Grammatical Structure ◽  
English Sentence ◽  
Improved Model

The article proceeds from the intended use of parsing for the purposes of automatic information search, question answering, logical conclusions, authorship verification, text authenticity verification, grammar check, natural language synthesis and other related tasks, such as ungrammatical speech analysis, morphological class definition, anaphora resolution etc. The study covers natural language processing challenges, namely of an English sentence. The article describes formal and linguistic problems, which might arise during the process and which are connected with graphic, semantic, and syntactic ambiguity. The article provides the description of how the problems had been solved before the automatic syntactic analysis was applied and the way, such analysis methods could be helpful in developing new analysis algorithms today. The analysis focuses on the issues, blocking the basis for the natural language processing — parsing — the process of sentence analysis according to their structure, content and meaning, which aims to examine the grammatical structure of the sentence, the division of sentences into constituent components and defining links between them. The analysis identifies a number of linguistic issues that will contribute to the development of an improved model of automatic syntactic analysis: lexical and grammatical synonymy and homonymy, hypo- and hyperonymy, lexical and semantic fields, anaphora resolution, ellipsis, inversion etc. The scope of natural language processing reveals obvious directions for the improvement of parsing models. The improvement will consequently expand the scope and improve the results in areas that already employ automatic parsing. Indispensable achievements in vocabulary and morphology processing shall not be neglected while improving automatic syntactic analysis mechanisms for natural languages.

Sign in / Sign up

Export Citation Format

Share Document