Text independent root word identification in Hindi language using natural language processing

2015 ◽  
Vol 7 (3/4) ◽  
pp. 240 ◽  
Author(s):  
Leena Jain ◽  
Prateek Agrawal
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Word Identification ◽  
Hindi Language ◽  
Root Word

A Hindi Image Caption Generation Framework Using Deep Learning

2021 ◽  
Vol 20 (2) ◽  
pp. 1-19
Author(s):  
Santosh Kumar Mishra ◽  
Rijul Dhir ◽  
Sriparna Saha ◽  
Pushpak Bhattacharyya
Keyword(s):  
Computer Vision ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
English Language ◽  
Image Captioning ◽  
Textual Description ◽  
Proposed Model ◽  
Hindi Language ◽  
The Given

Image captioning is the process of generating a textual description of an image that aims to describe the salient parts of the given image. It is an important problem, as it involves computer vision and natural language processing, where computer vision is used for understanding images, and natural language processing is used for language modeling. A lot of works have been done for image captioning for the English language. In this article, we have developed a model for image captioning in the Hindi language. Hindi is the official language of India, and it is the fourth most spoken language in the world, spoken in India and South Asia. To the best of our knowledge, this is the first attempt to generate image captions in the Hindi language. A dataset is manually created by translating well known MSCOCO dataset from English to Hindi. Finally, different types of attention-based architectures are developed for image captioning in the Hindi language. These attention mechanisms are new for the Hindi language, as those have never been used for the Hindi language. The obtained results of the proposed model are compared with several baselines in terms of BLEU scores, and the results show that our model performs better than others. Manual evaluation of the obtained captions in terms of adequacy and fluency also reveals the effectiveness of our proposed approach. Availability of resources : The codes of the article are available at https://github.com/santosh1821cs03/Image_Captioning_Hindi_Language ; The dataset will be made available: http://www.iitp.ac.in/∼ai-nlp-ml/resources.html .

scholarly journals A Survey on Using Gaze Behaviour for Natural Language Processing

2020 ◽  
Author(s):  
Sandeep Mathias ◽  
Diptesh Kanojia ◽  
Abhijit Mishra ◽  
Pushpak Bhattacharya
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Eye Tracking ◽  
Language Processing ◽  
Word Identification ◽  
Test Time ◽  
Gaze Behaviour ◽  
Cognitive Information ◽  
Essay Grading ◽  
Multiple Languages

Gaze behaviour has been used as a way to gather cognitive information for a number of years. In this paper, we discuss the use of gaze behaviour in solving different tasks in natural language processing (NLP) without having to record it at test time. This is because the collection of gaze behaviour is a costly task, both in terms of time and money. Hence, in this paper, we focus on research done to alleviate the need for recording gaze behaviour at run time. We also mention different eye tracking corpora in multiple languages, which are currently available and can be used in natural language processing. We conclude our paper by discussing applications in a domain - education - and how learning gaze behaviour can help in solving the tasks of complex word identification and automatic essay grading.

scholarly journals Modification of Stemming Algorithm Using A Non Deterministic Approach To Indonesian Text

2019 ◽  
Vol 13 (4) ◽  
pp. 379
Author(s):  
Wafda Rifai ◽  
Edi Winarko
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Deterministic Algorithm ◽  
Regular Word ◽  
Deterministic Approach ◽  
Candidate List ◽  
Root Word

 Natural Language Processing is part of Artificial Intelegence that focus on language processing. One of stage in Natural Language Processing is Preprocessing. Preprocessing is the stage to prepare data before it is processed. There are many types of proccess in preprocessing, one of them is stemming. Stemming is process to find the root word from regular word. Errors when determining root words can cause misinformation. In addition, stemming process does not always produce one root word because there are several words in Indonesian that have two possibilities as root word or affixes word, e.g.the word “beruang”.To handle these problems, this study proposes a stemmer with more accurate word results by employing a non deterministic algorithm which gives more than one word candidate result. All rules are checked and the word results are kept in a candidate list. In case there are several word candidates were found, then one result will be chosen.This stemmer has been tested to 15.934 word and results in an accurate level of 93%. Therefore the stemmer can be used to detect words with more than one root word.

Identification of Chinese Unknown Word Based on Finite Multi-List Method

2011 ◽  
Vol 474-476 ◽  
pp. 460-465
Author(s):  
Bo Sun ◽  
Sheng Hui Huang ◽  
Xiao Hua Liu
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Word Identification ◽  
Difficult Problem ◽  
Word Segmentation ◽  
Chinese Word ◽  
Unknown Word ◽  
List Method ◽  
Low Rate

Unknown word is a kind of word that is not included in the sub_word vocabulary, but must be cut out by the word segmentation program. Peoples’ names, place names and translated names are the major unknown words.Unknown Chinese words is a difficult problem in natural language processing, and also contributed to the low rate of correct segmention. This paper introduces the finite multi-list method that using the word fragments’ capability to composite a word and the location in the word tree to process the unknown Chinese words.The experiment recall is 70.67% ,the correct rate is 43.65% .The result of the experiment shows that unknown Chinese word identification based on the finite multi-list method is feasible.

scholarly journals Natural Language Processing and Its Challenges on Omotic Language Group of Ethiopia

2021 ◽  
Vol 3 (4) ◽  
Author(s):  
Girma Yohannis Bade
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Word Identification ◽  
Resource Limitation ◽  
Identification Problem ◽  
Digital Data ◽  
Data Resource ◽  
Language Groups ◽  
Recent Developments

This article reviews Natural Language Processing (NLP) and its challenge on Omotic language groups. All technological achievements are partially fuelled by the recent developments in NLP. NPL is one of component of an artificial intelligence (AI) and offers the facility to the companies that need to analyze their reliable business data. However, there are many challenges that tackle the effectiveness of NLP applications on Omotic language groups (Ometo) of Ethiopia. These challenges are irregularity of the words, stop word identification problem, compounding and languages ‘digital data resource limitation. Thus, this study opens the room to the upcoming researchers to further investigate the NLP application on these language groups.

scholarly journals Automatically Identifying the Source Words of Lexical Blends in English

2010 ◽  
Vol 36 (1) ◽  
pp. 129-149 ◽  
Author(s):  
Paul Cook ◽  
Suzanne Stevenson
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Statistical Model ◽  
Language Processing ◽  
Error Rate ◽  
Word Identification ◽  
Lexical Information ◽  
New Words ◽  
Preliminary Results ◽  
Source Word

Newly coined words pose problems for natural language processing systems because they are not in a system's lexicon, and therefore no lexical information is available for such words. A common way to form new words is lexical blending, as in cosmeceutical, a blend of cosmetic and pharmaceutical. We propose a statistical model for inferring a blend's source words drawing on observed linguistic properties of blends; these properties are largely based on the recognizability of the source words in a blend. We annotate a set of 1,186 recently coined expressions which includes 515 blends, and evaluate our methods on a 324-item subset. In this first study of novel blends we achieve an accuracy of 40% on the task of inferring a blend's source words, which corresponds to a reduction in error rate of 39% over an informed baseline. We also give preliminary results showing that our features for source word identification can be used to distinguish blends from other kinds of novel words.

scholarly journals An Efficient Framework for Detecting Various Moods in Hinglish and English Dataset

2019 ◽  
Vol 9 (2) ◽  
pp. 1943-1946
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Sentiment Analysis ◽  
Language Processing ◽  
Major Goal ◽  
Research Papers ◽  
Written Text ◽  
Web Technologies ◽  
Mixed Language ◽  
Hindi Language

Due to the paced growth in web technologies and natural language processing, research on Sentiment Analysis (SA) has become very popular in recent times. In recent years most of the research papers have focused on sentiment analysis based on polarity (positive and negative sentiments). This paper presents an effective framework for identification of various moods of person from its written text or sentences. The paper focuses on the mood detection in given text written in mixed language called “Hinglish”. Hinglish is actually a fusion of two languages, English with the Hindi language. The major goal of this research is to propose a methodology for extracting information of emotions from a given text in Hinglish. The framework tested on 700 sentences containing Hinglish data. Seven emotions anger, happy, joy, confidence, sadness, tentativeness and fear have been used for generation of results. The proposed approach yielded an accuracy of 93.96%.

Efficient Channel Attention Based Encoder–Decoder Approach for Image Captioning in Hindi

2022 ◽  
Vol 21 (3) ◽  
pp. 1-17
Author(s):  
Santosh Kumar Mishra ◽  
Gaurav Rai ◽  
Sriparna Saha ◽  
Pushpak Bhattacharyya
Keyword(s):  
Computer Vision ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
English Language ◽  
Image Understanding ◽  
Attention Mechanism ◽  
Image Captioning ◽  
Textual Description ◽  
Hindi Language

Image captioning refers to the process of generating a textual description that describes objects and activities present in a given image. It connects two fields of artificial intelligence, computer vision, and natural language processing. Computer vision and natural language processing deal with image understanding and language modeling, respectively. In the existing literature, most of the works have been carried out for image captioning in the English language. This article presents a novel method for image captioning in the Hindi language using encoder–decoder based deep learning architecture with efficient channel attention. The key contribution of this work is the deployment of an efficient channel attention mechanism with bahdanau attention and a gated recurrent unit for developing an image captioning model in the Hindi language. Color images usually consist of three channels, namely red, green, and blue. The channel attention mechanism focuses on an image’s important channel while performing the convolution, which is basically to assign higher importance to specific channels over others. The channel attention mechanism has been shown to have great potential for improving the efficiency of deep convolution neural networks (CNNs). The proposed encoder–decoder architecture utilizes the recently introduced ECA-NET CNN to integrate the channel attention mechanism. Hindi is the fourth most spoken language globally, widely spoken in India and South Asia; it is India’s official language. By translating the well-known MSCOCO dataset from English to Hindi, a dataset for image captioning in Hindi is manually created. The efficiency of the proposed method is compared with other baselines in terms of Bilingual Evaluation Understudy (BLEU) scores, and the results obtained illustrate that the method proposed outperforms other baselines. The proposed method has attained improvements of 0.59%, 2.51%, 4.38%, and 3.30% in terms of BLEU-1, BLEU-2, BLEU-3, and BLEU-4 scores, respectively, with respect to the state-of-the-art. Qualities of the generated captions are further assessed manually in terms of adequacy and fluency to illustrate the proposed method’s efficacy.

Sign in / Sign up

Export Citation Format

Share Document