Automatic scoring of Chinese fill-in-the-blank questions based on improved P-means

2021 ◽  
pp. 1-10
Author(s):  
Wang Dong ◽  
Zhao Yong ◽  
Lin Hong ◽  
Zuo Xin
Keyword(s):  
Semantic Similarity ◽  
Test Site ◽  
Learning Ability ◽  
Common Word ◽  
Dictionary Matching ◽  
Word Test ◽  
Word Level ◽  
Subject Term ◽  
Sentence Level ◽  
Automatic Scoring

Chinese fill-in-the-blank questions contain both objective and subjective characteristics, and thus it has always been difficult to score them automatically. In this paper, fill-in-the-blank items are divided into those with word-level or sentence-level granularity; then, the items are automatically scored by different strategies. The automatic scoring framework combines semantic dictionary matching and semantic similarity calculations. First, fill-in-the-blank items with word-level granularity are divided into two types of test sites: the subject term test site, and the common word test site. We propose an algorithm for identifying an item’s test site. Then, a subject term dictionary with self-feedback learning ability is constructed to support the scoring of subject term test sites. The Tongyici Cilin semantic dictionary is used for scoring common word test sites. For fill-in-the-blank items with sentence-level granularity, an improved P-means model is used to generate a sentence vector of the standard answer and the examinee’s answer, and then the semantic similarity between the two answers is obtained by calculating the cosine distance of the sentence vector. Experimental results on actual test data show that the proposed algorithm has a maximum accuracy of 94.3% and achieves good results.

scholarly journals Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension

2020 ◽  
Vol 34 (05) ◽  
pp. 9725-9732
Author(s):  
Xiaorui Zhou ◽  
Senlin Luo ◽  
Yunfang Wu
Keyword(s):  
Reading Comprehension ◽  
Semantic Similarity ◽  
State Of The Art ◽  
Deep Understanding ◽  
Hierarchical Architecture ◽  
Hierarchical Network ◽  
Word Level ◽  
Human Evaluation ◽  
Sentence Level ◽  
The Relationship

In reading comprehension, generating sentence-level distractors is a significant task, which requires a deep understanding of the article and question. The traditional entity-centered methods can only generate word-level or phrase-level distractors. Although recently proposed neural-based methods like sequence-to-sequence (Seq2Seq) model show great potential in generating creative text, the previous neural methods for distractor generation ignore two important aspects. First, they didn't model the interactions between the article and question, making the generated distractors tend to be too general or not relevant to question context. Second, they didn't emphasize the relationship between the distractor and article, making the generated distractors not semantically relevant to the article and thus fail to form a set of meaningful options. To solve the first problem, we propose a co-attention enhanced hierarchical architecture to better capture the interactions between the article and question, thus guide the decoder to generate more coherent distractors. To alleviate the second problem, we add an additional semantic similarity loss to push the generated distractors more relevant to the article. Experimental results show that our model outperforms several strong baselines on automatic metrics, achieving state-of-the-art performance. Further human evaluation indicates that our generated distractors are more coherent and more educative compared with those distractors generated by baselines.

Deaf and Hard-of-hearing Users Evaluating Designs for Highlighting Key Words in Educational Lecture Videos

2021 ◽  
Vol 14 (4) ◽  
pp. 1-24
Author(s):  
Sushant Kafle ◽  
Becca Dingman ◽  
Matt Huenerfauth
Keyword(s):  
Experimental Study ◽  
Key Words ◽  
Hard Of Hearing ◽  
Experimental Studies ◽  
Design Parameters ◽  
Educational Video ◽  
Word Level ◽  
Sentence Level ◽  
First Occurrence ◽  
Educational Videos

There are style guidelines for authors who highlight important words in static text, e.g., bolded words in student textbooks, yet little research has investigated highlighting in dynamic texts, e.g., captions during educational videos for Deaf or Hard of Hearing (DHH) users. In our experimental study, DHH participants subjectively compared design parameters for caption highlighting, including: decoration (underlining vs. italicizing vs. boldfacing), granularity (sentence level vs. word level), and whether to highlight only the first occurrence of a repeating keyword. In partial contrast to recommendations in prior research, which had not been based on experimental studies with DHH users, we found that DHH participants preferred boldface, word-level highlighting in captions. Our empirical results provide guidance for the design of keyword highlighting during captioned videos for DHH users, especially in educational video genres.

scholarly journals Understanding Syntactic and Semantic Errors in the Composition Writing of Jordanian EFL Learners

2017 ◽  
Vol 6 (6) ◽  
pp. 158
Author(s):  
Yazan Shaker Almahameed ◽  
May Al-Shaikhli
Keyword(s):  
Foreign Language ◽  
Language Learners ◽  
Semantic Knowledge ◽  
Resumptive Pronouns ◽  
Foreign Language Learners ◽  
Word Level ◽  
Sentence Level ◽  
Native Speakers Of English ◽  
Semantic Errors ◽  
Verb Tense

The current study aimed at investigating the salient syntactic and semantic errors made by Jordanian English foreign language learners as writing in English. Writing poses a great challenge for both native and non-native speakers of English, since writing involves employing most language sub-systems such as grammar, vocabulary, spelling and punctuation. A total of 30 Jordanian English foreign language learners participated in the study. The participants were instructed to write a composition of no more than one hundred and fifty words on a selected topic. Essays were collected and analyzed statistically to obtain the needed results. The results of the study displayed that syntactic errors produced by the participants were varied, in that eleven types of syntactic errors were committed as follows; verb-tense, agreement, auxiliary, conjunctions, word order, resumptive pronouns, null-subject, double-subject, superlative, comparative and possessive pronouns. Amongst syntactic errors, verb tense errors were the most frequent with 33%. The results additionally revealed that two types of semantic errors were made; errors at sentence level and errors at word level. Errors at word level outstripped by far errors at sentence level, scoring respectively 82% and 18%. It can be concluded that the syntactic and semantic knowledge of Jordanian learners of English is still insufficient.

scholarly journals Product reputation mining: bring informative review summaries to producers and consumers

2019 ◽  
Vol 16 (2) ◽  
pp. 359-380
Author(s):  
Zhehua Piao ◽  
Sang-Min Park ◽  
Byung-Won On ◽  
Gyu Choi ◽  
Myong-Soon Park
Keyword(s):  
Low Cost ◽  
Three Dimensional ◽  
Word Level ◽  
Sentence Level ◽  
Survey Results ◽  
Statistical Studies ◽  
Sentiment Orientation ◽  
Level Method ◽  
Points Of View ◽  
Graph Based Model

Product reputation mining systems can help customers make their buying decision about a product of interest. In addition, it will be helpful to investigate the preferences of recently released products made by enterprises. Unlike the conventional manual survey, it will give us quick survey results on a low cost budget. In this article, we propose a novel product reputation mining approach based on three dimensional points of view that are word, sentence, and aspect?levels. Given a target product, the aspect?level method assigns the sentences of a review document to the desired aspects. The sentence?level method is a graph-based model for quantifying the importance of sentences. The word?level method computes both importance and sentiment orientation of words. Aggregating these scores, the proposed approach measures the reputation tendency and preferred intensity and selects top-k informative review documents about the product. To validate the proposed method, we experimented with review documents relevant with K5 in Kia motors. Our experimental results show that our method is more helpful than the existing lexicon?based approach in the empirical and statistical studies.

The synergy of double attention: Combine sentence-level and word-level attention for image captioning

2020 ◽  
Vol 201 ◽  
pp. 103068
Author(s):  
Haiyang Wei ◽  
Zhixin Li ◽  
Canlong Zhang ◽  
Huifang Ma
Keyword(s):  
Image Captioning ◽  
Word Level ◽  
Sentence Level

scholarly journals Pushing the Limits of Translation Quality Estimation

2017 ◽  
Vol 5 ◽  
pp. 205-218 ◽  
Author(s):  
André F. T. Martins ◽  
Marcin Junczys-Dowmunt ◽  
Fabio N. Kepler ◽  
Ramón Astudillo ◽  
Chris Hokamp ◽  
...  
Keyword(s):  
Pearson Correlation ◽  
Neural Model ◽  
Quality Estimation ◽  
Translation Quality ◽  
Word Level ◽  
Current State ◽  
Sentence Level ◽  
Feature Based ◽  
Human Effort ◽  
Estimation System

Translation quality estimation is a task of growing importance in NLP, due to its potential to reduce post-editing human effort in disruptive ways. However, this potential is currently limited by the relatively low accuracy of existing systems. In this paper, we achieve remarkable improvements by exploiting synergies between the related tasks of word-level quality estimation and automatic post-editing. First, we stack a new, carefully engineered, neural model into a rich feature-based word-level quality estimation system. Then, we use the output of an automatic post-editing system as an extra feature, obtaining striking results on WMT16: a word-level FMULT1 score of 57.47% (an absolute gain of +7.95% over the current state of the art), and a Pearson correlation score of 65.56% for sentence-level HTER prediction (an absolute gain of +13.36%).

scholarly journals Textual Adversarial Attacking with Limited Queries

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2671
Author(s):  
Yu Zhang ◽  
Junan Yang ◽  
Xiaoshuai Li ◽  
Hui Liu ◽  
Kun Shao
Keyword(s):  
Language Processing ◽  
Main Idea ◽  
Local Model ◽  
Small Perturbations ◽  
Target Model ◽  
Word Level ◽  
Sentence Level ◽  
Adversarial Examples ◽  
Reducing Costs ◽  
The Cost

Recent studies have shown that natural language processing (NLP) models are vulnerable to adversarial examples, which are maliciously designed by adding small perturbations to benign inputs that are imperceptible to the human eye, leading to false predictions by the target model. Compared to character- and sentence-level textual adversarial attacks, word-level attack can generate higher-quality adversarial examples, especially in a black-box setting. However, existing attack methods usually require a huge number of queries to successfully deceive the target model, which is costly in a real adversarial scenario. Hence, finding appropriate models is difficult. Therefore, we propose a novel attack method, the main idea of which is to fully utilize the adversarial examples generated by the local model and transfer part of the attack to the local model to complete ahead of time, thereby reducing costs related to attacking the target model. Extensive experiments conducted on three public benchmarks show that our attack method can not only improve the success rate but also reduce the cost, while outperforming the baselines by a significant margin.

scholarly journals Improving the state-of-the-art in Thai semantic similarity using distributional semantics and ontological information

PLoS ONE ◽  
2021 ◽  
Vol 16 (2) ◽  
pp. e0246751
Author(s):  
Ponrudee Netisopakul ◽  
Gerhard Wohlgenannt ◽  
Aleksei Pulich ◽  
Zar Zar Hlaing
Keyword(s):  
Semantic Similarity ◽  
Language Processing ◽  
English Language ◽  
State Of The Art ◽  
Word Sense Disambiguation ◽  
Similarity Score ◽  
The State ◽  
Word Sense ◽  
Word Level ◽  
High Fraction

Research into semantic similarity has a long history in lexical semantics, and it has applications in many natural language processing (NLP) tasks like word sense disambiguation or machine translation. The task of calculating semantic similarity is usually presented in the form of datasets which contain word pairs and a human-assigned similarity score. Algorithms are then evaluated by their ability to approximate the gold standard similarity scores. Many such datasets, with different characteristics, have been created for English language. Recently, four of those were transformed to Thai language versions, namely WordSim-353, SimLex-999, SemEval-2017-500, and R&G-65. Given those four datasets, in this work we aim to improve the previous baseline evaluations for Thai semantic similarity and solve challenges of unsegmented Asian languages (particularly the high fraction of out-of-vocabulary (OOV) dataset terms). To this end we apply and integrate different strategies to compute similarity, including traditional word-level embeddings, subword-unit embeddings, and ontological or hybrid sources like WordNet and ConceptNet. With our best model, which combines self-trained fastText subword embeddings with ConceptNet Numberbatch, we managed to raise the state-of-the-art, measured with the harmonic mean of Pearson on Spearman ρ, by a large margin from 0.356 to 0.688 for TH-WordSim-353, from 0.286 to 0.769 for TH-SemEval-500, from 0.397 to 0.717 for TH-SimLex-999, and from 0.505 to 0.901 for TWS-65.

scholarly journals Preservation of Javanese Language on Ganjar Pranowo's Conversation in Pandemic of COVID-19

Humaniora ◽  
2021 ◽  
Vol 12 (1) ◽  
pp. 7-12
Author(s):  
Umi Farichah ◽  
Ani Rakhmawati ◽  
Nugraheni Eko Wardani
Keyword(s):  
Role Models ◽  
Primary Data ◽  
Social Sphere ◽  
Central Java ◽  
Video Recordings ◽  
Word Level ◽  
The People ◽  
Sentence Level ◽  
The Social ◽  
Data Source

The research aimed to see a relevance of the preservation of the Javanese language in Javanese conversations that Ganjar Pranowo carried out during the COVID-19 pandemic. The resulting data was about the level in the language that included word level, phrase level, and sentence level. Also, several manners, unggah ungguh, and ethics were produced that could become examples or role models for the people of Central Java. The research applied a qualitative method. The data source was the utterances contained in the uploads of Ganjar Pranowo in the form of video recordings that included primary data in the form of utterances or parts of spoken speech from various speeches and communications from the people of Central Java with Ganjar Pranowo. The results show that preservation of the Javanese language through conversations between leaders and the community has positive implications. This means that the preservation of the Javanese language is carried out optimally in the social sphere. This activity is well recorded and uploaded on social media, Ganjar Pranowo, a figure who has high credibility. The social sphere is an important component used to preserve Javanese language, culture, and traditions.

Sign in / Sign up

Export Citation Format

Share Document