scholarly journals Towards Evaluating Narrative Quality In Student Writing

2018 ◽  
Vol 6 ◽  
pp. 91-106 ◽  
Author(s):  
Swapna Somasundaran ◽  
Michael Flor ◽  
Martin Chodorow ◽  
Hillary Molloy ◽  
Binod Gyawali ◽  
...  
Keyword(s):  
Scoring System ◽  
State Of The Art ◽  
Student Writing ◽  
Narrative Writing ◽  
Automated Scoring ◽  
Linguistic Features

This work lays the foundation for automated assessments of narrative quality in student writing. We first manually score essays for narrative-relevant traits and sub-traits, and measure inter-annotator agreement. We then explore linguistic features that are indicative of good narrative writing and use them to build an automated scoring system. Experiments show that our features are more effective in scoring specific aspects of narrative quality than a state-of-the-art feature set.

scholarly journals An Evaluation of China’s Automated Scoring System Bingo English

2020 ◽  
Vol 10 (6) ◽  
pp. 30
Author(s):  
Jianmin Gao ◽  
Xin Li ◽  
Peiqi Gu ◽  
Ziqi Liu
Keyword(s):  
Scoring System ◽  
Automated Scoring ◽  
Linguistic Features ◽  
Automated Essay Scoring ◽  
Content Quality ◽  
Chinese University ◽  
Essay Scoring

The study evaluated the effectiveness of Bingo English, one of the representative automated essay scoring (AES) systems in China. 84 essays in an English test held in a Chinese university were collected as the research materials. All the essays were scored by both two trained and experienced human raters and Bingo English, and the linguistic features of them were also quantified in terms of complexity, accuracy, fluency (CAF), content quality, and organization. After examining the agreement between human scores and automated scores and the correlation of human and automated scores with the indicators of the essays’ linguistic features, it was found that Bingo English scores could only reflect the essays’ quality in a general way, and the use of it should be treated with caution.

scholarly journals Comparative Study on Feature-Based Scoring Using Vector Space Modelling System

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Tarandeep Singh Walia ◽  
Tarek Frikha ◽  
Omar Cheikhrouhou ◽  
Habib Hamam
Keyword(s):  
Scoring System ◽  
State Of The Art ◽  
Feature Analysis ◽  
Similarity Function ◽  
Semantic Features ◽  
Automated Scoring ◽  
Rule Based ◽  
Feature Based ◽  
Semantic Feature Analysis ◽  
Better Than

This paper shows the importance of automated scoring (AS) and that it is better than human graders in terms of degree of reproducibility. Considering the potential of the automated scoring system, there is further a need to refine and develop the existing system. The paper goes through the state of the art. It presents the results concerning the problems of existing systems. The paper also presents the semantic features that are indispensable in the scoring system as they have complete content. Moreover, in the present research, a huge deviation has been exhibited by the system which has been shown later in performance analysis of the study, and this clearly indicates the novelty and improved results of the system. It explains the algorithms included in the methodology of this proposed system. The novelty of our work consists in the use of its own similarity function and its notation mechanism. It does not use the cosine similarity function between two vectors. This paper describes and develops a more accurate system which employs a statistical method for scoring. This system adopts and integrates rule-based semantic feature analysis.

scholarly journals A Study of Applying Automated Assessment in Teaching College English Writing Based on Juku Correction Network

2019 ◽  
Vol 14 (11) ◽  
pp. 19
Author(s):  
Yusheng Wang
Keyword(s):  
Scoring System ◽  
Teaching Model ◽  
Network Technology ◽  
Automated Scoring ◽  
Teaching Mode ◽  
College English ◽  
English Writing ◽  
Writing Motivation ◽  
Teaching College ◽  
Writing Course

With the continuous advancement of modern network technology, the drawbacks of the tradition-al English writing course teaching mode have become increasingly prominent, and the automated scoring system has gradually been used in the writing course. This paper proposes a college English writing teaching model based on Juku Correction Network, and conducts empirical re-search on the use of Juku Correction Network in college English writing teaching. The research results show that the teaching mode based on Juku Correction Network can effectively improve the overall level of students' English writing, and stimulate students' English writing motivation.

scholarly journals Comparing Deep-Learning Architectures and Traditional Machine-Learning Approaches for Satire Identification in Spanish Tweets

Mathematics ◽  
2020 ◽  
Vol 8 (11) ◽  
pp. 2075
Author(s):  
Óscar Apolinario-Arzube ◽  
José Antonio García-Díaz ◽  
José Medina-Moreira ◽  
Harry Luna-Aveiga ◽  
Rafael Valencia-García
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
User Interfaces ◽  
State Of The Art ◽  
Learning Approaches ◽  
Word Embeddings ◽  
Linguistic Features ◽  
Intended Meaning ◽  
Language User ◽  
Learning Architectures

Automatic satire identification can help to identify texts in which the intended meaning differs from the literal meaning, improving tasks such as sentiment analysis, fake news detection or natural-language user interfaces. Typically, satire identification is performed by training a supervised classifier for finding linguistic clues that can determine whether a text is satirical or not. For this, the state-of-the-art relies on neural networks fed with word embeddings that are capable of learning interesting characteristics regarding the way humans communicate. However, as far as our knowledge goes, there are no comprehensive studies that evaluate these techniques in Spanish in the satire identification domain. Consequently, in this work we evaluate several deep-learning architectures with Spanish pre-trained word-embeddings and compare the results with strong baselines based on term-counting features. This evaluation is performed with two datasets that contain satirical and non-satirical tweets written in two Spanish variants: European Spanish and Mexican Spanish. Our experimentation revealed that term-counting features achieved similar results to deep-learning approaches based on word-embeddings, both outperforming previous results based on linguistic features. Our results suggest that term-counting features and traditional machine learning models provide competitive results regarding automatic satire identification, slightly outperforming state-of-the-art models.

scholarly journals Finding Convincing Arguments Using Scalable Bayesian Preference Learning

2018 ◽  
Vol 6 ◽  
pp. 357-371 ◽  
Author(s):  
Edwin Simpson ◽  
Iryna Gurevych
Keyword(s):  
State Of The Art ◽  
Training Data ◽  
Preference Learning ◽  
Word Embeddings ◽  
Inference Method ◽  
Linguistic Features ◽  
Bayesian Approaches ◽  
Stochastic Variational Inference ◽  
Crowdsourced Data ◽  
Previous State

We introduce a scalable Bayesian preference learning method for identifying convincing arguments in the absence of gold-standard ratings or rankings. In contrast to previous work, we avoid the need for separate methods to perform quality control on training data, predict rankings and perform pairwise classification. Bayesian approaches are an effective solution when faced with sparse or noisy training data, but have not previously been used to identify convincing arguments. One issue is scalability, which we address by developing a stochastic variational inference method for Gaussian process (GP) preference learning. We show how our method can be applied to predict argument convincingness from crowdsourced data, outperforming the previous state-of-the-art, particularly when trained with small amounts of unreliable data. We demonstrate how the Bayesian approach enables more effective active learning, thereby reducing the amount of data required to identify convincing arguments for new users and domains. While word embeddings are principally used with neural networks, our results show that word embeddings in combination with linguistic features also benefit GPs when predicting argument convincingness.

Development of a Fluorescence-Based Caries Scoring System for an Intraoral Scanner: An in vitro Study

2020 ◽  
Vol 54 (4) ◽  
pp. 324-335
Author(s):  
Stavroula Michou ◽  
Ana Raquel Benetti ◽  
Christoph Vannahme ◽  
Pétur Gordon Hermannsson ◽  
Azam Bakhshandeh ◽  
...  
Keyword(s):  
Scoring System ◽  
State Of The Art ◽  
Assessment System ◽  
Detection Methods ◽  
Caries Detection ◽  
Fluorescence Signal ◽  
Radiographic Assessment ◽  
Intraoral Scanner ◽  
Posterior Teeth ◽  
Occlusal Surfaces

<b><i>Objectives:</i></b> To develop an automated fluorescence-based caries scoring system for an intraoral scanner and to<i></i>test the performance of the system compared to state-of-the-art methods. <b><i>Methods:</i></b> Seventy-three permanent posterior teeth were scanned with a three-dimensional (3D) intraoral scanner prototype which emitted light at 415 nm. An overlay representing the fluorescence signal from the tissue was mapped onto 3D models of the teeth. Multiple examination sites (<i>n</i> = 139) on the occlusal surfaces were chosen, and their red and green fluorescence signal components were extracted. These components were used to calculate 4 mathematical functions upon which a caries scoring system for the scanner prototype could be based. Visual-tactile (International Caries Detection and Assessment System, ICDAS), radiographic (ICDAS), and histological assessments were conducted on the same examination sites. <b><i>Results:</i></b> Most index tests showed significant correlation with histology. The strongest correlation was observed for the visual-tactile examination (<i>r</i><sub>s</sub> = 0.80) followed by the scanner supported by the caries classification function that quantifies the overall fluorescence compared to sound surfaces (<i>r</i><sub>s</sub> = 0.78). Additionally, this function resulted in the highest intra-examiner reliability (κ = 0.964), and the highest sum of sensitivity (SE) and specificity (SP) (sum SE-SP: 1.60–1.84) at the 2 histological levels where the comparison with visual-tactile assessment was possible (κ = 0.886, sum SE-SP = 1.57–1.81) and at the 3 out of 4 histological levels where the comparison with radiographic assessment was possible (κ = 0.911, sum SE-SP = 1.37–1.78); the only exception was for the lesions in the outer third of dentin, where the radiographic assessment showed the highest sum SE-SP (1.78). <b><i>Conclusion:</i></b> A fluorescence-based caries scoring system was developed for the intraoral scanner showing promising performance compared to state-of-the-art caries detection methods. The intraoral scanner accompanied by an automated caries scoring system may improve objective caries detection and increase the efficiency and effectiveness of oral examinations. Furthermore, this device has the potential to support reliable monitoring of early caries lesions.

Sign in / Sign up

Export Citation Format

Share Document