scholarly journals Instruction Task for Malay Phrase Boundary Annotation

2018 ◽  
Vol 7 (3.15) ◽  
pp. 137
Author(s):  
Haslizatul Mohamed Hanum ◽  
Nur Atiqah Sia Abdullah ◽  
Zainab Abu Bakar
Keyword(s):  
Rater Agreement ◽  
Continuous Speech ◽  
Phrase Boundary ◽  
Prosodic Phrase

The paper presents a refined instruction task to assist evaluation of prosodic phrase (PPh) boundaries by naive listeners. The results from the perceptual experiments were compared to the boundaries produced by online automatic tagger. The Kappa evaluation shows the average of 85% on inter-rater agreement. More than 60% of the boundaries which are detected by the automatic tagger matched the reference boundaries, showing that the refined instruction task can be used to evaluate perception on phrase boundaries on continuous speech.  

An Alternative to Cohen's κ

2006 ◽  
Vol 11 (1) ◽  
pp. 12-24 ◽  
Author(s):  
Alexander von Eye
Keyword(s):  
Simulation Study ◽  
Null Hypothesis ◽  
Categorical Variables ◽  
Alternative Measure ◽  
Rater Agreement ◽  
Verbal Processing ◽  
Heavy Tailed ◽  
Applicant Selection

At the level of manifest categorical variables, a large number of coefficients and models for the examination of rater agreement has been proposed and used. The most popular of these is Cohen's κ. In this article, a new coefficient, κ s , is proposed as an alternative measure of rater agreement. Both κ and κ s allow researchers to determine whether agreement in groups of two or more raters is significantly beyond chance. Stouffer's z is used to test the null hypothesis that κ s = 0. The coefficient κ s allows one, in addition to evaluating rater agreement in a fashion parallel to κ, to (1) examine subsets of cells in agreement tables, (2) examine cells that indicate disagreement, (3) consider alternative chance models, (4) take covariates into account, and (5) compare independent samples. Results from a simulation study are reported, which suggest that (a) the four measures of rater agreement, Cohen's κ, Brennan and Prediger's κ n , raw agreement, and κ s are sensitive to the same data characteristics when evaluating rater agreement and (b) both the z-statistic for Cohen's κ and Stouffer's z for κ s are unimodally and symmetrically distributed, but slightly heavy-tailed. Examples use data from verbal processing and applicant selection.

538-P: Inter-Rater Agreement of Vibration and Pressure Sensation in the Assessment of Sensorimotor Dysfunction in Diabetes

Diabetes ◽  
2020 ◽  
Vol 69 (Supplement 1) ◽  
pp. 538-P
Author(s):  
EDWARD B. JUDE ◽  
ANASTASIOS TENTOLOURIS ◽  
IOANNA ELEFTHERIADOU ◽  
NIKOLAOS TENTOLOURIS
Keyword(s):  
Rater Agreement ◽  
Pressure Sensation
Sign in / Sign up

Export Citation Format

Share Document