Determinants of Interrater Reliability of Job Analysis: A Meta-analysis

2002 ◽  
Vol 18 (1) ◽  
pp. 52-62 ◽  
Author(s):  
Olga F. Voskuijl ◽  
Tjarda van Sliedregt
Keyword(s):  
Data Collection ◽  
Interrater Reliability ◽  
Meta Analysis ◽  
Job Analysis ◽  
Job Descriptions ◽  
Job Description ◽  
Research And Practice ◽  
Descriptive Information ◽  
Interrater Reliability Coefficients ◽  
Reliability Coefficients

Summary: This paper presents a meta-analysis of published job analysis interrater reliability data in order to predict the expected levels of interrater reliability within specific combinations of moderators, such as rater source, experience of the rater, and type of job descriptive information. The overall mean interrater reliability of 91 reliability coefficients reported in the literature was .59. The results of experienced professionals (job analysts) showed the highest reliability coefficients (.76). The method of data collection (job contact versus job description) only affected the results of experienced job analysts. For this group higher interrater reliability coefficients were obtained for analyses based on job contact (.87) than for those based on job descriptions (.71). For other rater categories (e.g., students, organization members) neither the method of data collection nor training had a significant effect on the interrater reliability. Analyses based on scales with defined levels resulted in significantly higher interrater reliability coefficients than analyses based on scales with undefined levels. Behavior and job worth dimensions were rated more reliable (.62 and .60, respectively) than attributes and tasks (.49 and .29, respectively). Furthermore, the results indicated that if nonprofessional raters are used (e.g., incumbents or students), at least two to four raters are required to obtain a reliability coefficient of .80. These findings have implications for research and practice.

scholarly journals MENATA TUGAS KELOMPOK SADAR WISATA (POKDARWIS) SUKODONO DENGAN PENDEKATAN ANALISIS JABATAN

2020 ◽  
Vol 9 (11) ◽  
pp. 3490
Author(s):  
Erna Resmiatini ◽  
Bramantyo Tri Asmoro
Keyword(s):  
Data Analysis ◽  
Data Collection ◽  
Qualitative Method ◽  
Qualitative Data ◽  
Applied Research ◽  
Job Analysis ◽  
Business Development ◽  
Job Descriptions ◽  
Job Description ◽  
Development Section

This research is an applied research focused on solving problems experienced by the Sukodono Tourism Awareness Group (Pokdarwis). The results of preliminary observations showed that they did not understand the tasks to be carried out. Therefore, this study aims to analyze and compile job descriptions of Pokdarwis Sukodono. The method used is a descriptive qualitative method, with a job analysis approach that adapts the steps of Dessler's (2010) job analysis and qualitative data analysis by Miles & Huberman's model (1994). Meanwhile, the data collection techniques in this study were interviews, observation, and documentation. The result obtained is job description of Pokdarwis Sukodono starting from the Chairman to the Business Development Section. This result is obtained after conducting repeated job analysis to obtain a credible job description. The results of this study bring enormous benefits to the management of Pokdarwis Sukodono, because now the management has a clear duty reference in working as a tourism agent in their area. Keywords: Job Analysis; Job Description; Pokdarwis.

ANALISIS JABATAN DAN PENENTUAN INDIKATOR PRESTASI KERJA DIVISI FINANCE DAN ACCOUNTING PADA PT. SPM

2016 ◽  
Vol 1 (2) ◽  
Author(s):  
Linda Mora Siregar
Keyword(s):  
Working Conditions ◽  
Work Performance ◽  
Job Analysis ◽  
Job Descriptions ◽  
Job Responsibilities ◽  
Job Description ◽  
Employment Relationships ◽  
Job Requirements ◽  
Job Functions ◽  
The Subject

This study aims to determine the work performance of the finance and accounting division at PT. SPM. The subject of this research is the finance and accounting division. The method in this research uses questionnaire and interview techniques. The results show that job descriptions, job requirements and employee KPIs that are in line with current company conditions can be compiled and determined. Job / job description contains all information regarding job identity, job functions, job authority, duties and main job responsibilities, other tasks, work documents, working conditions, and employment relationships. Keywords: Job Analysis, Work Performance Indocators Penelitian ini bertujuan untuk mengetahui prestasi kerja divisi finance dan accounting pada PT. SPM. Subjek penelitian ini adalah divisi finance dan accounting. Metode dalam penelitin ini menggunakan teknik koesioner dan wawancara. Hasil penelitian menunjukkan bahwa uraian pekerjaan, persyaratan jabatan dan KPI karyawan yang sesuai dengan kondisi perusahaan saat ini dapat disusun dan ditetapkan.Uraian pekerjaan/jabatan memuat semua informasi mengenai identitas jabatan, fungsi jabatan, wewenang jabatan, tugas dan tanggung jawab pokok pekerjaan, tugas-tugas lain, dokumen kerja, kondisi kerja, dan hubungan kerja.   Kata Kunci: Analisi Jabatan, Indikator Prestasi Kerja.

Interrater Agreement on a Socially Valid Measure of Students' Creativity

1987 ◽  
Vol 61 (3) ◽  
pp. 1009-1010 ◽  
Author(s):  
Mark A. Runco
Keyword(s):  
Interrater Reliability ◽  
Interrater Agreement ◽  
True Score ◽  
Valid Measure ◽  
True Variance ◽  
Interrater Reliability Coefficients ◽  
Reliability Coefficients

The reliability and true variance of a socially valid measure of creativity was assessed by asking three judges to rate the creativity of 29 adolescents. Interitem reliability was .93; interrater reliability was .48; and true score variance, estimated from the interitem and interrater reliability coefficients, was .65.

Percent of Agreement among Raters and Rater Reliability of the Copying Subtest of the Stanford-Binet Intelligence Scale: Fourth Edition

1992 ◽  
Vol 74 (2) ◽  
pp. 347-353 ◽  
Author(s):  
Elizabeth M. Mason
Keyword(s):  
Interrater Reliability ◽  
Fourth Edition ◽  
Cognitive Measures ◽  
Intelligence Scale ◽  
Technical Manual ◽  
Subjective Judgement ◽  
Visual Motor ◽  
Test Retest Reliability ◽  
Interrater Reliability Coefficients ◽  
Reliability Coefficients

The purpose of this study was to investigate the interrater reliability of the visual-motor portion of the Copying subtest of the Stanford-Binet Intelligence Scale: Fourth Edition. Eight raters independently scored 11 protocols completed by children aged 5 through 10 years, using the scoring criteria and guidelines in the manual. The raters marked each of 10 items pass or fail and computed a total raw score for each protocol. Interrater reliability coefficients were obtained for each child's protocol, and the Kappa coefficient was computed for each item. Significant raters' reliability coefficients ranged from .82 to .91, which were low in comparison to test-retest reliability and Kuder-Richardson-20 coefficients for this and other subtests of the Stanford-Binet in the technical manual. Percent agreement among 8 raters also indicated weak reliability. Although the obtained results suggested some interrater reliability coefficients within acceptable levels, questions were raised about the scoring criteria for individual items. Caution is warranted in the use of cognitive measures which include subjective judgement of the examiner in applying scoring criteria.

Reliability of Global Impressions for Assessing Methylphenidate Effects in Children with Attention-Deficit Hyperactivity Disorder

1993 ◽  
Vol 77 (3_suppl) ◽  
pp. 1215-1218 ◽  
Author(s):  
Susan Dickerson Mayes ◽  
Edward O. Bixler
Keyword(s):  
Attention Deficit Hyperactivity Disorder ◽  
Attention Deficit ◽  
Interrater Reliability ◽  
Double Blind ◽  
Double Blind Placebo ◽  
Hyperactivity Disorder ◽  
Cohen's Kappa ◽  
The Individual ◽  
Interrater Reliability Coefficients ◽  
Reliability Coefficients

Agreement between raters using global impressions to assess methylphenidate response was analyzed for children with Attention-Deficit Hyperactivity Disorder (ADHD) undergoing double-blind, placebo-controlled, crossover methylphenidate trials. Caregivers were more likely to disagree than agree when asked to rate the children as “better, same, or worse” during each day of the trial. Over-all agreement was 42.9%, only 9.6% above what would be expected based on chance alone. Further, none of the interrater reliability coefficients (Cohen's kappa) for the individual children were statistically significant.

scholarly journals Conceptualizing researchers’ perspectives on involving professionals in research: a group concept mapping study

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Christine E. Laustsen ◽  
Albert Westergren ◽  
Pia Petersson ◽  
Maria Haak
Keyword(s):  
Data Collection ◽  
Concept Mapping ◽  
Hierarchical Cluster ◽  
Research Process ◽  
Scaling Analysis ◽  
Research Projects ◽  
Research And Practice ◽  
Group Concept Mapping ◽  
Process Adaptation ◽  
Group Concept

Abstract Background Researchers have shown an increased interest in involving professionals from outside academia in research projects. Professionals are often involved in research on ageing and health when the purpose is to address the gap between research and practice. However, there is a need to acquire more knowledge about what the involvement might lead to by exploring researchers’ experiences of involving professionals in research on ageing and health and developing conceptual areas. Therefore, the aim of this study was to identify conceptual areas of professionals’ involvement in research on ageing and health, from the perspective of researchers themselves. Methods Group concept mapping, a participatory and mixed method, was used to conceptualize areas. Researchers with experience of involving professionals in research projects on ageing and health participated in qualitative data collection through brainstorming sessions (n = 26), and by sorting statements (n = 27). They then took part in quantitative data collection, where they rated statements according to how much a statement strengthened research (n = 26) and strengthened practice (n = 24). Data were analysed using multidimensional scaling analysis and hierarchical cluster analysis. In addition, a qualitative analysis of the latent meaning of the cluster map was conducted. Results Analysis of the sorting stage generated five clusters illustrating conceptual areas of professionals’ involvement in research projects on ageing and health. The five clusters are as follows: complex collaboration throughout the research process; adaptation of research to different stakeholders, mutual learning through partnership; applicable and sustainable knowledge; legitimate research on ageing and health. The qualitative latent meaning of the cluster map showed two themes: the process of involvement and the outcome of involvement. A positive strong correlation (0.87) was found between the rating of strengthened research and practice. Conclusions This study reveals conceptual areas on a comprehensive and illustrative map which contributes to the understanding of professionals’ involvement in research on ageing and health. A conceptual basis for further studies is offered, where the aim is to investigate the processes and outcomes entailed in involving professionals in research on ageing and health. The study also contributes to the development of instruments and theories for optimizing the involvement of professionals in research.

Sign in / Sign up

Export Citation Format

Share Document