Consistency, Inter-rater Reliability, and Validity of 441 Consecutive Mock Oral Examinations in Anesthesiology 

1999 ◽  
Vol 91 (1) ◽  
pp. 288-298 ◽  
Author(s):  
Armin Schubert ◽  
John E. Tetzlaff ◽  
Ming Tan ◽  
Victor J. Ryckman ◽  
Edward Mascha
Keyword(s):  
Internal Consistency ◽  
Clinical Judgment ◽  
Assessment Tool ◽  
Reliability And Validity ◽  
Rater Reliability ◽  
Faculty Evaluations ◽  
Oral Practice ◽  
Oral Examinations ◽  
Relationship Of ◽  
The Relationship

Background Oral practice examinations (OPEs) are used extensively in many anesthesiology programs for various reasons, including assessment of clinical judgment. Yet oral examinations have been criticized for their subjectivity. The authors studied the reliability, consistency, and validity of their OPE program to determine if it was a useful assessment tool. Methods From 1989 through 1993, we prospectively studied 441 OPEs given to 190 residents. The examination format closely approximated that used by the American Board of Anesthesiology. Pass-fail grade and an overall numerical score were the OPE results of interest. Internal consistency and inter-rater reliability were determined using agreement measures. To assess their validity in describing competence, OPE results were correlated with in-training examination results and faculty evaluations. Furthermore, we analyzed the relationship of OPE with implicit indicators of resident preparation such as length of training. Results The internal consistency coefficient for the overall numerical score was 0.82, indicating good correlation among component scores. The interexaminer agreement was 0.68, indicating moderate or good agreement beyond that expected by chance. The actual agreement among examiners on pass-fail was 84%. Correlation of overall numerical score with in-training examination scores and faculty evaluations was moderate (r = 0.47 and 0.41, respectively; P < 0.01). OPE results were significantly (P < 0.01) associated with training duration, previous OPE experience, trainee preparedness, and trainee anxiety. Conclusion Our results show the substantial internal consistency and reliability of OPE results at a single institution. The positive correlation of OPE scores with in-training examination scores, faculty evaluations, and other indicators of preparation suggest that OPEs are a reasonably valid tool for assessment of resident performance.

scholarly journals Validity and reliability of Korean version Competency Assessment Tool-Mental Health

2020 ◽  
Vol 7 (2) ◽  
pp. 143-152
Author(s):  
Hyun Mee Cho ◽  
Jeong Won Han ◽  
Eun Joung Choi ◽  
Hyo Eun Jeong ◽  
Bo Ram Hong ◽  
...  
Keyword(s):  
Mental Health ◽  
Health Care ◽  
Mental Health Care ◽  
Internal Consistency ◽  
Assessment Tool ◽  
Reliability And Validity ◽  
Basic Data ◽  
Measurement Tool ◽  
Competency Assessment ◽  
Validity And Reliability

AbstractObjectiveTo establish basic data for identifying the level of mental health care competency of general nurses through verification of the reliability and validity of Competency Assessment Tool-Mental Health in the development of a measurement tool for mental health care competency in Korea.MethodsThis study was conducted on nurses working at general wards, excluding those working at the Department of Psychiatry, in five hospitals with 200 beds or more located in Korea. Content, construct, concurrent validity, and internal consistency of the measurement were confirmed.ResultsAs a result of the construct validity, the section on importance of skills and knowledge for mental health care had 21 items, whereas the section on benefits of additional education had 22 items. The internal consistency of measurement was confirmed as follows: Cronbach's α = 0.96 for the section on importance and 0.96 for the section on benefits section.ConclusionsThis study verified the high validity and reliability of the tool in assessing the mental health care competency of nurses, and it is believed to be significant as basic data for enhancing such competency.

A Cross-Cultural Measure of Servant Leadership Behaviors

2013 ◽  
pp. 152-163
Author(s):  
Jeff R. Hale ◽  
Dail Fields
Keyword(s):  
United States ◽  
Servant Leadership ◽  
Leadership Behaviors ◽  
Reliability And Validity ◽  
The United States ◽  
Validity Evidence ◽  
Key Dimensions ◽  
Relationship Of ◽  
The Relationship ◽  
Cultural Measure

This chapter presents items comprising three scales that measure servant leadership using three key dimensions: service, humility, and vision. The instrument was used to measure servant leadership behaviors experienced by followers in the United States and Ghana. Reliability and validity evidence is included from two research studies. A discussion of the relationship of servant leadership behaviors with employee outcomes assessed in these studies concludes the chapter.

scholarly journals A study of the reliability and validity of the Chinese version of the Nociception Coma Scale–Revised

2020 ◽  
Vol 34 (8) ◽  
pp. 1112-1121
Author(s):  
Jing Wang ◽  
Haibo Di ◽  
Wen Hua ◽  
Liwen Cheng ◽  
Zhigang Xia ◽  
...  
Keyword(s):  
Internal Consistency ◽  
Concurrent Validity ◽  
Reliability And Validity ◽  
Disorders Of Consciousness ◽  
Chinese Version ◽  
Rater Reliability ◽  
Validity Data ◽  
Retest Reliability ◽  
Coma Scale ◽  
Test Retest Reliability

Objective: The aim of the study was to check on the reliability and validity of the translated version of Nociception Coma Scale–Revised. Design: Prospective psychometric study. Setting: Rehabilitation and neurology unit in hospital. Subjects: Patients with prolonged disorders of consciousness. Interventions: None. Main measures: The original English version of the Nociception Coma Scale–Revised was translated into Chinese. The reliability and validity were undertaken by trained raters. Intraclass correlation coefficients were used to assess inter-rater reliability and test–retest reliability. Cronbach’s alpha test was used to investigate internal consistency. Spearman’s correlation was used to calculate concurrent validity. The Coma Recovery Scale–revised was used to assess the consciousness of patients. Results: Eighty-four patients were enrolled in the study. Inter-rater reliability of the Chinese version of Nociception Coma Scale–Revised was high for total scores and motor and verbal subscores and good for facial subscores. Test–retest reliability was high for total score and for all subscores. Analysis revealed a moderate internal consistency for subscores. For the concurrent validity, a strong correlation was found between the Nociception Coma Scale–Revised and the Face, Legs, Activity, Cry, and Consolability behavioral scale for all patients. A moderate correlation was found between the Nociception Coma Scale–Revised and the Coma Recovery Scale–revised scores for all patients. Conclusion: The Chinese version of Nociception Coma Scale–Revised has good reliability and validity data for assessing responses to pain in patients with prolonged disorders of consciousness.

scholarly journals The Scarbase Duo ® : Intra-rater and inter-rater reliability and validity of a compact dual scar assessment tool

Burns ◽  
2016 ◽  
Vol 42 (2) ◽  
pp. 336-344 ◽  
Author(s):  
Matthew Fell ◽  
Jill Meirte ◽  
Mieke Anthonissen ◽  
Koen Maertens ◽  
Jonathon Pleat ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Reliability And Validity ◽  
Rater Reliability ◽  
Scar Assessment

Variation in Cephalometric Reference Lines

1981 ◽  
Vol 8 (4) ◽  
pp. 183-187 ◽  
Author(s):  
T. D. Foster ◽  
A. P. Howat ◽  
P. J. Naish
Keyword(s):  
Clinical Judgment ◽  
Head Position ◽  
The Other ◽  
Cephalometric Analysis ◽  
Head Posture ◽  
Positioning Method ◽  
Relationship Of ◽  
The Relationship ◽  
Range Of Variation

The use of intracephalic reference lines in cephalometric assessment is valid only if these lines bear some constant relationship to the true horizontal and to each other. Otherwise, the use of a true horizontal would give the benefit that cephalometric assessment would be based on the same standards as clinical judgment. In using a true horizontal, however, the radiograph must be taken with the head in a reproducible natural posture. A study was carried out to assess the reproducibility of a natural head posture using a simple positioning method in a standard cephalostat, and to assess the variation in the relationship of the sella-nasion, Frankfort, maxillary and mandibular reference lines, to a true horizontal and to each other. Though in their mean inclinations the Frankfort and the maxillary reference lines were almost parallel to each other and to the true horizontal, the range of variation for all the reference lines was considerable. The range of variation in the reference lines was greater than the error of reproduction of head position, and the range of variation of the slope of the mandibular line was no greater than that for the slope of each of the other three lines. The sella-nasion line was reproduced more accurately that the other reference lines, using a digitizer. The possibilities of a simple cephalometric analysis based on a true horizontal are outlined.

scholarly journals Creating a healthy eating and active environment survey (CHEERS) for childcare: an inter-rater, intra-rater reliability and validity study

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Lynne M. Z. Lafave
Keyword(s):  
Physical Activity ◽  
Internal Consistency ◽  
Healthy Eating ◽  
Concurrent Validity ◽  
Reliability And Validity ◽  
Phase 2 ◽  
Audit Tool ◽  
Rater Reliability ◽  
Physical Activity Environment ◽  
Nutrition And Physical Activity

Abstract Background The CHEERS is a self-administered tool to measure gaps, weaknesses, and strengths of an early childhood education and care (ECEC) centre-based nutrition and physical activity environment. ECEC settings have the potential to profoundly influence early dietary and physical activity behaviours. Content validation of the CHEERS tool has been previously reported. The purpose of this study was to develop reliability and validity evidence for the CHEERS audit tool and the proposed subscales of food served, healthy eating environment, program planning, and physical activity environment in ECEC centre-based programs. Methods This cross-sectional study consisted of 2 phases: Phase 1 included inter-, intra-rater and Cronbach’s α. A subset of this sample was invited to participate in a second survey (Trial 2) for intra-rater assessment within 3 weeks of completing the first survey (Trial 1). Phase 2 included concurrent validity assessment between a nutrition expert and the ECEC director using within a one-week period. Results One hundred two directors and 85 educators (total of 187) returned the survey. Of these, there were 75 matched pairs for inter-rater reliability analysis providing a CHEERS ICC score of 0.59 and ICC scores ranging from 0.40 to 0.58 for the subscales. The ICC for intra-rater reliability of the CHEERS score was 0.81 for 40 participants completing the survey a second time and a range of 0.72 to 0.79 for the subscales. The CHEERS tool demonstrated very good internal consistency (α = 0.91) and α scores ranging from 0.73 to 0.79 for the subscales. In phase 2, concurrent validation was ICC = 0.65 (n = 30) CHEERS scores with a range of 0.42 to 0.69 for the subscales. Conclusions This study provides evidence of inter-, intra-rater reliability, internal consistency, and concurrent validity of an environmental assessment audit tool to assess the nutrition and physical activity environment of ECEC centre-based programs. The results demonstrate that the self-administered CHEERS instrument is stable overtime and between evaluators at the same ECEC centre. The scores obtained with CHEERS self-administered audit tool are reasonably accurate compared to an expert rater (dietitian) assessment. This study adds additional support to establishing the psychometric soundness of the CHEERS tool.

scholarly journals Development and reliability of the Korean version of the Feeding Abilities Assessment

2019 ◽  
Vol 32 (1) ◽  
pp. 69-74
Author(s):  
Seul Gi Koo ◽  
Hae Yean Park ◽  
Jongbae Kim ◽  
Areum Han
Keyword(s):  
Correlation Coefficient ◽  
Internal Consistency ◽  
Content Validity ◽  
Assessment Tool ◽  
High Reliability ◽  
Intraclass Correlation ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Korean Version ◽  
Test Retest Reliability

Objective The purpose of this study is to introduce a standardised assessment tool by verifying the reliability of the translated Korean version of the Feeding Abilities Assessment (K-FAA), which was developed to suit Korean culture. Methods The research subjects were 65 patients with dementia living in nursing homes. The K-FAA was completed by verifying the suitability of translation and reverse translation. The validity of the K-FAA was established through content validity, while its reliability was analysed based on internal consistency reliability for the items, test–retest reliability and inter-rater reliability. Results The content validity index determined, based on the assessment of professors, occupational therapists, and nurses, was more than .70. Cronbach’s α was more than .929, showing good internal consistency. A test–retest reliability of .884 was derived using Pearson’s correlation coefficient (p < .01), and an inter-rater reliability of .800 was derived using the kappa coefficients; intraclass correlation coefficient was .897, which also indicated good reliability. Conclusion The K-FAA was modified to fit the Korean domestic situation, and this assessment had high reliability. Therefore, K-FAA can evaluate the feeding ability of patients with dementia. Future studies should focus on providing evidence-based data to maintain or supplement the feeding ability of patients with dementia in Korea.

Intention to Use Self-Service Technology: Blitzmegaplex Jakarta

2015 ◽  
Vol 21 (4) ◽  
pp. 740-744
Author(s):  
Adilla Anggraeni ◽  
Mandayu Victory Soepraptoyo
Keyword(s):  
Service Provider ◽  
Perceived Control ◽  
Reliability And Validity ◽  
Intention To Use ◽  
Competitive Advantages ◽  
Self Service ◽  
Self Service Technology ◽  
And Control ◽  
Relationship Of ◽  
The Relationship

Objective—Due to the advancement of technology, the service delivery process has changed in various industries. The service process has been utilized as a means to create competitive advantages by many industries. The main purpose of this research is to explore the perceptions of non-users of self-service technology towards the potential convenience and control benefits of using the technology and how they influence the non-users’ overall perceptions of speed of transaction, trust in the service provider and future intentions to use the self-service technology. Design/Methodology/Approach—Cronbach’s Alpha and Confirmatory Factorial Analyses were utilized to determine reliability and validity of the constructs. Multiple linear regression and mediation analysis were used to determine whether there were any relationships between the variables. Result and Conclusion—The result concluded that both the perceived control and perceived convenience act as mediators between the relationship of speed of transaction, and trust in a service provider to intention to use self-service technology. The findings also show that both perceived control and perceived convenience can directly influence an individual’s intention to use self-service technology. Practical Implications—The findings suggest that it is important for companies to offer self-service technology that can increase customer’s perception of control and convenience as those characteristics can affect the customers’ intention to use self-service technology in the future. It is also crucial to emphasize the speed of transaction and trust in the service provider as both factors have a mediating impact between customers’ perceived control and perceived convenience towards their intentions to use self-service technology.

Sign in / Sign up

Export Citation Format

Share Document