Development and Initial Validation of a Project-Based Rubric to Assess the Systems-Based Practice Competency of Residents in the Clinical Chemistry Rotation of a Pathology Residency

2014 ◽  
Vol 138 (6) ◽  
pp. 809-813
Author(s):  
Carolyn R. Vitek ◽  
Jane C. Dale ◽  
Henry A. Homburger ◽  
Sandra C. Bryant ◽  
Amy K. Saenger ◽  
...  
Keyword(s):  
Critical Thinking ◽  
Interrater Reliability ◽  
Clinical Chemistry ◽  
Core Competencies ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Thinking Skills ◽  
Project Evaluation ◽  
Critical Thinking Skills

Context.— Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature. Objective.— To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry. Design.— Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects. Results.— Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability. Conclusions.— We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Validation of Sherouk's Critical Thinking Test (SH-CTT)

2017 ◽  
Vol 8 (3) ◽  
pp. 14-24
Author(s):  
Sherouk J. Kadhm
Keyword(s):  
Critical Thinking ◽  
Reliability And Validity ◽  
Thinking Skills ◽  
Critical Thinking Skills ◽  
Arabic Version ◽  
Leadership Selection ◽  
Response Options ◽  
Academic Environments ◽  
Latent Constructs ◽  
Confirmatory Factor

This study aimed to examine the psychometric properties (reliability and validity) of the Arabic version of Sherouk's Critical Thinking Test. This test has four parts, each of which provides a story that is divided into an introduction and a scene; each story is then followed by a list of sensitive questions featuring two response options (Agree/Disagree). A sample of 158 university students participated in this study. The results showed that the reliability of the entire scale was 0.885, and the reliability of each of its four parts was 0.885, 0.829, 0.772, and 0.721, respectively. Confirmatory factor analysis of the collected data revealed a good fit with the latent constructs (CMIN/DF: 1.446; CMIN/DF: 1.357; CMIN/DF: 1.417; and CMIN/DF: 1.151) for parts one to four, respectively. The test's adaptability was considered when designing this instrument, and examinees were invited to review it. This paper recommended using this instrument in the fields of performance, leadership, selection, training, and improvement, and it was also created as a tool to measure individuals' critical thinking skills and abilities in organizations, companies, and academic environments. The English and German versions of this test were also examined; these versions are now currently available for use.

Development and validation of an instrument to measure undergraduate chemistry students’ critical thinking skills

2020 ◽  
Vol 21 (1) ◽  
pp. 62-78
Author(s):  
Stephen M. Danczak ◽  
Christopher D. Thompson ◽  
Tina L. Overton
Keyword(s):  
Critical Thinking ◽  
Convergent Validity ◽  
Chemistry Education ◽  
Reliability And Validity ◽  
Thinking Skills ◽  
Critical Thinking Skills ◽  
Internal Reliability ◽  
Validity Testing ◽  
Chemistry Students ◽  
Undergraduate Chemistry

The importance of developing and assessing student critical thinking at university can be seen through its inclusion as a graduate attribute for universities and from research highlighting the value employers, educators and students place on demonstrating critical thinking skills. Critical thinking skills are seldom explicitly assessed at universities. Commercial critical thinking assessments, which are often generic in context, are available. However, literature suggests that assessments that use a context relevant to the students more accurately reflect their critical thinking skills. This paper describes the development and evaluation of a chemistry critical thinking test (the Danczak–Overton–Thompson Chemistry Critical Thinking Test or DOT test), set in a chemistry context, and designed to be administered to undergraduate chemistry students at any level of study. Development and evaluation occurred over three versions of the DOT test through a variety of quantitative and qualitative reliability and validity testing phases. The studies suggest that the final version of the DOT test has good internal reliability, strong test–retest reliability, moderate convergent validity relative to a commercially available test and is independent of previous academic achievement and university of study. Criterion validity testing revealed that third year students performed statistically significantly better on the DOT test relative to first year students, and postgraduates and academics performed statistically significantly better than third year students. The statistical and qualitative analysis indicates that the DOT test is a suitable instrument for the chemistry education community to use to measure the development of undergraduate chemistry students’ critical thinking skills.

scholarly journals Alberta Infant Motor Scale: Reliability and Validity When Used on Preterm Infants in Taiwan

2000 ◽  
Vol 80 (2) ◽  
pp. 168-178 ◽  
Author(s):  
Suh-Fang Jeng ◽  
Kuo-Inn Tsou Yau ◽  
Li-Chiou Chen ◽  
Shu-Fang Hsiao
Keyword(s):  
Preterm Infants ◽  
Interrater Reliability ◽  
Physical Therapist ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Intraclass Correlation Coefficients ◽  
Scale Reliability ◽  
Scale Scores ◽  
Acceptable Reliability

Abstract Background and Purpose. The goal of this study was to examine the reliability and validity of measurements obtained with the Alberta Infant Motor Scale (AIMS) for evaluation of preterm infants in Taiwan. Subjects. Two independent groups of preterm infants were used to investigate the reliability (n=45) and validity (n=41) for the AIMS. Methods. In the reliability study, the AIMS was administered to the infants by a physical therapist, and infant performance was videotaped. The performance was then rescored by the same therapist and by 2 other therapists to examine the intrarater and interrater reliability. In the validity study, the AIMS and the Bayley Motor Scale were administered to the infants at 6 and 12 months of age to examine criterion-related validity. Results. Intraclass correlation coefficients (ICCs) for intrarater and interrater reliability of measurements obtained with the AIMS were high (ICC=.97–.99). The AIMS scores correlated with the Bayley Motor Scale scores at 6 and 12 months (r=.78 and .90), although the AIMS scores at 6 months were only moderately predictive of the motor function at 12 months (r=.56). Conclusion and Discussion. The results suggest that measurements obtained with the AIMS have acceptable reliability and concurrent validity but limited predictive value for evaluating preterm Taiwanese infants.

Validation of Sherouk's Critical Thinking Test (SH-CTT)

2021 ◽  
pp. 1345-1356
Author(s):  
Sherouk J. Kadhm
Keyword(s):  
Critical Thinking ◽  
Reliability And Validity ◽  
Thinking Skills ◽  
Critical Thinking Skills ◽  
Arabic Version ◽  
Leadership Selection ◽  
Response Options ◽  
Academic Environments ◽  
Latent Constructs ◽  
Confirmatory Factor

This study aimed to examine the psychometric properties (reliability and validity) of the Arabic version of Sherouk's Critical Thinking Test. This test has four parts, each of which provides a story that is divided into an introduction and a scene; each story is then followed by a list of sensitive questions featuring two response options (Agree/Disagree). A sample of 158 university students participated in this study. The results showed that the reliability of the entire scale was 0.885, and the reliability of each of its four parts was 0.885, 0.829, 0.772, and 0.721, respectively. Confirmatory factor analysis of the collected data revealed a good fit with the latent constructs (CMIN/DF: 1.446; CMIN/DF: 1.357; CMIN/DF: 1.417; and CMIN/DF: 1.151) for parts one to four, respectively. The test's adaptability was considered when designing this instrument, and examinees were invited to review it. This paper recommended using this instrument in the fields of performance, leadership, selection, training, and improvement, and it was also created as a tool to measure individuals' critical thinking skills and abilities in organizations, companies, and academic environments. The English and German versions of this test were also examined; these versions are now currently available for use.

scholarly journals Critical thinking for the federal auditor

2020 ◽  
Vol 27 (91) ◽  
pp. 2-26
Author(s):  
Gabrielle G. McClure-Nelson
Keyword(s):  
Critical Thinking ◽  
Private Sector ◽  
Core Competencies ◽  
Public Trust ◽  
Thinking Skills ◽  
Critical Thinking Skills ◽  
Recent Survey ◽  
Certified Public Accountants ◽  
American Institute ◽  
The Core

In the overly constrained space of the federal audit environment, to what extent can critical thinking skills be applied in a profession characterized by arduous public trust expectations, controlling auditing standards, prescriptive federal acquisition policies, frequently changing guidance, continual peer oversight, and the slow implementation of audit findings? Promoting the increased use of private sector auditors may suggest that federal auditors perceive competencies differently. However, a recent survey administered to 645 auditors of a federal audit agency region indicated that the majority of the core competencies identified by the American Institute of Certified Public Accountants are perceived as relevant in auditing government contractors. However, of concern, the data were mixed in support of critical thinking as an important competency. Given employer preference for skills in this area, the author attempts to identify applications to increase auditor critical thinking skills and to offer suggestions for increasing the relevance of the federal audit.

Assessment of Reliability and Validity of the Cochlear Implant Skills Review: A New Measure to Evaluate Cochlear Implant Users' Device Skills and Knowledge

2021 ◽  
pp. 1-23
Author(s):  
Kara Vasil ◽  
Jessica Lewis ◽  
Christin Ray ◽  
Jodi Baxter ◽  
Claire Bernstein ◽  
...  
Keyword(s):  
Construct Validity ◽  
Cochlear Implant ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Intraclass Correlation Coefficients ◽  
Assessment Measure ◽  
Valid Assessment ◽  
Device Use

Purpose The Cochlear Implant Skills Review (CISR) was developed as a measure of cochlear implant (CI) users' skills and knowledge regarding device use. This study aimed to determine intra- and interrater reliability and agreement and establish construct validity for the CISR. Method In this study, the CISR was developed and administered to a cohort of 30 adult CI users. Participants included new CI users with less than 1 year of CI experience and experienced CI users with greater than 1 year of CI experience. The CISR administration required participants to demonstrate skills using the various features of their CI processors. Intra- and interrater reliability were assessed using intraclass correlation coefficients, agreement was assessed using Cohen's kappa, and construct validity was assessed by relating CISR performance to duration of CI use. Results Overall reliability for the entire instrument was 92.7%. Inter- and intrarater agreement were generally substantial or higher. Duration of CI use was a significant predictor of CISR performance. Conclusions The CISR is a reliable and valid assessment measure of device skills and knowledge for adult CI users. Clinicians can use this tool to evaluate areas of needed instruction and counseling and to assess users' skills over time.

scholarly journals Critical thinking, literature and teachers’ instructional activities: An Indonesian case of community service program for English teachers

2021 ◽  
Vol 4 (2) ◽  
pp. 138
Author(s):  
Isti Gandana ◽  
Nia Nafisah ◽  
Ernie Imperiani
Keyword(s):  
Critical Thinking ◽  
Critical Literacy ◽  
Core Competencies ◽  
Critical Role ◽  
English Teachers ◽  
Thinking Skills ◽  
Critical Thinking Skills ◽  
Literary Works ◽  
Instructional Activities ◽  
Inadequate Knowledge

Critical thinking (CT) has generally been known as one of the core competencies of the 21st-century skills that students need to acquire to succeed in today’s world. To attain this goal, teachers undoubtedly have a critical role to play. Involving a group of Indonesian teachers of English, this study aims to explore the teachers’ initial conceptions of critical thinking as reflected in their planned classroom activities. Specifically, the data collected were in the form of written instructional activities that  used literary works as learning materials. These data were obtained following the teachers’ participation in an online workshop on promoting critical thinking through literature  held for English teachers from MGMP (i.e., English Teachers Working Group) in Bandung Regency. The workshop introduced a model of teaching critical thinking skills using literary works  and required the teachers to design their own instructional activities based on the proposed CT model. Adopting content analysis as the methodological tool to scrutinize the teachers’ prepared materials,  our initial findings indicate that, in general, the teachers had inadequate knowledge of how to incorporate  critical thinking skills into their teaching. Literature was seen merely as a tool to develop students’ linguistic abilities rather than as a way to explore humans’ experiences and to connect ‘the word and the world’. This study recommends that the teachers be given more time to better explore the concepts surrounding  critical literacy  pedagogy  as well as be given more opportunities  for practical application of the promoted instructional model.

scholarly journals Impact of Critical Thinking Skills on Prospective Teachers’ Academic Achievement

2021 ◽  
Vol 4 (4) ◽  
pp. 773-781
Author(s):  
Ayesha Kanwal ◽  
Intzar Hussain Butt
Keyword(s):  
Academic Achievement ◽  
Teacher Education ◽  
Critical Thinking ◽  
Prospective Teachers ◽  
Reliability And Validity ◽  
Thinking Skills ◽  
Critical Thinking Skills ◽  
Five Dimensions ◽  
Study Results ◽  
The Impact

The study intention was to find the impact of critical thinking skills (CTS) on prospective teachers’ academic achievement. The method of the study was quantitative with correlation research design. The sample of the study was 113 prospective teachers using convenience sampling from three departments of teacher education institutions. To assess the Critical Thinking skills of prospective teachers, test was developed by the research comprised of five dimensions i.e. analyzing, assumption, deduction, inferences and interpreting information. Eight statements for each dimension required 40 minute to solve. Psychometric properties i.e. Reliability and validity of test was insured by applying Cronbach alpha which was 0.72. The study results reflect that level of critical thinking skills in prospective teachers were not up to mark (Mean=55.59%), the association amongst CTS and academic achievement was significant and moderate (r=0.365). It is recommended that administrators of teacher education programs should provide opportunities to educators to promote CT skills in prospective teachers.

scholarly journals Content Analysis for Critical Thinking Skills in the Lower Primary School Science Textbooks in Malaysia

2019 ◽  
Vol 6 (1) ◽  
pp. 83
Author(s):  
Mohammed Y. M. Mai ◽  
Muhammed Yusuf ◽  
Maria Saleh
Keyword(s):  
Content Analysis ◽  
Critical Thinking ◽  
Science Teachers ◽  
Science Curriculum ◽  
Reliability And Validity ◽  
Kappa Statistic ◽  
Thinking Skills ◽  
School Science ◽  
Critical Thinking Skills ◽  
Science Textbooks

The purpose of this study is to analyse lower primary science textbooks (grade 1-3), which were implemented by the Ministry of Education in Malaysia, in terms of thinking skills. To achieve this, a tool for content analysis including thinking skill items will be constructed. Two science teachers have be trained to conduct the content analysis for grade 1-3 science textbooks, the reliability and validity of analysis have be verified using Cohen’s kappa statistic. From the results, it can be seen clearly that Prioritizing”, “Evaluation” and “Detecting Bias” thinking skills are not included at all in the science textbook from year one until year three. While, the most frequent critical thinking skills are “attributing”, “analysing”, “Grouping and Classifying” and “Sequencing” gradually. Most of the skills have been in the pictures for year one, and in the “text” and then within the “picture” and finally within the “activities” for both of year two and three. Such research help teachers and curriculum developers in the development and implementation of science curriculum to raise the level of achievement of thinking skills for students.Keyword: Thinking Skills, Critical Thinking Skills, Content Analysis and Science Textbooks

Interobserver Reliability Using the Phonetic Level Evaluation With Severely and Profoundly Hearing-Impaired Children

1991 ◽  
Vol 34 (5) ◽  
pp. 989-999 ◽  
Author(s):  
Stephanie Shaw ◽  
Truman E. Coggins
Keyword(s):  
Interrater Reliability ◽  
Interobserver Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Hearing Impaired ◽  
Intraclass Correlation Coefficients ◽  
Assessment Measure ◽  
Impaired Children ◽  
Speech Assessment ◽  
Hearing Impaired Children

This study examines whether observers reliably categorize selected speech production behaviors in hearing-impaired children. A group of experienced speech-language pathologists was trained to score the elicited imitations of 5 profoundly and 5 severely hearing-impaired subjects using the Phonetic Level Evaluation (Ling, 1976). Interrater reliability was calculated using intraclass correlation coefficients. Overall, the magnitude of the coefficients was found to be considerably below what would be accepted in published behavioral research. Failure to obtain acceptably high levels of reliability suggests that the Phonetic Level Evaluation may not yet be an accurate and objective speech assessment measure for hearing-impaired children.

Sign in / Sign up

Export Citation Format

Share Document