scholarly journals Application of Rasch to Verify Different Building Competency Sub Constructs for Competency Evaluation of Graduates in Nigeria

2020 ◽  
Vol 9 (6) ◽  
pp. 253-260
Keyword(s):  
Construct Validity ◽  
Assessment Tool ◽  
Survey Design ◽  
Sampling Technique ◽  
Survey Instrument ◽  
Assessment Instrument ◽  
Theoretical Interpretation ◽  
Validity And Reliability ◽  
Expert Opinions ◽  
Competency Evaluation

Incessant concerns from employers and private sectors about the incompetence of graduates in Nigeria call for the creation of an assessment tool that could verify their skills. But there is no clear generally accepted and validated assessment instrument available for evaluating graduate performance. The aim of this study was to develop a valid instrument for assessing competency levels of building construction graduates in Nigeria. Survey design was adopted to obtain expert opinions on the validity of the sub-constructs and the related items about the employers 'needs. Three experts from the academic, public, and private sectors subjected the survey instrument to face, content, and construct validity and reliability. The survey instrument, which was analysed using IBM SPSS and WINSTEP version 3.73.3 was answered by a total of 200 building experts selected by proportionate stratified sampling technique. The consistency of the instrument was determined by fit statistics and point measure correlation (PTMEA Corr), for construct validity. The results revealed a very good items and person reliability of 0.97 and 0.94 respectively. Likewise, appropriate PTMEA Corr range from 0.36 to 0.68. Infit and outfit means square range obtained between 0.58 to 1.39. The findings give students, employers and academic institution a realistic and theoretical interpretation of the reality of labour market needs.

scholarly journals The Construct Validity and Reliability of an Assessment Tool for Competency in Cochlear Implant Surgery

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Patorn Piromchai ◽  
Pornthep Kasemsiri ◽  
Sudanthi Wijewickrema ◽  
Ioanna Ioannou ◽  
Gregor Kennedy ◽  
...  
Keyword(s):  
Construct Validity ◽  
Cochlear Implant ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Assessment Instrument ◽  
Validity And Reliability ◽  
Implant Surgery ◽  
Surgical Trainees ◽  
Cochlear Implant Surgery ◽  
Objective Feedback

Introduction. We introduce a rating tool that objectively evaluates the skills of surgical trainees performing cochlear implant surgery.Methods. Seven residents and seven experts performed cochlear implant surgery sessions from mastoidectomy to cochleostomy on a standardized virtual reality temporal bone. A total of twenty-eight assessment videos were recorded and two consultant otolaryngologists evaluated the performance of each participant using these videos.Results. Interrater reliability was calculated using the intraclass correlation coefficient for both the global and checklist components of the assessment instrument. The overall agreement was high. The construct validity of this instrument was strongly supported by the significantly higher scores in the expert group for both components.Conclusion. Our results indicate that the proposed assessment tool for cochlear implant surgery is reliable, accurate, and easy to use. This instrument can thus be used to provide objective feedback on overall and task-specific competency in cochlear implantation.

Urdu Translation and Validation of Activities-Specific Balance Confidence (ABC) Scale

2021 ◽  
Vol 15 (12) ◽  
pp. 3505-3508
Author(s):  
Noor Ul Ain Fatima ◽  
Qurat-Ul- Ain ◽  
Fareeha Kausar ◽  
Mian Ali Raza ◽  
Misbah Waris ◽  
...  
Keyword(s):  
Older Adults ◽  
Construct Validity ◽  
Correlation Coefficient ◽  
Content Validity ◽  
Assessment Tool ◽  
Test Reliability ◽  
Validity And Reliability ◽  
A Value ◽  
Abc Scale ◽  
Translation And Validation

Objective: To translate and validate the ABC-Scale in Urdu language to predict risk of fall in older population. Study design: Cross-cultural Translation and validation Place and Duration: Study was conducted in older adult community of Sialkot from March 2020 to December 2020. Methodology: Translation of ABC in Urdu was conducted by using Beaton et al guidelines. Two bilingual translators translated the original version into Urdu language step wise, correction process was followed. Then two backward translations were done by language expert. After all this process, the translated version was reviewed by the professionals and the final version was applied on 15 individuals. Its reliability and validity was tested on 60 older adults. Results: For test re test reliability, intra class correlation coefficient ICC was measured with a value of 0.984 Which shows good test re-test reliability. The internal consistency and reliability of ABC was calculated by Cronbach’s alpha for total score with a value of 0.985. Content validity was good with values of CVI ranging from 0.767 to 0.955. To test the discriminative validity, independent t test was used to show the difference between the healthy and unhealthy adults. Factor analysis of UABC showed total variance 81.277 and cumulative variance was also 81.277. To calculate construct validity of U-ABC Pearson’s correlation coefficient was used and measured as 0.558. Conclusion: It was concluded that Urdu version of UABC is a valid assessment tool for older adults with fear of fall. It has good content validity, construct validity and reliability. Keywords: activities specific balance scale, validation, Urdu translation, reliability, tool translation

VALIDITAS INSTRUMEN ASSESMEN (ANALISIS VALIDITAS ISI DAN KONSTRUK INSTRUMEN ASESMEN BUKU PELAJARAN AL QURAN HADIS KELAS 6 MADRASAH IBTIDAIYAH MATERI SURAT AD DHUHA BAB VI)

2021 ◽  
Vol 2 (01) ◽  
pp. 21-42
Author(s):  
Muhammad Afifullah Nizary ◽  
Ahmad Nur Kholik Nur Kholik
Keyword(s):  
Construct Validity ◽  
Content Validity ◽  
Assessment Instrument ◽  
Measuring Instrument ◽  
Validity And Reliability

This article attempts to examine how to analyze the validity of the assessment instrument. In fact, not all teachers are able to make the correct instruments. Many teachers only take instruments from other schools to be used as a tool in measuring their students, even though the same instrument is not necessarily a measuring instrument with different objects. This habit needs to be improved. The teacher should be able to make their own instruments, because the teacher knows best about the differences in the abilities of each student. When the teacher uses instruments from other schools, where there are parts that the teacher has not conveyed in class, the students themselves are the losers. In making an assessment instrument the teacher must pay attention to two characteristics. Namely validity and reliability. Validity itself is divided into content validity, construct validity, and eksternal validity. Each validity has its own characteristics. Keywords: evaluation and teacher, the validity of the assessment instrument, the validity of the content

scholarly journals Designing Valid and Reliable Vignette Experiments for Survey Research: A Case Study on the Fair Gender Income Gap

2017 ◽  
Vol 7 (2) ◽  
Author(s):  
Peter M. Steiner ◽  
Christiane Atzmüller ◽  
Dan Su
Keyword(s):  
Experimental Design ◽  
Construct Validity ◽  
Factorial Design ◽  
Survey Research ◽  
Survey Design ◽  
Validity And Reliability ◽  
Design Elements ◽  
The Real ◽  
Income Gap

In survey research, vignette experiments typically employ short, systematically varied descriptions of situations or persons (called vignettes) to elicit the beliefs, attitudes, or behaviors of respondents with respect to the presented scenarios. Using a case study on the fair gender income gap in Austria, we discuss how different design elements can be used to increase a vignette experiment’s validity and reliability. With respect to the experimental design, the design elements considered include a confounded factorial design, a between-subjects factor, anchoring vignettes, and blocking by respondent strata and interviewers. The design elements for the sampling and survey design consist of stratification, covariate measurements, and the systematic assignment of vignette sets to respondents and interviewers. Moreover, the vignettes’ construct validity is empirically validated with respect to the real gender income gap in Austria. We demonstrate how a broad range of design elements can successfully increase a vignette study’s validity and reliability.

Investigation of physical education teachers’ first aid response with vignette technique

2021 ◽  
Vol 9 (1) ◽  
pp. 259-272
Author(s):  
Aysel Kizilkaya Namli ◽  
Keyword(s):  
Content Analysis ◽  
Physical Education ◽  
Random Sampling ◽  
Sampling Technique ◽  
Simple Random Sampling ◽  
First Aid ◽  
Physical Education Teachers ◽  
Validity And Reliability ◽  
Expert Opinions ◽  
First Aid Training

In the study, the aim is to determine the effects of the application of first aid training lessons on sports injuries. The group of the study was determined by the simple random sampling method included in the random sampling technique and consists of 20 physical education teachers who are still on duty in the 2020-2021 academic year. All of the physical education teachers are experienced teachers who have taken first aid lessons in undergraduate education. Vignette technique, one of the qualitative research methods, was used in the research. With the technique in question, teachers were asked to express their opinions following the first aid scenarios. The interviews were made over the phone and recorded with the consent of the participants. Content analysis was used in the study, and the validity and reliability of the data were provided. The opinions received from the participants were examined and themes were created by the content analysis and given with direct quotations. The opinions of the participants were interpreted by comparing with the findings of the experts in the field in accordance with the vignette technique. According to the results of the study, it was determined that physical education teachers gave similar answers to expert opinions about all scenarios. However, the accuracy rates of these answers vary with the scenarios. Therefore, it was determined that teachers' first aid knowledge varies, in some cases, they behave closely to expert opinions, and in some others, they move away from expert opinions. It has been observed that this situation resembles similar studies in the literature. When the mentioned results are compared with the literature; it is recommended that first aid training be sustainable and implementation techniques are continuously improved.

scholarly journals Short Assessment of Health Literacy for Portuguese-speaking Adults

2012 ◽  
Vol 46 (4) ◽  
pp. 702-711 ◽  
Author(s):  
Daniel Apolinario ◽  
Rafaela de Castro Oliveira Pereira Braga ◽  
Regina Miksian Magaldi ◽  
Alexandre Leopold Busse ◽  
Flavia Campora ◽  
...  
Keyword(s):  
Older Adults ◽  
Health Literacy ◽  
Construct Validity ◽  
Psychometric Properties ◽  
Assessment Tool ◽  
Characteristic Curve ◽  
Intraclass Correlation ◽  
Validity And Reliability ◽  
Convenience Sample ◽  
Inadequate Health Literacy

OBJECTIVE: To develop and validate a short health literacy assessment tool for Portuguese-speaking adults. METHODS: The Short Assessment of Health Literacy for Portuguese-speaking Adults is an assessment tool which consists of 50 items that assess an individual's ability to correctly pronounce and understand common medical terms. We evaluated the instrument's psychometric properties in a convenience sample of 226 Brazilian older adults. Construct validity was assessed by correlating the tool scores with years of schooling, self-reported literacy, and global cognitive functioning. Discrimination validity was assessed by testing the tool's accuracy in detecting inadequate health literacy, defined as failure to fully understand standard medical prescriptions. RESULTS: Moderate to high correlations were found in the assessment of construct validity (Spearman's coefficients ranging from 0.63 to 0.76). The instrument showed adequate internal consistency (Cronbach's alpha=0.93) and adequate test-retest reliability (intraclass correlation coefficient=0.95). The area under the receiver operating characteristic curve for detection of inadequate health literacy was 0.82. A version consisting of 18 items was tested and showed similar psychometric properties. CONCLUSIONS: The instrument developed showed good validity and reliability in a sample of Brazilian older adults. It can be used in research and clinical settings for screening inadequate health literacy.

scholarly journals Construct Validity and Reliability of the Work Environment Assessment Instrument WE-10

2020 ◽  
Vol 17 (20) ◽  
pp. 7364
Author(s):  
Rudy de Barros Ahrens ◽  
Luciana da Silva Lirani ◽  
Antonio Carlos de Francisco
Keyword(s):  
Factor Analysis ◽  
Construct Validity ◽  
Organizational Climate ◽  
Work Environment ◽  
Assessment Instrument ◽  
Quality Of Work Life ◽  
Validity And Reliability ◽  
Rotation Method ◽  
Commonality Analysis

The purpose of this study was to validate the construct and reliability of an instrument to assess the work environment as a single tool based on quality of life (QL), quality of work life (QWL), and organizational climate (OC). The methodology tested the construct validity through Exploratory Factor Analysis (EFA) and reliability through Cronbach’s alpha. The EFA returned a Kaiser–Meyer–Olkin (KMO) value of 0.917; which demonstrated that the data were adequate for the factor analysis; and a significant Bartlett’s test of sphericity (χ² = 7465.349; Df = 1225; p ≤ 0.000). After the EFA; the varimax rotation method was employed for a factor through commonality analysis; reducing the 14 initial factors to 10. Only question 30 presented commonality lower than 0.5; and the other questions returned values higher than 0.5 in the commonality analysis. Regarding the reliability of the instrument; all of the questions presented reliability as the values varied between 0.953 and 0.956. Thus; the instrument demonstrated construct validity and reliability

scholarly journals Validity and Reliability Assessment Tool of Nursing Students Professional Behavior

Jurnal NERS ◽  
2015 ◽  
Vol 10 (2) ◽  
pp. 208
Author(s):  
Fatikhu Yatuni Asmara
Keyword(s):  
Nursing Students ◽  
Construct Validity ◽  
Content Validity ◽  
Assessment Tool ◽  
Reliability Assessment ◽  
Reliability Test ◽  
Validity And Reliability ◽  
Professional Behaviour ◽  
Rater Reliability ◽  
Validity Test

Introduction: Shieffield Peer Review Assessment (SPRAT) is an instrument to assess medical student’s performance using Multi Source Feedback (MSF) method. The previous study stated that MSF with SPRAT is an effective tool assessing professional behaviour (PB) of nursing students both in clinical setting and community setting. However it needs more items to be added. Based on that explanation, it needs conducting validity and reliability test to make sure that the tool is valid and reliable. Method: There were two types of validity test used, content validity test and construct validity test as well as reliability test, namely inter-rater reliability test and item covariance test. Participants were four clinical instructors and 116 nursing students. Result: Content validity test showed that two items must be added as part of assessment item, namely diciplines and faithness. Furthermore construct validity test showed that five items were not valid since they had pearson correlation score <0.3. However the items were included as consideration of nursing students’s PB. Inter-rater reliability test and item covariance reliability test showed that the tool was reliable with score 0.460 and 0.912 respectively. Discussion: The assessment tool can be applied to assess PB of nursing students since it valid and reliable. It needs to investigate the effectiveness of the tool in difference of PB of nursing students.Keywords: validity, reliability, assessment tool of PB (professional behaviour)

A Learning Outcome Inspired Survey Instrument for Assessing the Quality of Continuous Improvement Cycle

2019 ◽  
Vol 15 (2) ◽  
pp. 108-129 ◽  
Author(s):  
Abdallah Namoun ◽  
Ahmad Taleb ◽  
Mohammed Al-Shargabi ◽  
Mohamed Benaida
Keyword(s):  
Quality Improvement ◽  
Continuous Quality Improvement ◽  
Delphi Study ◽  
Assessment Tool ◽  
Survey Instrument ◽  
Assessment Instrument ◽  
Self Assessment ◽  
Continuous Quality ◽  
System Requirements

Measuring the effectiveness of a continuous quality improvement cycle in education is a cumbersome and sophisticated process. This article contributes a comprehensive self-assessment instrument for identifying the strengths and weaknesses of all phases of a continuous quality improvement cycle, including planning, data collection, analysis and reporting, and implementation of improvements. To this end, a four round Delphi study soliciting a total of 23 program quality experts from four universities was conducted. The produced survey instrument contains a total of 50 questions. The instrument may be used by quality experts in education to judge the quality of their continuous quality improvement cycle that endeavours to assess the attainment of learning outcomes in various undergraduate educational programs. Moreover, the instrument could be exploited to infer relevant user and system requirements and guide the development of an automated self-assessment tool aimed at identifying the shortcomings in educational continuous quality improvement cycles.

scholarly journals Validation of the Equine Behaviour Assessment and Research Questionnaire (E-BARQ): A New Survey Instrument for Exploring and Monitoring the Domestic Equine Triad

Animals ◽  
2020 ◽  
Vol 10 (11) ◽  
pp. 1982 ◽  
Author(s):  
Kate Fenner ◽  
Sarah Matlock ◽  
Jane Williams ◽  
Bethany Wilson ◽  
Andrew McLean ◽  
...  
Keyword(s):  
Construct Validity ◽  
Quantitative Data ◽  
Assessment Tool ◽  
Questionnaire Validation ◽  
Subjective Ratings ◽  
Validity And Reliability ◽  
Rater Reliability ◽  
Validity Testing ◽  
Significant Alignment ◽  
Behavioural Assessment

The Equine Behaviour Assessment and Research Questionnaire (E-BARQ) was developed to obtain quantitative data on the domestic equine triad: training, management and behaviour. It can be taken repeatedly, thus collecting longitudinal data to enable evaluation of how changes in a horse’s training and management are reflected in that horse’s behaviour over time and how these changes can impact horse welfare in the longer term. Questionnaire validation and reliability were tested by determining (a) whether an owner’s subjective ratings of their horse’s problematic behaviours or undesirable temperament traits were reflected in the questionnaire scores obtained for that horse (construct validity), (b) whether two respondents, equally familiar with a particular horse, reported comparable scores for that horse through the questionnaire (inter-rater reliability), and (c) whether the same respondent, scoring the same horse after a known interval of time, recorded similar responses (intra-rater reliability). Construct validity testing of 1923 responses showed significant alignment between owners’ reported experience of focal horses’ behaviour and those horses’ E-BARQ scores, with scores varying from 1.13 to 1.34 for ridden horse behaviour (all p < 0.001) and from 1.06 to 1.43 for non-ridden horse behaviour (all p < 0.001). Inter-rater reliability testing of ten horse–rider pairs revealed that 203 of the 215 question items were significantly aligned (p < 0.001) when tested by two independent raters. Of the remaining 19 items, four had fair alignment (ĸ = 0.174–0.316; p = 0.281) and ten items, largely related to whether the horse shows behavioural signs related to anxiety when taken away from home, did not align (ĸ = 0; p = 1). Intra-rater reliability tests showed that the responses significantly aligned on all 215 question items tested (p < 0.001). The results of these tests confirmed the construct validity and reliability of E-BARQ as a standardised behavioural assessment tool for horses.

Sign in / Sign up

Export Citation Format

Share Document