The Content Validity and Inter-Rater Reliability of the Occupational Therapy Pediatric Inventory of Cognitive Skills (OT-PICS): An Assessment Tool of Functional Cognition in Children

2021 ◽  
pp. 1-17
Author(s):  
Christina M. Dumas ◽  
Lenin C. Grajo
Keyword(s):  
Occupational Therapy ◽  
Content Validity ◽  
Cognitive Skills ◽  
Assessment Tool ◽  
Rater Reliability

scholarly journals Psychometric evaluation of the Work Readiness Questionnaire in schizophrenia

CNS Spectrums ◽  
2014 ◽  
Vol 21 (2) ◽  
pp. 199-206 ◽  
Author(s):  
Steven G. Potkin ◽  
Dragana Bugarski-Kirola ◽  
Chris J. Edgar ◽  
Sherif Soliman ◽  
Stephanie Le Scouiller ◽  
...  
Keyword(s):  
Content Validity ◽  
Negative Symptoms ◽  
Assessment Tool ◽  
Symptom Control ◽  
Cultural Influences ◽  
Therapeutic Interventions ◽  
Work Readiness ◽  
Rater Reliability ◽  
Predictive Values ◽  
Supported Work

Objective/IntroductionUnemployment can negatively impact quality of life among patients with schizophrenia. Employment status depends on ability, opportunity, education, and cultural influences. A clinician-rated scale of work readiness, independent of current work status, can be a valuable assessment tool. A series of studies were conducted to create and validate a Work Readiness Questionnaire (WoRQ) for clinicians to assess patient ability to engage in socially useful activity, independent of work availability.MethodsContent validity, test–retest and inter-rater reliability, and construct validity were evaluated in three separate studies.ResultsContent validity was supported. Cronbach’s α was 0.91, in the excellent range. Clinicians endorsed WoRQ concepts, including treatment adherence, physical appearance, social competence, and symptom control. The final readiness decision showed good test–retest reliability and moderate inter-rater reliability. Work readiness was associated with higher function and lower levels of negative symptoms. Low positive and high negative predictive values confirmed the concept validity.DiscussionThe WoRQ has suitable psychometric properties for use in a clinical trial for patients with a broad range of symptom severity. The scale may be applicable to assess therapeutic interventions. It is not intended to assess eligibility for supported work interventions.ConclusionsThe WoRQ is suitable for use in schizophrenia clinical trials to assess patient work functional potential.

scholarly journals Validity and Reliability Assessment Tool of Nursing Students Professional Behavior

Jurnal NERS ◽  
2015 ◽  
Vol 10 (2) ◽  
pp. 208
Author(s):  
Fatikhu Yatuni Asmara
Keyword(s):  
Nursing Students ◽  
Construct Validity ◽  
Content Validity ◽  
Assessment Tool ◽  
Reliability Assessment ◽  
Reliability Test ◽  
Validity And Reliability ◽  
Professional Behaviour ◽  
Rater Reliability ◽  
Validity Test

Introduction: Shieffield Peer Review Assessment (SPRAT) is an instrument to assess medical student’s performance using Multi Source Feedback (MSF) method. The previous study stated that MSF with SPRAT is an effective tool assessing professional behaviour (PB) of nursing students both in clinical setting and community setting. However it needs more items to be added. Based on that explanation, it needs conducting validity and reliability test to make sure that the tool is valid and reliable. Method: There were two types of validity test used, content validity test and construct validity test as well as reliability test, namely inter-rater reliability test and item covariance test. Participants were four clinical instructors and 116 nursing students. Result: Content validity test showed that two items must be added as part of assessment item, namely diciplines and faithness. Furthermore construct validity test showed that five items were not valid since they had pearson correlation score <0.3. However the items were included as consideration of nursing students’s PB. Inter-rater reliability test and item covariance reliability test showed that the tool was reliable with score 0.460 and 0.912 respectively. Discussion: The assessment tool can be applied to assess PB of nursing students since it valid and reliable. It needs to investigate the effectiveness of the tool in difference of PB of nursing students.Keywords: validity, reliability, assessment tool of PB (professional behaviour)

scholarly journals Development and reliability of the Korean version of the Feeding Abilities Assessment

2019 ◽  
Vol 32 (1) ◽  
pp. 69-74
Author(s):  
Seul Gi Koo ◽  
Hae Yean Park ◽  
Jongbae Kim ◽  
Areum Han
Keyword(s):  
Correlation Coefficient ◽  
Internal Consistency ◽  
Content Validity ◽  
Assessment Tool ◽  
High Reliability ◽  
Intraclass Correlation ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Korean Version ◽  
Test Retest Reliability

Objective The purpose of this study is to introduce a standardised assessment tool by verifying the reliability of the translated Korean version of the Feeding Abilities Assessment (K-FAA), which was developed to suit Korean culture. Methods The research subjects were 65 patients with dementia living in nursing homes. The K-FAA was completed by verifying the suitability of translation and reverse translation. The validity of the K-FAA was established through content validity, while its reliability was analysed based on internal consistency reliability for the items, test–retest reliability and inter-rater reliability. Results The content validity index determined, based on the assessment of professors, occupational therapists, and nurses, was more than .70. Cronbach’s α was more than .929, showing good internal consistency. A test–retest reliability of .884 was derived using Pearson’s correlation coefficient (p < .01), and an inter-rater reliability of .800 was derived using the kappa coefficients; intraclass correlation coefficient was .897, which also indicated good reliability. Conclusion The K-FAA was modified to fit the Korean domestic situation, and this assessment had high reliability. Therefore, K-FAA can evaluate the feeding ability of patients with dementia. Future studies should focus on providing evidence-based data to maintain or supplement the feeding ability of patients with dementia in Korea.

Preliminary Investigation of the Content Validity and Clinical Utility of the Predischarge Assessment Tool

1998 ◽  
Vol 65 (1) ◽  
pp. 3-11 ◽  
Author(s):  
Deborah Laliberte Rudman ◽  
Jennifer Tooke ◽  
Tanya Glencross Eimantas ◽  
Mary Hall ◽  
Karen Brennan Maloney
Keyword(s):  
Occupational Therapy ◽  
Acute Care ◽  
Content Validity ◽  
Clinical Utility ◽  
Assessment Tool ◽  
Preliminary Investigation ◽  
Occupational Performance ◽  
Hospital Stays ◽  
Initial Support

With the trend towards decreasing lengths of hospital stays, comprehensive discharge assessment is essential to facilitate continuity of care between hospital and community, minimize readmissions, and maximize client satisfaction. Predischarge assessment is often part of the occupational therapy role, especially in acute care. The Predischarge Assessment Tool (PAT) outlines a method of conducting comprehensive assessments in a manner consistent with the Occupational Therapy Guidelines for Client-Centred Practice. The tool is designed to inform decisions regarding: occupational readiness for discharge to the community from acute care; physical and social supports required to support occupational performance; and follow-up therapy. Two studies provide initial support for the content validity and clinical utility of the PAT and, at the same time, indicate that some modifications to the PAT are required.

scholarly journals SIGNIFICANCE OF COGNITIVE ASSESSMENT IN RELATION TO OCCUPATIONAL THERAPY PRACTICE, AN EXPLORATORY STUDY FOR PAKISTAN OCCUPATIONAL THERAPY GUIDELINES

2021 ◽  
Vol 10 (1) ◽  
pp. 14-20
Author(s):  
Aliza Imtiaz
Keyword(s):  
Occupational Therapy ◽  
Cognitive Assessment ◽  
Cognitive Skills ◽  
Assessment Tool ◽  
Occupational Therapist ◽  
Assessment Tools ◽  
Outpatient Rehabilitation ◽  
Cross Sectional Survey ◽  
Cross Sectional ◽  
Authentic Evaluation

BACKGROUND AND AIMS Cognition is defined as the ability to perceive process and comprehend the information from the surrounding. Impairment in cognitive skills can significantly affect individual’s performance. The objective of the study is to identify the commonly used cognitive assessment tools by occupational therapist and determine its significance in occupational therapy practice. METHODOLOGY A total of 150 participants were enrolled in the cross-sectional survey that was responded by occupational therapist working in pediatric domain specifically in outpatient rehabilitation setting. The self-structured questionnaire was validated by factor analysis through SPSS. RESULTS: The findings of this study revealed that 96% of occupational therapists performed cognitive assessment out of which only 9.6% occupational therapist administer standardized assessments due to incompetency and lack of resources the rest use informal mode of assessments. Mini mental state examination (MMSE) is found to be most common cognitive assessment tool while tools like Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) is rarely used though it is very significant for Activities of daily living (ADL) cognition CONCLUSION It was concluded that the need of cognitive assessment in pediatric setting is an integral component in occupational therapy process for authentic evaluation and effective intervention plan. The curriculum must promote efficient training of standardize assessment and resources should be provided for better outcome and prognosis.

Inter-rater Reliability of Preceptors on Clinical Pharmacy Competency Evaluation

2020 ◽  
Vol 15 ◽  
Author(s):  
Dixon Thomas ◽  
Sherief Khalifa ◽  
Jayadevan Sreedharan ◽  
Rucha Bond
Keyword(s):  
United Arab Emirates ◽  
Clinical Pharmacy ◽  
Clinical Competence ◽  
Assessment Tool ◽  
Clinical Skills ◽  
Group Discussion ◽  
Pharmacy Students ◽  
Rater Reliability ◽  
Quasi Experimental ◽  
Competency Evaluation

Background:: Clinical competence of pharmacy students is better evaluated at their practice sites. compared to the classroom. A clinical pharmacy competency evaluation rubric like that of the American College of Clinical Pharmacy (ACCP)is an effective assessment tool for clinical skills and can be used to show item reliability. The preceptors should be trained on how to use the rubrics as many inherent factors could influence inter-rater reliability. Objective:: To evaluate inter-rater reliability among preceptors on evaluating clinical competence of pharmacy students, before and after a group discussion intervention. Methods:: In this quasi experimental study in a United Arab Emirates teaching hospital, Seven clinical pharmacy preceptors rated clinical pharmacy competencies of ten recent PharmD graduates referring to their portfolios and preceptorship. Clinical pharmacy competencies were adopted from ACCP and mildly modified to be relevant for the local settings. Results:: Inter-rater reliability (Cronbach's Alpha) among preceptors was reasonable being practitioners at a single site for 2-4 years. At domain level, inter-rater reliability ranged from 0.79 - 0.93 before intervention and 0.94 - 0.99 after intervention. No inter-rater reliability was observed in relation to certain competency elements ranging from 0.31 – 0.61 before intervention, but improved to 0.79 – 0.97 after intervention. Intra-class correlation coefficient improved among all individual preceptors being reliable with each other after group discussion though some had no reliability with each other before group discussion. Conclusion:: Group discussion among preceptors at the training site was found to be effective in improving inter-rater reliability on all elements of the clinical pharmacy competency evaluation. Removing a preceptor from analysis did not affect inter-rater reliability after group discussion.

scholarly journals Chinesisation, adaptation and validation of the Chelsea Critical Care Physical Assessment Tool in critically ill patients: a cross-sectional observational study

BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e045550
Author(s):  
Zhigang Zhang ◽  
Guoqiang Wang ◽  
Yuchen Wu ◽  
Jin Guo ◽  
Nannan Ding ◽  
...  
Keyword(s):  
Critical Care ◽  
Observational Study ◽  
Critically Ill ◽  
Content Validity ◽  
Assessment Tool ◽  
Western China ◽  
Validity Index ◽  
Cross Sectional ◽  
Physical Assessment ◽  
Content Validity Index

PurposeTo translate and adapt the Chelsea Critical Care Physical Assessment Tool (CPAx) into Chinese version (‘CPAx-Chi’), test the reliability and validity of CPAx-Chi, and verify the cut-off point for the diagnosis of intensive care unit-acquired weakness (ICU-AW).Study designCross-sectional observational study.MethodsForward and back translation, cross-cultural adaptation and pretesting of CPAx into CPAx-Chi were based on the Brislin model. Participants were recruited from the general ICU of five third-grade class-A hospitals in western China. Two hundred critically ill adult patients (median age: 53 years; 64% men) with duration of ICU stay ≥48 hours and Glasgow Coma Scale ≥11 were included in this study. Two researchers simultaneously and independently assessed eligible patients using the Medical Research Council Muscle Score (MRC-Score) and CPAx-Chi.ResultsThe content validity index of items was 0.889. The content validity index of scale was 0.955. Taking the MRC-Score scale as standard, the criterion validity of CPAx-Chi was r=0.758 (p<0.001) for researcher A, and r=0.65 (p<0.001) for researcher B. Cronbach’s α was 0.939. The inter-rater reliability was 0.902 (p<0.001). The area under the receiver operating characteristic curves of CPAx-Chi for diagnosing ICU-AW based on MRC-Score ≤48 were 0.899 (95% CI 0.862 to 1.025) and 0.874 (95% CI 0.824 to 0.925) for researcher B. The best cut-off point for CPAx-Chi for the diagnosis of ICU-AW was 31.5. The sensitivity was 87% and specificity was 77% for researcher A, whereas it was 0.621, 31.5, 75% and 87% for researcher B, respectively. The consistency was high when taking CPAx-Chi ≤31 and MRC-Score ≤48 as the cut-off points for the diagnosis of ICU-AW. Cohen’s kappa=0.845 (p=0.02) in researcher A and 0.839 (p=0.04) for researcher B.ConclusionsCPAx-Chi demonstrated content validity, criterion-related validity and reliability. CPAx-Chi showed the best accuracy in assessment of patients at risk of ICU-AW with good sensitivity and specificity at a recommended cut-off of 31.

scholarly journals Validity and Internal Consistency of the Preschool-FLAT, a New Tool for the Assessment of Food Literacy in Young Children from the Training-To-Health Project

2020 ◽  
Vol 17 (8) ◽  
pp. 2759
Author(s):  
Garden Tabacchi ◽  
Giuseppe Battaglia ◽  
Giuseppe Messina ◽  
Antonio Paoli ◽  
Antonio Palma ◽  
...  
Keyword(s):  
Internal Consistency ◽  
Content Validity ◽  
Structural Equation ◽  
Discriminant Validity ◽  
Assessment Tool ◽  
Intervention Group ◽  
Equation Modeling ◽  
Control Group ◽  
Food Literacy ◽  
Health Project

Background: The importance of assessing “food literacy” since youth has been highlighted and, to this purpose, valid and consistent instruments are needed. This study aimed to assess the validity and internal consistency of the preschool-FLAT (Food Literacy Assessment Tool). Methods. 505 children from 21 kindergartens, recruited within the Training-to-Health Project in Palermo (Italy), underwent oral sessions and activities on food-related aspects. Their knowledge/skills were recorded in the preschool-FLAT. The following scale measures were assessed: Content validity; internal consistency (Chronbach’s alpha coefficients); construct validity (Structural Equation Modeling—SEM); discriminant validity (intervention subgroup of 100 children vs. control group of 27 children). Results. Acceptable content validity of a 16-items scale and overall adequate internal consistency were revealed: Content validity index (CVI) 0.94, content validity ratio (CVR) 0.88, Chronbach’s alpha 0.76. The SEM revealed a 4-factor model fitting the data well (comparative fit index 0.939, root mean square error of approximation 0.033). Discriminant validity was good (intervention group scoring higher than control, p < 0.001, unpaired Student’s t-test). Conclusion. The preschool-FLAT revealed good psychometric properties, adequate validity and internal consistency. This is the only instrument in the literature specifically targeted to 3–6 years old children that could be effectively used to assess food literacy.

scholarly journals Development and psychometric properties of the Caring Behaviors Assessment Tool Nursing Version-Short Form

2020 ◽  
Vol 32 (10) ◽  
pp. 701-707
Author(s):  
Mehtap Akgün ◽  
Selma Turan Kavradim ◽  
İlkay Boz ◽  
Zeynep Özer
Keyword(s):  
Psychometric Properties ◽  
Root Mean Square ◽  
Internal Consistency ◽  
Content Validity ◽  
Assessment Tool ◽  
Short Form ◽  
Mean Square ◽  
Structural Validity ◽  
Caring Behaviors ◽  
Fit Index

Abstract Objectives To develop and examine the psychometric properties of the Caring Behaviors Assessment Tool Nursing Version-Short Form (CBAN-SF) based on the Theory of Human Caring to assess the nurses’ perceptions about caring behaviors. Design This study is based on the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. Setting The study was conducted at the medical-surgical services of Akdeniz University Hospital between October 2019 and January 2020. Participants This study was conducted with 216 nurses working in the surgery and internal clinics. Main Outcome Measures Psychometric properties of the Turkish version of the CBAN-SF with 27 items. Results It was found that the Content Validity Index (CVI) for the items of the draft scale was between 0.972 and 1.00 and the instrument’s CVI had an average score of 0.994. The CBAN-SF had good fit indexes (chi-square goodness of fit / degrees of freedom = 2.914, root mean square error of approximation = 0.075, comparative fit index = 0.984, non-normed fit index = 0.983, normed fit index = 0.972 and standardized root mean square residuals = 0.054) in structural validity. For internal consistency, the Cronbach’s alpha, Spearman–Brown and the Guttman split-half coefficients were all 0.974. The Cronbach’s alpha coefficient for the seven subfactors of the scale ranged between 0.793 and 0.904 and had acceptable internal consistency. The item-total score correlation of the scale was 0.648–0.829, and the factor loadings were 0.455–0.769. Conclusion The structural validity, internal consistency and content validity of the CBAN-SF supported to be a reliable and valid tool for assessment of caring behaviors by nurses.

Psychometric Testing of the Health Care Evidence-Based Practice Assessment Tool

2015 ◽  
Vol 23 (3) ◽  
pp. 485-498
Author(s):  
Martha R. Sleutel ◽  
Celestina Barbosa-Leiker ◽  
Marian Wilson
Keyword(s):  
Health Care ◽  
Content Validity ◽  
Assessment Tool ◽  
Evidence Based Practice ◽  
Factor Model ◽  
Psychometric Testing ◽  
Evidence Based ◽  
Validity And Reliability ◽  
Health Care Outcomes ◽  
Optimal Health

Background and Purpose: Evidence-based practice (EBP) is essential to optimal health care outcomes. Interventions to improve use of evidence depend on accurate assessments from reliable, valid, and user-friendly tools. This study reports psychometric analyses from a modified version of a widely used EBP questionnaire, the information literacy for nursing practice (ILNP). Methods: After content validity assessments by nurse researchers, a convenience sam ple of 2,439 nurses completed the revised 23-item questionnaire. We examined internal consistency and used factor analyses to assess the factor structure. Results: A modified 4-factor model demonstrated adequate fit to the data. Cronbach’s alpha was .80–.92 for the subscales. Conclusions: The shortened ILNP (renamed Healthcare EBP Assessment Tool or HEAT) demonstrated adequate content validity, construct validity, and reliability.

Sign in / Sign up

Export Citation Format

Share Document