scholarly journals Assessment of the nursing care product (APROCENF): a reliability and construct validity study

2017 ◽  
Vol 25 (0) ◽  
Author(s):  
Danielle Fabiana Cucolo ◽  
Márcia Galan Perroca
Keyword(s):  
Construct Validity ◽  
Nursing Care ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Principal Component ◽  
Weighted Kappa ◽  
Kappa Index ◽  
Rater Reliability ◽  
Care Product ◽  
Spearman’S Correlation

ABSTRACT Objectives: to verify the reliability and construct validity estimates of the "Assessment of nursing care product" scale (APROCENF) and its applicability. Methods: this validation study included a sample of 40 (inter-rater reliability) and 172 (construct validity) assessments performed by nurses at the end of the work shift at nine inpatient services of a teaching hospital in the Brazilian Southeast. The data were collected between February and September/2014 with interruptions. Cronbach's alpha and Spearman's correlation coefficients were calculated, as well as the intraclass correlation and the weighted kappa index (inter-rater reliability). Exploratory factor analysis was used with principal component extraction and varimax rotation (construct validity). Results: the internal consistency revealed an alpha coefficient of 0.85, item-item correlation ranging between 0.13 and 0.61 and item-total correlation between 0.43 and 0.69. Inter-rater equivalence was obtained and all items evidenced significant factor loadings. Conclusion: this research evidenced the reliability and construct validity of the scale to assess the nursing care product. Its application in nursing practice permits identifying improvements needed in the production process, contributing to management and care decisions.

scholarly journals Development and Relative Validity of a Semiquantitative Food Frequency Questionnaire to Estimate Dietary Intake among a Multi-Ethnic Population in the Malaysian Cohort Project

Nutrients ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 1163
Author(s):  
Suzana Shahar ◽  
Mohd Razif Shahril ◽  
Noraidatulakma Abdullah ◽  
Boekhtiar Borhanuddin ◽  
Mohd Arman Kamaruddin ◽  
...  
Keyword(s):  
Vitamin C ◽  
Correlation Coefficient ◽  
Food Frequency Questionnaire ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Dietary Intakes ◽  
Food Frequency ◽  
Relative Validity ◽  
Spearman’S Correlation

Measuring dietary intakes in a multi-ethnic and multicultural setting, such as Malaysia, remains a challenge due to its diversity. This study aims to develop and evaluate the relative validity of an interviewer-administered food frequency questionnaire (FFQ) in assessing the habitual dietary exposure of The Malaysian Cohort (TMC) participants. We developed a nutrient database (with 203 items) based on various food consumption tables, and 803 participants were involved in this study. The output of the FFQ was then validated against three-day 24-h dietary recalls (n = 64). We assessed the relative validity and its agreement using various methods, such as Spearman’s correlation, weighed Kappa, intraclass correlation coefficient (ICC), and Bland–Altman analysis. Spearman’s correlation coefficient ranged from 0.24 (vitamin C) to 0.46 (carbohydrate), and almost all nutrients had correlation coefficients above 0.3, except for vitamin C and sodium. Intraclass correlation coefficients ranged from −0.01 (calcium) to 0.59 (carbohydrates), and weighted Kappa exceeded 0.4 for 50% of nutrients. In short, TMC’s FFQ appears to have good relative validity for the assessment of nutrient intake among its participants, as compared to the three-day 24-h dietary recalls. However, estimates for iron, vitamin A, and vitamin C should be interpreted with caution.

scholarly journals A Comparison of Reliability Coefficients for Ordinal Rating Scales

2021 ◽  
Author(s):  
Alexandra de Raadt ◽  
Matthijs J. Warrens ◽  
Roel J. Bosker ◽  
Henk A. L. Kiers
Keyword(s):  
Empirical Data ◽  
Rating Scales ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Rater Reliability ◽  
Intraclass Correlations ◽  
Applied Researcher ◽  
Highly Correlated ◽  
Reliability Coefficients

AbstractKappa coefficients are commonly used for quantifying reliability on a categorical scale, whereas correlation coefficients are commonly applied to assess reliability on an interval scale. Both types of coefficients can be used to assess the reliability of ordinal rating scales. In this study, we compare seven reliability coefficients for ordinal rating scales: the kappa coefficients included are Cohen’s kappa, linearly weighted kappa, and quadratically weighted kappa; the correlation coefficients included are intraclass correlation ICC(3,1), Pearson’s correlation, Spearman’s rho, and Kendall’s tau-b. The primary goal is to provide a thorough understanding of these coefficients such that the applied researcher can make a sensible choice for ordinal rating scales. A second aim is to find out whether the choice of the coefficient matters. We studied to what extent we reach the same conclusions about inter-rater reliability with different coefficients, and to what extent the coefficients measure agreement in a similar way, using analytic methods, and simulated and empirical data. Using analytical methods, it is shown that differences between quadratic kappa and the Pearson and intraclass correlations increase if agreement becomes larger. Differences between the three coefficients are generally small if differences between rater means and variances are small. Furthermore, using simulated and empirical data, it is shown that differences between all reliability coefficients tend to increase if agreement between the raters increases. Moreover, for the data in this study, the same conclusion about inter-rater reliability was reached in virtually all cases with the four correlation coefficients. In addition, using quadratically weighted kappa, we reached a similar conclusion as with any correlation coefficient a great number of times. Hence, for the data in this study, it does not really matter which of these five coefficients is used. Moreover, the four correlation coefficients and quadratically weighted kappa tend to measure agreement in a similar way: their values are very highly correlated for the data in this study.

scholarly journals Reliability and Validity of Trunk Assessment for People With Multiple Sclerosis

2006 ◽  
Vol 86 (1) ◽  
pp. 66-76 ◽  
Author(s):  
Geert Verheyden ◽  
Godelieve Nuyens ◽  
Alice Nieuwboer ◽  
Pol Van Asch ◽  
Piet Ketelaer ◽  
...  
Keyword(s):  
Multiple Sclerosis ◽  
Construct Validity ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Altman Analysis ◽  
Bland Altman Analysis ◽  
Intraclass Correlation Coefficients ◽  
Test Retest Reliability

Abstract Background and Purpose. Standardized scales are a prerequisite for rehabilitation and research. This study was designed to determine the reliability and validity of scores on items of the trunk assessment of the Melsbroek Disability Scoring Test (MDST) and Trunk Impairment Scale (TIS) in people with multiple sclerosis (MS). Subjects. Thirty people with MS participated in the study. Methods. Interrater and test-retest reliability and construct validity were assessed. Results. Kappa and weighted kappa values for the items of the trunk assessment of the MDST ranged from .74 to .95, and the kappa and weighted kappa values for the TIS items ranged from .46 to 1.00. Intraclass correlation coefficients for interrater and test-retest agreement were .93 and .92, respectively, for the trunk assessment of the MDST and .97 and .95, respectively, for the TIS. Bland-Altman analysis showed consistency of scores without observer bias. Construct validity was established. Discussion and Conclusion. The MDST and TIS provide reliable assessments of the trunk and are valid scales for measuring trunk performance in people with MS. [Verheyden G, Nuyens G, Nieuwboer A, et al. Reliability and validity of trunk assessment for people with multiple sclerosis.

Appraisal of a scoring instrument for training and testing neonatal intubation skills

2018 ◽  
Vol 104 (5) ◽  
pp. F521-F527 ◽  
Author(s):  
Romy N Bouwmeester ◽  
Mathijs Binkhorst ◽  
Nicole K Yamada ◽  
Rosa Geurtzen ◽  
Arno F J van Heijst ◽  
...  
Keyword(s):  
Construct Validity ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Rater Reliability ◽  
Patient Simulator ◽  
Training Centre ◽  
Intraclass Correlation Coefficients ◽  
Tube Position ◽  
The Usa ◽  
Neonatal Patient

ObjectiveTo determine the validity, reliability, feasibility and applicability of a neonatal intubation scoring instrument.DesignProspective observational study.SettingSimulation-based research and training centre (Center for Advanced Pediatric and Perinatal Education), California, USA.SubjectsForty clinicians qualified for neonatal intubation.InterventionsVideotaped elective intubations on a neonatal patient simulator were scored by two independent raters. One rater scored the intubations twice. We scored the preparation of equipment and premedication, intubation performance, tube position/fixation, communication, number of attempts, duration and successfulness of the procedure.Main outcome measuresIntraclass correlation coefficients (ICC) were calculated for intrarater and inter-rater reliability. Kappa coefficients for individual items and mean kappa coefficients for all items combined were calculated. Construct validity was assessed with one-way analysis of variance using the hypothesis that experienced clinicians score higher than less experienced clinicians. The approximate time to score one intubation and the instrument’s applicability in another setting were evaluated.ResultsICCs for intrarater and inter-rater reliability were 0.99 (95% CI 0.98 to 0.99) and 0.89 (95% CI 0.35 to 0.96), and mean kappa coefficients were 0.93 (95% CI 0.85 to 1.01) and 0.71 (95% CI 0.56 to 0.92), respectively. There were no differences between the more and less experienced clinicians regarding preparation, performance, communication and total scores. The experienced group scored higher only on tube position/fixation (p=0.02). Scoring one intubation took approximately 15 min. Our instrument, developed in The Netherlands, could be readily applied in the USA.ConclusionsOur scoring instrument for simulated neonatal intubations appears to be reliable, feasible and applicable in another centre. Construct validity could not be established.

scholarly journals Intra and Inter-Rater Reliability and Convergent Validity of FIT-HaNSA in Individuals with Grade П Whiplash Associated Disorder

2016 ◽  
Vol 10 (1) ◽  
pp. 179-189 ◽  
Author(s):  
Michael Pierrynowski ◽  
Colleen McPhee ◽  
Saurabh P. Mehta ◽  
Joy C. MacDermid ◽  
Anita Gross
Keyword(s):  
Construct Validity ◽  
Rating Scale ◽  
Motor Vehicle ◽  
Intraclass Correlation ◽  
Neck Disability Index ◽  
Correlation Coefficients ◽  
Control Group ◽  
Healthy Controls ◽  
Rater Reliability ◽  
Study Results

Background: Whiplash-Associated Disorders (WAD) are common following a motor vehicle accident. The Functional Impairment Test - Hand, and Neck/Shoulder/Arm (FIT-HaNSA) assesses upper extremity physical performance. It has been validated in patients with shoulder pathology but not in those with WAD. Objectives: Establish the Intra and inter-rater reliability and the known-group and construct validity of the FIT-HaNSA in patients with Grade II WAD (WAD2). Methods: Twenty-five patients with WAD2 and 41 healthy controls were recruited. Numeric Pain Rating Scale (NPRS), Neck Disability Index (NDI), Disabilities of the Arm, Shoulder and Hand (DASH), cervical range of motion (CROM), and FIT-HaNSA were completed at two sessions conducted 2 to 7 days apart by two raters. Intraclass correlation coefficients (ICC) were used to describe Intra and inter-rater reliability. Spearman rank correlation coefficients (ρ) were used to quantify the associations between scores of the FIT-HaNSA and other measures in the WAD2 group (convergent construct validity). Results: The Intra and inter-ICCs for the FIT-HaNSA scores ranged from 0.88 to 0.89 in the control group and 0.78 to 0.85 in the WAD2 group. Statistically significant differences in FIT-HaNSA performance between the two groups suggested known group construct validity (P < 0.001). The correlations between the NPRS, NDI, DASH, CROM and FIT-HaNSA were generally poor (ρ < 0.4). Conclusion: The study results indicate that the total FIT-HaNSA score has good Intra and inter-rater reliability and the construct validity in WAD2 and healthy controls.

Evaluation of previously embolized intracranial aneurysms: inter-and intra-rater reliability among neurosurgeons and interventional neuroradiologists

2017 ◽  
Vol 10 (5) ◽  
pp. 462-466 ◽  
Author(s):  
Scott L Zuckerman ◽  
Nikita Lakomkin ◽  
Jordan A Magarik ◽  
Jan Vargas ◽  
Marcus Stephens ◽  
...  
Keyword(s):  
Intracranial Aneurysms ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Years Of Experience ◽  
Rater Agreement ◽  
Rater Reliability ◽  
Intraclass Correlation Coefficients ◽  
Level Of Agreement ◽  
Strong Agreement

BackgroundThe angiographic evaluation of previously coiled aneurysms can be difficult yet remains critical for determining re-treatment.ObjectiveThe main objective of this study was to determine the inter-rater reliability for both the Raymond Scale and per cent embolization among a group of neurointerventionalists evaluating previously embolized aneurysms.MethodsA panel of 15 neurointerventionalists examined 92 distinct cases of immediate post-coil embolization and 1 year post-embolization angiographs. Each case was presented four times throughout the study, along with alterations in demographics in order to evaluate intra-rater reliability. All respondents were asked to provide the per cent embolization (0–100%) and Raymond Scale grade (1-3) for each aneurysm. Inter-rater reliability was evaluated by computing weighted kappa values (for the Raymond Scale) and intraclass correlation coefficients (ICC) for per cent embolization.Results10 neurosurgeons and 5 interventional neuroradiologists evaluated 368 simulated cases. The agreement among all readers employing the Raymond Scale was fair (κ=0.35) while concordance in per cent embolization was good (ICC=0.64). Clinicians with fewer than 10 years of experience demonstrated a significantly greater level of agreement than the group with greater than 10 years (κ=0.39 and ICC=0.70 vs κ=0.28 and ICC=0.58). When the same aneurysm was presented multiple times, clinicians demonstrated excellent consistency when assessing per cent embolization (ICC=0.82), but moderate agreement when employing the Raymond classification (κ=0.58).ConclusionsIdentifying the per cent embolization in previously coiled aneurysms resulted in good inter- and intra-rater agreement, regardless of years of experience. The strong agreement among providers employing per cent embolization may make it a valuable tool for embolization assessment in this patient population.

scholarly journals Comparison of the Mayo Endoscopy Score and the Ulcerative Colitis Endoscopy Index of Severity and the Ulcerative Colitis Colonoscopy Index of Severity

2021 ◽  
Vol 09 (02) ◽  
pp. E130-E136
Author(s):  
María Belvis Jiménez ◽  
Pedro Hergueta-Delgado ◽  
Blas Gómez Rodríguez ◽  
Belén Maldonado Pérez ◽  
Luisa Castro Laria ◽  
...  
Keyword(s):  
Ulcerative Colitis ◽  
Interobserver Variability ◽  
Intraclass Correlation ◽  
Weighted Kappa ◽  
Clinical Situation ◽  
Kappa Index ◽  
Perfect Correlation ◽  
Index Of Severity ◽  
Severity Of The Disease ◽  
Normal Clinical Practice

Abstract Background and study aims: Endoscopy plays an essential role in managing patients with ulcerative colitis (UC), as it allows us to visualize and assess the severity of the disease. As such assessments are not always objective, different scores have been devised to standardize the findings. The main aim of this study was to assess the interobserver variability between the Mayo Endoscopy Score (MES), Ulcerative Colitis Endoscopy Index of Severity (UCEIS) and Ulcerative Colitis Colonoscopy Index of Severity (UCCIS) analyzing the severity of the endoscopic lesions in patients with ulcerative colitis. Patients and methods: This was a single-cohort observational study in which a colonoscopy was carried out on patients with UC, as normal clinical practice, and a video was recorded. The results from the video were classified according to the MES, UCEIS and UCCIS by three endoscopic specialists independently, and they were compared to each other. The Mayo Endoscopy Score (MES) was used to assess the clinical situation of the patient. The therapeutic impact was analyzed after colonoscopy was carried out. Results: Sixty-seven patients were included in the study. The average age was 51 (SD ± 16.7) and the average MES was 3.07 (SD ± 2.54). The weighted Kappa index between endoscopists A and B for the MES was 0.8; between A and C 0.52; and between B and C 0.49. The intraclass correlation coefficient for UCEIS was 0.92 among the three endoscopists (CI 95 %: 0.83–0.96) and 0.96 for UCCIS among the three endoscopists (CI 95 % 0.94–0.97). A change in treatment for 34.3 % of the patients was implemented on seeing the results of the colonoscopy. Conclusions: There was an adequate, but not perfect, correlation between the different endoscopists for MES, UCEIS, UCCIS. This was higher with the last two scores. Thus, there is still some subjectivity to be minimized through special training, on assessing the seriousness of the endoscopic lesions in patients with UC.

Adaptation and Validation of a Nutrition Environment Measures Survey for University Grab-and-Go Establishments

2016 ◽  
Vol 77 (1) ◽  
pp. 17-24 ◽  
Author(s):  
Brian K.C. Lo ◽  
Leia Minaker ◽  
Alicia N.T. Chan ◽  
Jessica Hrgetic ◽  
Catherine L. Mah
Keyword(s):  
Construct Validity ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Comparison Method ◽  
Nutrition Information ◽  
Perfect Agreement ◽  
Cross Sectional ◽  
Rater Reliability ◽  
University Of Toronto ◽  
Nutrition Environment

Purpose: To adapt and validate a survey instrument to assess the nutrition environment of grab-and-go establishments at a university campus. Methods: A version of the Nutrition Environment Measures Survey for grab-and-go establishments (NEMS-GG) was adapted from existing NEMS instruments and tested for reliability and validity through a cross-sectional assessment of the grab-and-go establishments at the University of Toronto. Product availability, price, and presence of nutrition information were evaluated. Cohen’s kappa coefficient and intra-class correlation coefficients (ICC) were assessed for inter-rater reliability, and construct validity was assessed using the known-groups comparison method (via store scores). Results: Fifteen grab-and-go establishments were assessed. Inter-rater reliability was high with an almost perfect agreement for availability (mean κ = 0.995) and store scores (ICC = 0.999). The tool demonstrated good face and construct validity. About half of the venues carried fruit and vegetables (46.7% and 53.3%, respectively). Regular and healthier entrée items were generally the same price. Healthier grains were cheaper than regular options. Six establishments displayed nutrition information. Establishments operated by the university’s Food Services consistently scored the highest across all food premise types for nutrition signage, availability, and cost of healthier options. Conclusions: Health promotion strategies are needed to address availability and variety of healthier grab-and-go options in university settings.

scholarly journals Adaptation and Psychometric Evaluation of the Chinese Counseling Competencies Scale-Revised

2021 ◽  
Vol 12 ◽  
Author(s):  
Wei Xia ◽  
William Ho Cheung Li ◽  
Tingna Liang ◽  
Yuanhui Luo ◽  
Laurie Long Kwan Ho ◽  
...  
Keyword(s):  
Concurrent Validity ◽  
Convergent Validity ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Psychometric Evaluation ◽  
Counseling Competencies ◽  
Counselors In Training ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Test Retest Reliability

Objectives: This study conducted a linguistic and psychometric evaluation of the Chinese Counseling Competencies Scale-Revised (CCS-R).Methods: The Chinese CCS-R was created from the original English version using a standard forward-backward translation process. The psychometric properties of the Chinese CCS-R were examined in a cohort of 208 counselors-in-training by two independent raters. Fifty-three counselors-in-training were asked to undergo another counseling performance evaluation for the test-retest. The confirmatory factor analysis (CFA) was conducted for the Chinese CCS-R, followed by internal consistency, test-retest reliability, inter-rater reliability, convergent validity, and concurrent validity.Results: The results of the CFA supported the factorial validity of the Chinese CCS-R, with adequate construct replicability. The scale had a McDonald's omega of 0.876, and intraclass correlation coefficients of 0.63 and 0.90 for test-retest reliability and inter-rater reliability, respectively. Significantly positive correlations were observed between the Chinese CCS-R score and scores of performance checklist (Pearson's γ = 0.781), indicating a large convergent validity, and knowledge on drug abuse (Pearson's γ = 0.833), indicating a moderate concurrent validity.Conclusion: The results support that the Chinese CCS-R is a valid and reliable measure of the counseling competencies.Practice implication: The CCS-R provides trainers with a reliable tool to evaluate counseling students' competencies and to facilitate discussions with trainees about their areas for growth.

Is the location of the signal intensity weighted centroid a reliable measurement of fluid displacement within the disc?

2018 ◽  
Vol 63 (4) ◽  
pp. 453-460 ◽  
Author(s):  
Vahid Abdollah ◽  
Eric C. Parent ◽  
Michele C. Battié
Keyword(s):  
Signal Intensity ◽  
Water Distribution ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Region Of Interest ◽  
Rater Reliability ◽  
Fluid Displacement ◽  
Intraclass Correlation Coefficients ◽  
The Mean ◽  
Standard Error Of Measurement

Abstract Degenerated discs have shorter T2-relaxation time and lower MR signal. The location of the signal-intensity-weighted-centroid reflects the water distribution within a region-of-interest (ROI). This study compared the reliability of the location of the signal-intensity-weighted-centroid to mean signal intensity and area measurements. L4-L5 and L5-S1 discs were measured on 43 mid-sagittal T2-weighted 3T MRI images in adults with back pain. One rater analysed images twice and another once, blinded to measurements. Discs were semi-automatically segmented into a whole disc, nucleus, anterior and posterior annulus. The coordinates of the signal-intensity-weighted-centroid for all regions demonstrated excellent intraclass-correlation-coefficients for intra- (0.99–1.00) and inter-rater reliability (0.97–1.00). The standard error of measurement for the Y-coordinates of the signal-intensity-weighted-centroid for all ROIs were 0 at both levels and 0 to 2.7 mm for X-coordinates. The mean signal intensity and area for the whole disc and nucleus presented excellent intra-rater reliability with intraclass-correlation-coefficients from 0.93 to 1.00, and 0.92 to 1.00 for inter-rater reliability. The mean signal intensity and area had lower reliability for annulus ROIs, with intra-rater intraclass-correlation-coefficient from 0.5 to 0.76 and inter-rater from 0.33 to 0.58. The location of the signal-intensity-weighted-centroid is a reliable biomarker for investigating the effects of disc interventions.

Sign in / Sign up

Export Citation Format

Share Document