scholarly journals Interrater reliability and convergent validity of F1000Prime peer review

2015 ◽  
Vol 66 (12) ◽  
pp. 2415-2426 ◽  
Author(s):  
Lutz Bornmann
Keyword(s):  
Peer Review ◽  
Convergent Validity ◽  
Interrater Reliability

CODEM Instrument

GeroPsych ◽  
2014 ◽  
Vol 27 (1) ◽  
pp. 23-31 ◽  
Author(s):  
Anne Kuemmel (This author contributed eq ◽  
Julia Haberstroh (This author contributed ◽  
Johannes Pantel
Keyword(s):  
Convergent Validity ◽  
Interrater Reliability ◽  
Discriminant Validity ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Well Being ◽  
Communication Behavior ◽  
People With Dementia ◽  
Pearson's R ◽  
Pearson’S R

Communication and communication behaviors in situational contexts are essential conditions for well-being and quality of life in people with dementia. Measuring methods, however, are limited. The CODEM instrument, a standardized observational communication behavior assessment tool, was developed and evaluated on the basis of the current state of research in dementia care and social-communicative behavior. Initially, interrater reliability was examined by means of videoratings (N = 10 people with dementia). Thereupon, six caregivers in six German nursing homes observed 69 residents suffering from dementia and used CODEM to rate their communication behavior. The interrater reliability of CODEM was excellent (mean κ = .79; intraclass correlation = .91). Statistical analysis indicated that CODEM had excellent internal consistency (Cronbach’s α = .95). CODEM also showed excellent convergent validity (Pearson’s R = .88) as well as discriminant validity (Pearson’s R = .63). Confirmatory factor analysis verified the two-factor solution of verbal/content aspects and nonverbal/relationship aspects. With regard to the severity of the disease, the content and relational aspects of communication exhibited different trends. CODEM proved to be a reliable, valid, and sensitive assessment tool for examining communication behavior in the field of dementia. CODEM also provides researchers a feasible examination tool for measuring effects of psychosocial intervention studies that strive to improve communication behavior and well-being in dementia.

scholarly journals Are disruption index indicators convergently valid? The comparison of several indicator variants with assessments by peers

2020 ◽  
Vol 1 (3) ◽  
pp. 1242-1259
Author(s):  
Lutz Bornmann ◽  
Sitaram Devarakonda ◽  
Alexander Tekles ◽  
George Chacko
Keyword(s):  
Factor Analysis ◽  
Peer Review ◽  
Convergent Validity ◽  
Scientific Publication ◽  
Peer Review System ◽  
Regression Analyses ◽  
Review System ◽  
New Family ◽  
New Findings ◽  
Better Than

Recently, Wu, Wang, and Evans (2019) proposed a new family of indicators, which measure whether a scientific publication is disruptive to a field or tradition of research. Such disruptive influences are characterized by citations to a focal paper, but not its cited references. In this study, we are interested in the question of convergent validity. We used external criteria of newness to examine convergent validity: In the postpublication peer review system of F1000Prime, experts assess papers whether the reported research fulfills these criteria (e.g., reports new findings). This study is based on 120,179 papers from F1000Prime published between 2000 and 2016. In the first part of the study we discuss the indicators. Based on the insights from the discussion, we propose alternate variants of disruption indicators. In the second part, we investigate the convergent validity of the indicators and the (possibly) improved variants. Although the results of a factor analysis show that the different variants measure similar dimensions, the results of regression analyses reveal that one variant ( DI5) performs slightly better than the others.

Psychometric Properties of Functional Balance Assessment in Children With Cerebral Palsy

2008 ◽  
Vol 22 (6) ◽  
pp. 745-753 ◽  
Author(s):  
Sue-Mae Gan ◽  
Li-Chen Tung ◽  
Yue-Her Tang ◽  
Chun-Hou Wang
Keyword(s):  
Cerebral Palsy ◽  
Motor Function ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Walking Speed ◽  
Retest Reliability ◽  
Functional Balance ◽  
Sit To Stand ◽  
Children With Cerebral Palsy ◽  
Test Retest Reliability

Background. Children with cerebral palsy often suffer from a lack of balance compared with typically developing children. Because balance capacity is relevant to functional activities, reliable and valid functional balance measures are crucial for the pediatric clinical setting. Objective. This study examined the reliability and validity of 3 functional balance measures. Methods. Thirty children aged 60 to 142 months with Gross Motor Function Classification System (GMFCS) levels of I to IV were recruited. For test-retest reliability, the same physical therapist administered the Functional Reach Test (FRT), Berg Balance Scale (BBS), and Timed Up and Go (TUG) twice. For interrater reliability, the testing processes were video recorded and later scored by another therapist. For convergent validity, children with cerebral palsy received the Gross Motor Function Measures (GMFM), walking speed, and 10-second sit-to-stand test within 1 week and the results evaluated. Results. The 3 functional balance measures had excellent test-retest reliability (intraclass correlation coefficient [ICC] >0.95) and interrater reliability (ICC = 0.98-1.00). With regard to convergent validity, the BBS and the TUG were highly correlated with GMFM total score, walking speed, and the 10-second sit-to-stand test. The discriminate validity indicates that the FRT can distinguish children with cerebral palsy with different GMFCS levels, whereas the BBS total score and TUG failed to distinguish between children with cerebral palsy with GMFCS levels of I and II. Conclusion. The 3 functional balance measures are simple, valid, and reliable for examining children with cerebral palsy and are thus suitable for clinical practice.

Psychometric Properties of a Structured Diagnostic Interview for DSM-5 Anxiety, Mood, and Obsessive-Compulsive and Related Disorders

Assessment ◽  
2016 ◽  
Vol 25 (1) ◽  
pp. 3-13 ◽  
Author(s):  
David F. Tolin ◽  
Christina Gilliam ◽  
Bethany M. Wootton ◽  
William Bowe ◽  
Laura B. Bragdon ◽  
...  
Keyword(s):  
Convergent Validity ◽  
Interrater Reliability ◽  
Diagnostic Interview ◽  
Self Report ◽  
Obsessive Compulsive ◽  
Reliability Data ◽  
Retest Reliability ◽  
Dsm 5 ◽  
Structured Diagnostic Interview ◽  
Test Retest Reliability

Three hundred sixty-two adult patients were administered the Diagnostic Interview for Anxiety, Mood, and OCD and Related Neuropsychiatric Disorders (DIAMOND). Of these, 121 provided interrater reliability data, and 115 provided test–retest reliability data. Participants also completed a battery of self-report measures that assess symptoms of anxiety, mood, and obsessive-compulsive and related disorders. Interrater reliability of DIAMOND anxiety, mood, and obsessive-compulsive and related diagnoses ranged from very good to excellent. Test–retest reliability of DIAMOND diagnoses ranged from good to excellent. Convergent validity was established by significant between-group comparisons on applicable self-report measures for nearly all diagnoses. The results of the present study indicate that the DIAMOND is a promising semistructured diagnostic interview for DSM-5 disorders.

scholarly journals Assessing Balance Function in Patients With Total Knee Arthroplasty

2015 ◽  
Vol 95 (10) ◽  
pp. 1397-1407 ◽  
Author(s):  
Andy C.M. Chan ◽  
Marco Y.C. Pang
Keyword(s):  
Total Knee Arthroplasty ◽  
Internal Consistency ◽  
Knee Arthroplasty ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Cronbach Alpha ◽  
Ceiling Effects ◽  
Total Knee ◽  
Abc Scale

BackgroundThe Balance Evaluation Systems Test (BESTest) is a relatively new balance assessment tool. Recently, the Mini-BESTest and the Brief-BESTest, which are shortened versions of the BESTest, were developed.ObjectiveThe purpose of this study was to estimate interrater and intrarater-interoccasion reliability, internal consistency, concurrent and convergent validity, and floor and ceiling effects of the 3 BESTests and other related measures, namely, the Berg Balance Scale (BBS), Functional Gait Assessment (FGA), and Activities-specific Balance Confidence (ABC) Scale, among patients with total knee arthroplasty (TKA).DesignThis was an observational measurement study.MethodsTo establish interrater reliability, the 3 BESTests were administered by 3 independent raters to 25 participants with TKA. Intrarater-interoccasion reliability was evaluated in 46 participants with TKA (including the 25 individuals who participated in the interrater reliability experiments) by repeating the 3 BESTests, BBS, and FGA within 1 week by the same rater. Internal consistency of each test also was assessed with Cronbach alpha. Validity was assessed in another 46 patients with TKA by correlating the 3 BESTests with BBS, FGA, and ABC. The floor and ceiling effects also were examined.ResultsThe 3 BESTests demonstrated excellent interrater reliability (intraclass correlation coefficient [ICC] [2,1]=.96–.99), intrarater-interoccasion reliability (ICC [2,1]=.92–.96), and internal consistency (Cronbach alpha=.96–.98). These values were comparable to those for the BBS and FGA. The 3 BESTests also showed moderate-to-strong correlations with the BBS, FGA, and ABC (r=.35–.81), thus demonstrating good concurrent and convergent validity. No significant floor and ceiling effects were observed, except for the BBS.LimitationsThe results are generalizable only to patients with TKA due to end-stage knee osteoarthritis.ConclusionsThe 3 BESTests have good reliability and validity for evaluating balance in people with TKA. The Brief-BESTest is the least time-consuming and may be more useful clinically.

scholarly journals Development and psychometric evaluation of performance-based upper extremity motor control test (PB-UE-MCT) for people with cerebral palsy

2021 ◽  
Vol 29 ◽  
Author(s):  
Fatemeh Etemad Shams ◽  
Malek Amini ◽  
Afsoon Hassani Mehraban ◽  
Mehdi Alizadeh Zarei ◽  
Dorsa Kalani
Keyword(s):  
Cerebral Palsy ◽  
Motor Control ◽  
Upper Extremity ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Correlation Coefficients ◽  
Psychometric Evaluation ◽  
Control Test ◽  
Intrarater Reliability

Abstract Introduction Upper extremity disorders limits the performance in the activities of daily living, especially, in bilateral (two-handed) activities. Objective This study was designed to develop a performance-based upper extremity motor control test (PB-UE-MCT) and to measure its psychometric properties (including, convergent validity, intrarater reliability, and interrater reliability) for people with cerebral palsy (CP). Method The PB-UE-MCT was developed in three phases, including planning, construction, and psychometric evaluation. The participants included 50 people with CP with an age range of 6 to 18 years. To measure internal consistency, Cronbach's alpha was run (n=50). Intrarater and interrater reliability was measured for 25 participants. To assess convergent validity, the correlations of the PB-UE-MCT with the Manual Ability Classification System (MACS) and with the Selective Control Upper Extremity Scale (SCUES) were calculated. Results The values obtained for Cronbach's alpha (.90 to .96) indicated the excellent internal consistency of the PB-UE-MCT. The ICC values for intrarater reliability and interrater reliability were between .84 and .99 and between .89 and .99, respectively. The correlation coefficients obtained for the items of the PB-UE-MCT and those of the MACS were between .51 and .73. The correlation coefficients of the items of the PB-UE-MCT with those of the SCUES were in the range of .67 to .98, which proves the PB-UE-MCT's good to excellent levels of convergent validity. Conclusion The results confirm that the PB-UE-MCT is a valid and reliable tool for evaluating the upper extremity performance of people with CP through task analysis.

scholarly journals Adaptation and Validation of the Program Sustainability Assessment Tool (PSAT) for Use in the Elementary School Setting

2021 ◽  
Vol 18 (21) ◽  
pp. 11414
Author(s):  
Alix Hall ◽  
Adam Shoesmith ◽  
Rachel C. Shelton ◽  
Cassandra Lane ◽  
Luke Wolfenden ◽  
...  
Keyword(s):  
Public Health ◽  
Physical Activity ◽  
Elementary School ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Assessment Tool ◽  
Sustainability Assessment ◽  
School Setting ◽  
Program Sustainability ◽  
Structural Validity

There is a lack of valid and reliable measures of determinants of sustainability specific to public health interventions in the elementary school setting. This study aimed to adapt and evaluate the Program Sustainability Assessment Tool (PSAT) for use in this setting. An expert reference group adapted the PSAT to ensure face validity. Elementary school teachers participating in a multi-component implementation intervention to increase their scheduling of physical activity completed the adapted PSAT. Structural validity was assessed via confirmatory factor analysis. Convergent validity was assessed using linear mixed regression evaluating the associations between scheduling of physical activity and adapted PSAT scores. Cronbach’s alpha was used to evaluate internal consistency and intracluster correlation coefficients for interrater reliability. Floor and ceiling effects were also evaluated. Following adaptation and psychometric evaluation, the final measure contained 26 items. Domain Cronbach’s alpha ranged from 0.77 to 0.92. Only one domain illustrated acceptable interrater reliability. Evidence for structural validity was mixed and was lacking for convergent validity. There were no floor and ceiling effects. Efforts to adapt and validate the PSAT for the elementary school setting were mixed. Future work to develop and improve measures specific to public health program sustainment that are relevant and psychometrically robust for elementary school settings are needed.

Sign in / Sign up

Export Citation Format

Share Document