Assessment of reliability and validity of the 5-scale grading system of the point-of-care immunoassay for tear matrix metalloproteinase-9

AbstractWe evaluated the reliability and validity of the 5-scale grading system to interpret the point-of-care immunoassay for tear matrix metalloproteinase (MMP)-9. Six observers graded red bands of photographs of the readout window in MMP-9 immunoassay kit (InflammaDry) two times with 2-week interval based on the 5-scale grading system (i.e. grade 0–4). Interobserver and intraobserver reliability were evaluated using intraclass correlation coefficients. The interobserver agreements were analyzed according to the severity of tear MMP-9 expression. To validate the system, a concentration calibration curve was made using MMP-9 solutions with reference concentrations, then the distribution of MMP-9 concentrations was analyzed according to the 5-scale grading system. Both intraobserver and interobserver reliability was excellent. The readout grades were significantly correlated with the quantified colorimetric densities. The interobserver variance of readout grades had no correlation with the severity of the measured densities. The band density continued to increase up to a maximal concentration (i.e. 5000 ng/mL) according to the calibration curve. The difference of grades reflected the change of MMP-9 concentrations sensitively, especially between grade 2 and 4. Together, our data indicate that the subjective 5-scale grading system in the point-of-care MMP-9 immunoassay is an easy and reliable method with acceptable accuracy.

Download Full-text

Toronto Facial Grading System: Interobserver reliability

Otolaryngology ◽

10.1016/s0194-5998(00)70241-5 ◽

2000 ◽

Vol 122 (2) ◽

pp. 212-215 ◽

Cited By ~ 40

Author(s):

Fatma Tulin Kayhan ◽

David Zurakowski ◽

Steven D. Rauch

Keyword(s):

Facial Nerve ◽

Interobserver Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Ease Of Use ◽

Composite Score ◽

Grading System ◽

Perfect Agreement ◽

Intraclass Correlation Coefficients ◽

Facial Function

The Toronto Facial Grading System (TFGS) is an observer scale for rating facial nerve dysfunction. The TFGS scores aspects of resting symmetry, symmetry of voluntary movement, and synkinesis for each division of the face (subscores) and then provides calculated total scores and an overall composite score of facial function. The developers of the scale have validated its sensitivity for identifying small changes in facial dysfunction and the independence of the different components measured. Herein we report our results in a study of interob-server reliability using the TFGS. Twenty-five patients from the Massachusetts Eye and Ear Infirmary Facial Nerve Center with varying degrees of facial paresis, paralysis, and synkinesis were videotaped, and the video recordings were scored by 5 independent observers using the TFGS. Intraclass correlation coefficients (κ) and 95% confidence intervals were calculated for subscores and for each total and composite score. Intraclass correlation coefficients ranged from 0.59 to 0.85, all considered substantial to near-perfect agreement between observers. We believe the TFGS is superior to other scales by virtue of its sensitivity, comprehensiveness, ease of use, and interobserver reliability. The TFGS presently appears to be the best option in those situations in which accurate and precise documentation of facial function is required.

Download Full-text

Interobserver Reliability Using the Phonetic Level Evaluation With Severely and Profoundly Hearing-Impaired Children

Journal of Speech Language and Hearing Research ◽

10.1044/jshr.3405.989 ◽

1991 ◽

Vol 34 (5) ◽

pp. 989-999 ◽

Cited By ~ 6

Author(s):

Stephanie Shaw ◽

Truman E. Coggins

Keyword(s):

Interrater Reliability ◽

Interobserver Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Hearing Impaired ◽

Intraclass Correlation Coefficients ◽

Assessment Measure ◽

Impaired Children ◽

Speech Assessment ◽

Hearing Impaired Children

This study examines whether observers reliably categorize selected speech production behaviors in hearing-impaired children. A group of experienced speech-language pathologists was trained to score the elicited imitations of 5 profoundly and 5 severely hearing-impaired subjects using the Phonetic Level Evaluation (Ling, 1976). Interrater reliability was calculated using intraclass correlation coefficients. Overall, the magnitude of the coefficients was found to be considerably below what would be accepted in published behavioral research. Failure to obtain acceptably high levels of reliability suggests that the Phonetic Level Evaluation may not yet be an accurate and objective speech assessment measure for hearing-impaired children.

Download Full-text

Interobserver Reliability and Change in the Sagittal Tibial Tubercle–Trochlear Groove Distance with Increasing Knee Flexion Angles

The Journal of Knee Surgery ◽

10.1055/s-0041-1729547 ◽

2021 ◽

Author(s):

Ian S. MacLean ◽

Taylor M. Southworth ◽

Ian J. Dempsey ◽

Neal B. Naveen ◽

Hailey P. Huddleston ◽

...

Keyword(s):

Knee Flexion ◽

Sagittal Plane ◽

Interobserver Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Flexion Angle ◽

Tibial Tubercle ◽

Trochlear Groove ◽

Knee Flexion Angle ◽

Intraclass Correlation Coefficients

AbstractThe tibial tubercle–trochlear groove (TT-TG) distance is currently utilized to evaluate knee alignment in patients with patellar instability. Sagittal plane pathology measured by the sagittal tibial tubercle–trochlear groove (sTT-TG) distance has been described in instability but may also be important to consider in patients with cartilage injury. This study aims to (1) describe interobserver reliability of the sTT-TG distance and (2) characterize the change in the sTT-TG distance with respect to changing knee flexion angles. In this cadaveric study, six nonpaired cadaveric knees underwent magnetic resonance imaging (MRI) studies at each of the following degrees of knee flexion: −5, 0, 5, 10, 15, and 20. The sTT-TG distance was measured on the axial T2 sequence. Four reviewers measured this distance for each cadaver at each flexion angle. Intraclass correlation coefficients were calculated to determine interobserver reliability and reproducibility of the sTT-TG measurement. Analysis of variance (ANOVA) tests and Friedman's tests with a Bonferroni's correction were performed for each cadaver to compare sTT-TG distances at each flexion angle. Significance was defined as p < 0.05. There was excellent interobserver reliability of the sTT-TG distance with all intraclass correlation coefficients >0.9. The tibial tubercle progressively becomes more posterior in relation to the trochlear groove (more negative sTT-TG distance) with increasing knee flexion. The sTT-TG distance is a measurement that is reliable between attending surgeons and across training levels. The sTT-TG distance is affected by small changes in knee flexion angle. Awareness of knee flexion angle on MRI is important when this measurement is utilized by surgeons.

Download Full-text

Using Smart Bracelets to Assess Heart Rate Among Students During Physical Education Lessons: Feasibility, Reliability, and Validity Study (Preprint)

10.2196/preprints.17699 ◽

2020 ◽

Author(s):

Jiangang Sun ◽

Yang Liu

Keyword(s):

Heart Rate ◽

Physical Education ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Left Wrist ◽

Intraclass Correlation Coefficients ◽

Estimate Reliability ◽

Left And Right ◽

The Right

BACKGROUND An increasing number of wrist-worn wearables are being examined in the context of health care. However, studies of their use during physical education (PE) lessons remain scarce. OBJECTIVE We aim to examine the reliability and validity of the Fizzo Smart Bracelet (Fizzo) in measuring heart rate (HR) in the laboratory and during PE lessons. METHODS In Study 1, 11 healthy subjects (median age 22.0 years, IQR 3.75 years) twice completed a test that involved running on a treadmill at 6 km/h for 12 minutes and 12 km/h for 5 minutes. During the test, participants wore two Fizzo devices, one each on their left and right wrists, to measure their HR. At the same time, the Polar Team2 Pro (Polar), which is worn on the chest, was used as the standard. In Study 2, we went to 10 schools and measured the HR of 24 students (median age 14.0 years, IQR 2.0 years) during PE lessons. During the PE lessons, each student wore a Polar device on their chest and a Fizzo on their right wrist to measure HR data. At the end of the PE lessons, the students and their teachers completed a questionnaire where they assessed the feasibility of Fizzo. The measurements taken by the left wrist Fizzo and the right wrist Fizzo were compared to estimate reliability, while the Fizzo measurements were compared to the Polar measurements to estimate validity. To measure reliability, intraclass correlation coefficients (ICC), mean difference (MD), standard error of measurement (SEM), and mean absolute percentage errors (MAPE) were used. To measure validity, ICC, limits of agreement (LOA), and MAPE were calculated and Bland-Altman plots were constructed. Percentage values were used to estimate the feasibility of Fizzo. RESULTS The Fizzo showed excellent reliability and validity in the laboratory and moderate validity in a PE lesson setting. In Study 1, reliability was excellent (ICC>0.97; MD<0.7; SEM<0.56; MAPE<1.45%). The validity as determined by comparing the left wrist Fizzo and right wrist Fizzo was excellent (ICC>0.98; MAPE<1.85%). Bland-Altman plots showed a strong correlation between left wrist Fizzo measurements (bias=0.48, LOA=–3.94 to 4.89 beats per minute) and right wrist Fizzo measurements (bias=0.56, LOA=–4.60 to 5.72 beats per minute). In Study 2, the validity of the Fizzo was lower compared to that found in Study 1 but still moderate (ICC>0.70; MAPE<9.0%). The Fizzo showed broader LOA in the Bland-Altman plots during the PE lessons (bias=–2.60, LOA=–38.89 to 33.69 beats per minute). Most participants considered the Fizzo very comfortable and easy to put on. All teachers thought the Fizzo was helpful. CONCLUSIONS When participants ran on a treadmill in the laboratory, both left and right wrist Fizzo measurements were accurate. The validity of the Fizzo was lower in PE lessons but still reached a moderate level. The Fizzo is feasible for use during PE lessons.

Download Full-text

3D Biometrics for Hindfoot Alignment Using Weightbearing Computed Tomography

Foot & Ankle International ◽

10.1177/1071100719835492 ◽

2019 ◽

Vol 40 (6) ◽

pp. 720-726 ◽

Cited By ~ 24

Author(s):

Jian Zhong Zhang ◽

François Lintz ◽

Alessio Bernasconi ◽

Shu Zhang ◽

Keyword(s):

Computed Tomography ◽

Comparative Study ◽

Interobserver Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Level Of Evidence ◽

Hindfoot Alignment ◽

Mean Values ◽

Intraclass Correlation Coefficients ◽

Prospective Comparative Study

Background: Weightbearing computed tomography (WBCT) is a useful tool for the assessment of hindfoot alignment (HA). Foot ankle offset (FAO) is a recently introduced parameter, determined from WBCT images using semiautomatic software. The aim of this study was to determine the clinical relevance and reproducibility of FAO for the evaluation of HA. Methods: A prospective comparative study was performed on consecutive patients requiring bilateral WBCT between September 2017 and April 2018. Based on the clinical assessment of HA, patients were divided into 3 groups: (1) normal alignment group (G1), (2) valgus (G2), and (3) varus (G3). FAO and long axial view (HACT) were measured on WBCT images, and the groups were compared. The reproducibility of FAO and HACT was determined through intraclass correlation coefficients (ICCs). Regression analysis was performed to investigate the correlation between the 2 methods. Overall, 249 feet (126 patients) were included (G1 = 115, G2 = 78, and G3 = 56 feet). Results: The mean values for FAO and HACT were 1.2% ± 2.8% and 3.9 ± 3.1, respectively, in G1; 8.1% ± 3.7% and 9.7 ± 4.9 in G2; and −6.6% ± 4.8% and −8.2 ± 6.6 in G3. Intra- and interobserver reliability was 0.987 and 0.988 for FAO and 0.949 and 0.949 for HACT, respectively. There was a good linear correlation between HACT and FAO ( R2 = 0.744), with a regression slope of 1.064. Conclusions: WBCT was a useful method for the characterization of HA. FAO was reproducible and correlated well with physical examination. Level of Evidence: Level II, prospective comparative study.

Download Full-text

The Reliability and Validity of the Loadsol® under Various Walking and Running Conditions

Sensors ◽

10.3390/s19020265 ◽

2019 ◽

Vol 19 (2) ◽

pp. 265 ◽

Cited By ~ 12

Author(s):

Kristen Renner ◽

DS Williams ◽

Robin Queen

Keyword(s):

Reliable Method ◽

Loading Rate ◽

Reaction Force ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Validity And Reliability ◽

Intraclass Correlation Coefficients ◽

Single Sensor ◽

Force Data

The assessment of loading during walking and running has historically been limited to data collection in laboratory settings or with devices that require a computer connection. This study aims to determine if the loadsol®—a single sensor wireless insole—is a valid and reliable method of assessing force. Thirty (17 male and 13 female) recreationally active individuals were recruited for a two visit study where they walked (1.3 m/s) and ran (3.0 and 3.5 m/s) at a 0%, 10% incline, and 10% decline, with the visits approximately one week apart. Ground reaction force data was collected on an instrumented treadmill (1440 Hz) and with the loadsol® (100 Hz). Ten individuals completed the day 1 protocol with a newer 200 Hz loadsol®. Intraclass correlation coefficients (ICC3,k) were used to assess validity and reliability and Bland–Altman plots were generated to better understand loadsol® validity. Across conditions, the peak force ICCs ranged from 0.78 to 0.97, which increased to 0.84–0.99 with the 200 Hz insoles. Similarly, the loading rate ICCs improved from 0.61 to 0.97 to 0.80–0.96 and impulse improved from 0.61 to 0.97 to 0.90–0.97. The 200 Hz insoles may be needed for loading rate and impulse in running. For both walking and running, the loadsol® has excellent between-day reliability (>0.76).

Download Full-text

Alberta Infant Motor Scale: Reliability and Validity When Used on Preterm Infants in Taiwan

Physical Therapy ◽

10.1093/ptj/80.2.168 ◽

2000 ◽

Vol 80 (2) ◽

pp. 168-178 ◽

Cited By ~ 76

Author(s):

Suh-Fang Jeng ◽

Kuo-Inn Tsou Yau ◽

Li-Chiou Chen ◽

Shu-Fang Hsiao

Keyword(s):

Preterm Infants ◽

Interrater Reliability ◽

Physical Therapist ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Intraclass Correlation Coefficients ◽

Scale Reliability ◽

Scale Scores ◽

Acceptable Reliability

Abstract Background and Purpose. The goal of this study was to examine the reliability and validity of measurements obtained with the Alberta Infant Motor Scale (AIMS) for evaluation of preterm infants in Taiwan. Subjects. Two independent groups of preterm infants were used to investigate the reliability (n=45) and validity (n=41) for the AIMS. Methods. In the reliability study, the AIMS was administered to the infants by a physical therapist, and infant performance was videotaped. The performance was then rescored by the same therapist and by 2 other therapists to examine the intrarater and interrater reliability. In the validity study, the AIMS and the Bayley Motor Scale were administered to the infants at 6 and 12 months of age to examine criterion-related validity. Results. Intraclass correlation coefficients (ICCs) for intrarater and interrater reliability of measurements obtained with the AIMS were high (ICC=.97–.99). The AIMS scores correlated with the Bayley Motor Scale scores at 6 and 12 months (r=.78 and .90), although the AIMS scores at 6 months were only moderately predictive of the motor function at 12 months (r=.56). Conclusion and Discussion. The results suggest that measurements obtained with the AIMS have acceptable reliability and concurrent validity but limited predictive value for evaluating preterm Taiwanese infants.

Download Full-text

Reliability and Validity of the Posture and Fine Motor Assessment of Infants

The Occupational Therapy Journal of Research ◽

10.1177/153944928900900501 ◽

1989 ◽

Vol 9 (5) ◽

pp. 259-272 ◽

Cited By ~ 3

Author(s):

Jane Case-Smith

Keyword(s):

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Fine Motor ◽

Motor Assessment ◽

Intraclass Correlation Coefficients ◽

Highly Correlated ◽

Peabody Developmental Motor Scales ◽

The Individual ◽

Test Retest Reliability

The Posture and Fine Motor Assessment of Infants (PFMAI) (Case-Smith, 1987) is a newly developed instrument for assessing the quality of motor function in infants. The test measures components of posture and fine motor control as they first develop. The purpose of this study was to support the test's reliability and validity. Interrater reliability, analyzed with intraclass correlation coefficients (ICCs), was high (.989 for total scores). Test-retest reliability, measured by ICCs, was .853 and .913 for the two test sections. The PFMAI demonstrated concurrent validity with the Peabody Developmental Motor Scales, Revised (Folio & Fewell, 1983) (correlations were .673 and .829 for the individual sections). Scores on the PFMAI were highly correlated with the infant's ages (.892 to .941); this finding provided one indication of construct validity.

Download Full-text

Spinal Instability Neoplastic Score: An Analysis of Reliability and Validity From the Spine Oncology Study Group

Journal of Clinical Oncology ◽

10.1200/jco.2010.34.3897 ◽

2011 ◽

Vol 29 (22) ◽

pp. 3072-3077 ◽

Cited By ~ 277

Author(s):

Daryl R. Fourney ◽

Evan M. Frangou ◽

Timothy C. Ryken ◽

Christian P. DiPaola ◽

Christopher I. Shaffrey ◽

...

Keyword(s):

Predictive Validity ◽

Interobserver Reliability ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Intraobserver Reliability ◽

Spinal Tumors ◽

Spinal Instability ◽

Study Group ◽

Reliable Classification

PurposeStandardized indications for treatment of tumor-related spinal instability are hampered by the lack of a valid and reliable classification system. The objective of this study was to determine the interobserver reliability, intraobserver reliability, and predictive validity of the Spinal Instability Neoplastic Score (SINS).MethodsClinical and radiographic data from 30 patients with spinal tumors were classified as stable, potentially unstable, and unstable by members of the Spine Oncology Study Group. The median category for each patient case (consensus opinion) was used as the gold standard for predictive validity testing. On two occasions at least 6 weeks apart, each rater also scored each patient using SINS. Each total score was converted into a three-category data field, with 0 to 6 as stable, 7 to 12 as potentially unstable, and 13 to 18 as unstable.ResultsThe κ statistics for interobserver reliability were 0.790, 0.841, 0.244, 0.456, 0.462, and 0.492 for the fields of location, pain, bone quality, alignment, vertebral body collapse, and posterolateral involvement, respectively. The κ statistics for intraobserver reliability were 0.806, 0.859, 0.528, 0.614, 0.590, and 0.662 for the same respective fields. Intraclass correlation coefficients for inter- and intraobserver reliability of total SINS score were 0.846 (95% CI, 0.773 to 0.911) and 0.886 (95% CI, 0.868 to 0.902), respectively. The κ statistic for predictive validity was 0.712 (95% CI, 0.676 to 0.766).ConclusionSINS demonstrated near-perfect inter- and intraobserver reliability in determining three clinically relevant categories of stability. The sensitivity and specificity of SINS for potentially unstable or unstable lesions were 95.7% and 79.5%, respectively.

Download Full-text

Assessment of Technetium-99m Labeled Macroaggregated Albumin Rhinoscintigraphy for the Measurement of Nasal Mucociliary Transport Rate: Intratest, Interobserver, and Intraobserver Reproducibility

Scientifica ◽

10.1155/2014/982515 ◽

2014 ◽

Vol 2014 ◽

pp. 1-5

Author(s):

Zeki Dostbil ◽

Yusuf Dag ◽

Ozlem Cetinkaya ◽

Mehmet Akdag ◽

Bekir Tasdemir

Keyword(s):

Interobserver Reliability ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Transport Rate ◽

Mucociliary Transport ◽

Intraclass Correlation Coefficients ◽

Intraobserver Reproducibility ◽

Macroaggregated Albumin ◽

Statistical Equivalence ◽

Intraobserver Variation

Objectives.The measurement of mucociliary transport velocity by rhinoscintigraphy with Tc-99m-macroaggregated albumin (99mTc-MAA) is reliable measure of mucociliary clearance. The aim of this study is to assess the intratest, interobserver, and intraobserver reproducibility of nasal mucociliary transport rate (NMTR) measurement.Materials and Methods.Twenty-two subjects were evaluated to determine intratest reproducibility and a group of 35 subjects was examined to determine inter- and intraobserver reproducibility. Rhinoscintigraphy with99mTc-MAA was used to measure NMTR in all study subjects. Paired NMTR measurements were compared using a range of statistical methodologies. Intraclass correlation coefficients (ICC) and repeatability coefficients and Bland-Altman plots were applied to assess the degree of intratest, interobserver, and intraobserver variation.Results.Statistical analysis of test and retest experiments demonstrated the statistical equivalence of intratest NMTR measurements, interobserver NMTR measurements, and intraobserver NMTR measurements. The intratest ICC, interobserver ICC, and intraobserver ICC were 0.96, 0.83, and 0.91, respectively, indicating that intratest and intraobserver reproducibility are excellent and interobserver reproducibility is good.Conclusions.Rhinoscintigraphy using99mTc-MAA results in highly reproducible measurement of NMTR. The use of radionuclide imaging in measuring NMTR results in excellent intratest and intraobserver reproducibility and good interobserver reliability.

Download Full-text