Reliability and validity of the iSense optical scanner for measuring volume of transtibial residual limb models

Background: Residual limb volume is often measured as part of routine care for people with amputations. These measurements assist in the timing of prosthetic fitting or replacement. In order to make well informed decisions, clinicians need access to measurement tools that are valid and reliable. Objectives: To assess the reliability and criterion validity of the iSense optical scanner in measuring volume of transtibial residual limb models. Study Design: Three assessors performed two measurements each on 13 residual limb models with an iSense optical scanner (3D systems, USA). Intra-rater and inter-rater reliability were calculated using intraclass correlation coefficients. Bland Altman plots were inspected for agreement. Criterion validity was assessed using a steel rod of known dimensions. Ten repeated measurements were performed by one assessor. A t-test was used to determine differences between measured and true rod volume. Results: Intra-rater reliability was excellent (range of intraclass correlation coefficients: 0.991–0.997, all with narrow 95% confidence intervals). While the intraclass correlation coefficients suggest excellent inter-rater reliability between all three assessors (range of intraclass correlation coefficients: 0.952–0.986), the 95% confidence intervals were wide between assessor 3 and the other two assessors. Poor agreement with assessor 3 was also seen in the Bland-Altman plots. Criterion validity was very poor with a significant difference between the mean iSense measurement and the true rod volume (difference: 221.18 mL; p < 0.001). Conclusions: Although intra-rater reliability was excellent for the iSense scanner, we did not find similar results for inter-rater reliability and validity. These results suggest that further testing of the iSense scanner is required prior to use in clinical practice. Clinical relevance The iSense offers a low cost scanning option for residual limb volume measurement. Intra-rater reliability was excellent, but inter-rater reliability and validity were such that clinical adoption is not indicated at present.

Download Full-text

Assessment of reliability and validity of the 5-scale grading system of the point-of-care immunoassay for tear matrix metalloproteinase-9

Scientific Reports ◽

10.1038/s41598-021-92020-6 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Minjeong Kim ◽

Ja Young Oh ◽

Seon Ha Bae ◽

Seung Hyeun Lee ◽

Won Jun Lee ◽

...

Keyword(s):

Matrix Metalloproteinase ◽

Calibration Curve ◽

Point Of Care ◽

Interobserver Reliability ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Grading System ◽

Intraclass Correlation Coefficients ◽

The Difference

AbstractWe evaluated the reliability and validity of the 5-scale grading system to interpret the point-of-care immunoassay for tear matrix metalloproteinase (MMP)-9. Six observers graded red bands of photographs of the readout window in MMP-9 immunoassay kit (InflammaDry) two times with 2-week interval based on the 5-scale grading system (i.e. grade 0–4). Interobserver and intraobserver reliability were evaluated using intraclass correlation coefficients. The interobserver agreements were analyzed according to the severity of tear MMP-9 expression. To validate the system, a concentration calibration curve was made using MMP-9 solutions with reference concentrations, then the distribution of MMP-9 concentrations was analyzed according to the 5-scale grading system. Both intraobserver and interobserver reliability was excellent. The readout grades were significantly correlated with the quantified colorimetric densities. The interobserver variance of readout grades had no correlation with the severity of the measured densities. The band density continued to increase up to a maximal concentration (i.e. 5000 ng/mL) according to the calibration curve. The difference of grades reflected the change of MMP-9 concentrations sensitively, especially between grade 2 and 4. Together, our data indicate that the subjective 5-scale grading system in the point-of-care MMP-9 immunoassay is an easy and reliable method with acceptable accuracy.

Download Full-text

Image processing and machine learning for telehealth craniosynostosis screening in newborns

Journal of Neurosurgery Pediatrics ◽

10.3171/2020.9.peds20605 ◽

2021 ◽

pp. 1-8

Author(s):

Markus J. Bookland ◽

Edward S. Ahn ◽

Petronella Stoltz ◽

Jonathan E. Martin

Keyword(s):

Image Processing ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Asymmetry Index ◽

Optical Measurements ◽

Shape Classification ◽

Data Set ◽

Intraclass Correlation Coefficients ◽

Optical Scanner ◽

Cranial Shape

OBJECTIVE The authors sought to evaluate the accuracy of a novel telehealth-compatible diagnostic software system for identifying craniosynostosis within a newborn (< 1 year old) population. Agreement with gold standard craniometric diagnostics was also assessed. METHODS Cranial shape classification software accuracy was compared to that of blinded craniofacial specialists using a data set of open-source (n = 40) and retrospectively collected newborn orthogonal top-down cranial images, with or without additional facial views (n = 339), culled between April 1, 2008, and February 29, 2020. Based on image quality, midface visibility, and visibility of the cranial equator, 351 image sets were deemed acceptable. Accuracy, sensitivity, and specificity were calculated for the software versus specialist classification. Software agreement with optical craniometrics was assessed with intraclass correlation coefficients. RESULTS The cranial shape classification software had an accuracy of 93.3% (95% CI 86.8–98.8; p < 0.001), with a sensitivity of 92.0% and specificity of 94.3%. Intraclass correlation coefficients for measurements of the cephalic index and cranial vault asymmetry index compared to optical measurements were 0.95 (95% CI 0.84–0.98; p < 0.001) and 0.67 (95% CI 0.24–0.88; p = 0.003), respectively. CONCLUSIONS These results support the use of image processing–based neonatal cranial deformity classification software for remote screening of nonsyndromic craniosynostosis in a newborn population and as a substitute for optical scanner– or CT-based craniometrics. This work has implications that suggest the potential for the development of software for a mobile platform that would allow for screening by telemedicine or in a primary care setting.

Download Full-text

Using Smart Bracelets to Assess Heart Rate Among Students During Physical Education Lessons: Feasibility, Reliability, and Validity Study (Preprint)

10.2196/preprints.17699 ◽

2020 ◽

Author(s):

Jiangang Sun ◽

Yang Liu

Keyword(s):

Heart Rate ◽

Physical Education ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Left Wrist ◽

Intraclass Correlation Coefficients ◽

Estimate Reliability ◽

Left And Right ◽

The Right

BACKGROUND An increasing number of wrist-worn wearables are being examined in the context of health care. However, studies of their use during physical education (PE) lessons remain scarce. OBJECTIVE We aim to examine the reliability and validity of the Fizzo Smart Bracelet (Fizzo) in measuring heart rate (HR) in the laboratory and during PE lessons. METHODS In Study 1, 11 healthy subjects (median age 22.0 years, IQR 3.75 years) twice completed a test that involved running on a treadmill at 6 km/h for 12 minutes and 12 km/h for 5 minutes. During the test, participants wore two Fizzo devices, one each on their left and right wrists, to measure their HR. At the same time, the Polar Team2 Pro (Polar), which is worn on the chest, was used as the standard. In Study 2, we went to 10 schools and measured the HR of 24 students (median age 14.0 years, IQR 2.0 years) during PE lessons. During the PE lessons, each student wore a Polar device on their chest and a Fizzo on their right wrist to measure HR data. At the end of the PE lessons, the students and their teachers completed a questionnaire where they assessed the feasibility of Fizzo. The measurements taken by the left wrist Fizzo and the right wrist Fizzo were compared to estimate reliability, while the Fizzo measurements were compared to the Polar measurements to estimate validity. To measure reliability, intraclass correlation coefficients (ICC), mean difference (MD), standard error of measurement (SEM), and mean absolute percentage errors (MAPE) were used. To measure validity, ICC, limits of agreement (LOA), and MAPE were calculated and Bland-Altman plots were constructed. Percentage values were used to estimate the feasibility of Fizzo. RESULTS The Fizzo showed excellent reliability and validity in the laboratory and moderate validity in a PE lesson setting. In Study 1, reliability was excellent (ICC>0.97; MD<0.7; SEM<0.56; MAPE<1.45%). The validity as determined by comparing the left wrist Fizzo and right wrist Fizzo was excellent (ICC>0.98; MAPE<1.85%). Bland-Altman plots showed a strong correlation between left wrist Fizzo measurements (bias=0.48, LOA=–3.94 to 4.89 beats per minute) and right wrist Fizzo measurements (bias=0.56, LOA=–4.60 to 5.72 beats per minute). In Study 2, the validity of the Fizzo was lower compared to that found in Study 1 but still moderate (ICC>0.70; MAPE<9.0%). The Fizzo showed broader LOA in the Bland-Altman plots during the PE lessons (bias=–2.60, LOA=–38.89 to 33.69 beats per minute). Most participants considered the Fizzo very comfortable and easy to put on. All teachers thought the Fizzo was helpful. CONCLUSIONS When participants ran on a treadmill in the laboratory, both left and right wrist Fizzo measurements were accurate. The validity of the Fizzo was lower in PE lessons but still reached a moderate level. The Fizzo is feasible for use during PE lessons.

Download Full-text

Is the location of the signal intensity weighted centroid a reliable measurement of fluid displacement within the disc?

Biomedical Engineering / Biomedizinische Technik ◽

10.1515/bmt-2016-0178 ◽

2018 ◽

Vol 63 (4) ◽

pp. 453-460 ◽

Cited By ~ 7

Author(s):

Vahid Abdollah ◽

Eric C. Parent ◽

Michele C. Battié

Keyword(s):

Signal Intensity ◽

Water Distribution ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Region Of Interest ◽

Rater Reliability ◽

Fluid Displacement ◽

Intraclass Correlation Coefficients ◽

The Mean ◽

Standard Error Of Measurement

Abstract Degenerated discs have shorter T2-relaxation time and lower MR signal. The location of the signal-intensity-weighted-centroid reflects the water distribution within a region-of-interest (ROI). This study compared the reliability of the location of the signal-intensity-weighted-centroid to mean signal intensity and area measurements. L4-L5 and L5-S1 discs were measured on 43 mid-sagittal T2-weighted 3T MRI images in adults with back pain. One rater analysed images twice and another once, blinded to measurements. Discs were semi-automatically segmented into a whole disc, nucleus, anterior and posterior annulus. The coordinates of the signal-intensity-weighted-centroid for all regions demonstrated excellent intraclass-correlation-coefficients for intra- (0.99–1.00) and inter-rater reliability (0.97–1.00). The standard error of measurement for the Y-coordinates of the signal-intensity-weighted-centroid for all ROIs were 0 at both levels and 0 to 2.7 mm for X-coordinates. The mean signal intensity and area for the whole disc and nucleus presented excellent intra-rater reliability with intraclass-correlation-coefficients from 0.93 to 1.00, and 0.92 to 1.00 for inter-rater reliability. The mean signal intensity and area had lower reliability for annulus ROIs, with intra-rater intraclass-correlation-coefficient from 0.5 to 0.76 and inter-rater from 0.33 to 0.58. The location of the signal-intensity-weighted-centroid is a reliable biomarker for investigating the effects of disc interventions.

Download Full-text

The Reliability and Validity of the Loadsol® under Various Walking and Running Conditions

Sensors ◽

10.3390/s19020265 ◽

2019 ◽

Vol 19 (2) ◽

pp. 265 ◽

Cited By ~ 12

Author(s):

Kristen Renner ◽

DS Williams ◽

Robin Queen

Keyword(s):

Reliable Method ◽

Loading Rate ◽

Reaction Force ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Validity And Reliability ◽

Intraclass Correlation Coefficients ◽

Single Sensor ◽

Force Data

The assessment of loading during walking and running has historically been limited to data collection in laboratory settings or with devices that require a computer connection. This study aims to determine if the loadsol®—a single sensor wireless insole—is a valid and reliable method of assessing force. Thirty (17 male and 13 female) recreationally active individuals were recruited for a two visit study where they walked (1.3 m/s) and ran (3.0 and 3.5 m/s) at a 0%, 10% incline, and 10% decline, with the visits approximately one week apart. Ground reaction force data was collected on an instrumented treadmill (1440 Hz) and with the loadsol® (100 Hz). Ten individuals completed the day 1 protocol with a newer 200 Hz loadsol®. Intraclass correlation coefficients (ICC3,k) were used to assess validity and reliability and Bland–Altman plots were generated to better understand loadsol® validity. Across conditions, the peak force ICCs ranged from 0.78 to 0.97, which increased to 0.84–0.99 with the 200 Hz insoles. Similarly, the loading rate ICCs improved from 0.61 to 0.97 to 0.80–0.96 and impulse improved from 0.61 to 0.97 to 0.90–0.97. The 200 Hz insoles may be needed for loading rate and impulse in running. For both walking and running, the loadsol® has excellent between-day reliability (>0.76).

Download Full-text

Alberta Infant Motor Scale: Reliability and Validity When Used on Preterm Infants in Taiwan

Physical Therapy ◽

10.1093/ptj/80.2.168 ◽

2000 ◽

Vol 80 (2) ◽

pp. 168-178 ◽

Cited By ~ 76

Author(s):

Suh-Fang Jeng ◽

Kuo-Inn Tsou Yau ◽

Li-Chiou Chen ◽

Shu-Fang Hsiao

Keyword(s):

Preterm Infants ◽

Interrater Reliability ◽

Physical Therapist ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Intraclass Correlation Coefficients ◽

Scale Reliability ◽

Scale Scores ◽

Acceptable Reliability

Abstract Background and Purpose. The goal of this study was to examine the reliability and validity of measurements obtained with the Alberta Infant Motor Scale (AIMS) for evaluation of preterm infants in Taiwan. Subjects. Two independent groups of preterm infants were used to investigate the reliability (n=45) and validity (n=41) for the AIMS. Methods. In the reliability study, the AIMS was administered to the infants by a physical therapist, and infant performance was videotaped. The performance was then rescored by the same therapist and by 2 other therapists to examine the intrarater and interrater reliability. In the validity study, the AIMS and the Bayley Motor Scale were administered to the infants at 6 and 12 months of age to examine criterion-related validity. Results. Intraclass correlation coefficients (ICCs) for intrarater and interrater reliability of measurements obtained with the AIMS were high (ICC=.97–.99). The AIMS scores correlated with the Bayley Motor Scale scores at 6 and 12 months (r=.78 and .90), although the AIMS scores at 6 months were only moderately predictive of the motor function at 12 months (r=.56). Conclusion and Discussion. The results suggest that measurements obtained with the AIMS have acceptable reliability and concurrent validity but limited predictive value for evaluating preterm Taiwanese infants.

Download Full-text

Reliability and Validity of the Posture and Fine Motor Assessment of Infants

The Occupational Therapy Journal of Research ◽

10.1177/153944928900900501 ◽

1989 ◽

Vol 9 (5) ◽

pp. 259-272 ◽

Cited By ~ 3

Author(s):

Jane Case-Smith

Keyword(s):

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Fine Motor ◽

Motor Assessment ◽

Intraclass Correlation Coefficients ◽

Highly Correlated ◽

Peabody Developmental Motor Scales ◽

The Individual ◽

Test Retest Reliability

The Posture and Fine Motor Assessment of Infants (PFMAI) (Case-Smith, 1987) is a newly developed instrument for assessing the quality of motor function in infants. The test measures components of posture and fine motor control as they first develop. The purpose of this study was to support the test's reliability and validity. Interrater reliability, analyzed with intraclass correlation coefficients (ICCs), was high (.989 for total scores). Test-retest reliability, measured by ICCs, was .853 and .913 for the two test sections. The PFMAI demonstrated concurrent validity with the Peabody Developmental Motor Scales, Revised (Folio & Fewell, 1983) (correlations were .673 and .829 for the individual sections). Scores on the PFMAI were highly correlated with the infant's ages (.892 to .941); this finding provided one indication of construct validity.

Download Full-text

The Reliability and Validity of the School Function Assessment—Chinese Version

OTJR Occupation Participation and Health ◽

10.1177/153944920502500202 ◽

2005 ◽

Vol 25 (2) ◽

pp. 44-54 ◽

Cited By ~ 4

Author(s):

Jeng-Liang Hwang

Keyword(s):

Content Validity ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Chinese Version ◽

Intraclass Correlation Coefficients ◽

Validity Indices ◽

Function Assessment ◽

School Function ◽

School Function Assessment

The purpose of this study was to examine the reliability and validity of the School Function Assessment (SFA)-Chinese version. The data drawn from the standardization sample of the instrument (N = 320) were used in the analyses. Using internal consistency procedures, Cronbach's alpha for each scale ranged from .94 to .98. The test—retest study reported intraclass correlation coefficients from .87 to .98 among the scales. Content validity of the SFA—Chinese version was confirmed by the overall high content validity indices (85% to 100%) determined by educational experts. Construct-related validity based on an exploratory factor analysis revealed two correlated functional domains of the SFA—Chinese version, “cognitive/behavioral” and “physical,” which were consistent with the theoretical construct hypothesized in the instrument. The known groups method demonstrated different patterns of the SFA scores among three different diagnostic groups. Moreover, discriminant analysis of the scores showed a high percentage of children correctly classified into their diagnostic groups. The overall results supported the psychometric properties of the SFA—Chinese version.

Download Full-text

Reliability and Validity of a Wireless Accelerometer for the Assessment of Postural Sway

Journal of Applied Biomechanics ◽

10.1123/jab.2014-0232 ◽

2015 ◽

Vol 31 (3) ◽

pp. 159-163 ◽

Cited By ~ 5

Author(s):

Nathan W. Saunders ◽

Panagiotis Koutakis ◽

Anne D. Kloos ◽

Deborah A. Kegelmeyer ◽

Jessica D. Dicke ◽

...

Keyword(s):

Postural Sway ◽

Center Of Mass ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Discrimination Capability ◽

Intraclass Correlation Coefficients ◽

Balance Task ◽

Eyes Open ◽

Test Retest Reliability

Clinicians are in need of valid and objective measures of postural sway. Accelerometers have been shown to be suitable alternatives to expensive and stationary force plates. We evaluated the test-retest reliability and balance task discrimination capability of a new wireless triaxial accelerometer (YEI 3-Space Sensor). Four testing conditions (eyes open or closed, while on a firm or compliant surface) were used to progressively challenge the static balance of 20 healthy male (n = 8) and female (n = 12) older adults (mean age 81 ± 4.3 y). Subjects completed 2 blocks of three 30-second trials per condition. The accelerometer was positioned on the lower back to acquire mediolateral (M-L) and anterior-posterior (A-P) accelerations. Intraclass correlation coefficients were all good to excellent, with values ranging from .736 to .972 for trial-to-trial and from .760 to .954 for block-to-block. A significant stepwise increase in center of mass acceleration root mean square values was found across the 4 balance conditions (F[1.49, 28.26] = 39.54, P < .001). The new accelerometer exhibited good to excellent trial-to-trial and block-to-block reliability and was sensitive to differences in visual and surface conditions and acceleration axes.

Download Full-text

The Brazilian version of the Neuropsychiatric Inventory-Clinician rating scale (NPI-C): reliability and validity in dementia

International Psychogeriatrics ◽

10.1017/s1041610213000811 ◽

2013 ◽

Vol 25 (9) ◽

pp. 1503-1511 ◽

Cited By ~ 16

Author(s):

Florindo Stella ◽

Orestes Vicente Forlenza ◽

Jerson Laks ◽

Larissa Pires de Andrade ◽

Michelle A. Ljubetic Avendaño ◽

...

Keyword(s):

Concurrent Validity ◽

Convergent Validity ◽

Rating Scale ◽

Neuropsychiatric Symptoms ◽

Intraclass Correlation ◽

Reliability And Validity ◽

Correlation Coefficients ◽

Neuropsychiatric Inventory ◽

Rater Reliability ◽

Clinician Rating

ABSTRACTBackground:Patients with dementia may be unable to describe their symptoms, and caregivers frequently suffer emotional burden that can interfere with judgment of the patient's behavior. The Neuropsychiatric Inventory-Clinician rating scale (NPI-C) was therefore developed as a comprehensive and versatile instrument to assess and accurately measure neuropsychiatric symptoms (NPS) in dementia, thereby using information from caregiver and patient interviews, and any other relevant available data. The present study is a follow-up to the original, cross-national NPI-C validation, evaluating the reliability and concurrent validity of the NPI-C in quantifying psychopathological symptoms in dementia in a large Brazilian cohort.Methods:Two blinded raters evaluated 312 participants (156 patient-knowledgeable informant dyads) using the NPI-C for a total of 624 observations in five Brazilian centers. Inter-rater reliability was determined through intraclass correlation coefficients for the NPI-C domains and the traditional NPI. Convergent validity included correlations of specific domains of the NPI-C with the Brief Psychiatric Rating Scale (BPRS), the Cohen-Mansfield Agitation Index (CMAI), the Cornell Scale for Depression in Dementia (CSDD), and the Apathy Inventory (AI).Results:Inter-rater reliability was strong for all NPI-C domains. There were high correlations between NPI-C/delusions and BPRS, NPI-C/apathy-indifference with the AI, NPI-C/depression-dysphoria with the CSDD, NPI-C/agitation with the CMAI, and NPI-C/aggression with the CMAI. There was moderate correlation between the NPI-C/aberrant vocalizations and CMAI and the NPI-C/hallucinations with the BPRS.Conclusion:The NPI-C is a comprehensive tool that provides accurate measurement of NPS in dementia with high concurrent validity and inter-rater reliability in the Brazilian setting. In addition to universal assessment, the NPI-C can be completed by individual domains.

Download Full-text