A Pilot Study to Develop an Assessment Tool for Dogs Undergoing Trap-Neuter-Release (TNR) in Italy. An Overview on the National Implementation of TNR Programmes

A descriptive analysis, inter-observer and test–retest reliability of the animal-based measures (ABMs) included in the protocol were performed. This study aimed at the development of a welfare assessment protocol for dogs recruited in the trap-neuter-release (TNR) programmes and the description of the implantation of these programmes in Italy. Nine Italian regions carried out TNR programmes. A varied scenario, along with some critical issues, emerged. Fifty dogs were recruited and assessed simultaneously by two assessors to determine the reliability of ABMs included in the protocol. A subsample of ten dogs were assessed three times to assess test–retest reliability. All females were neutered against 36% of males. Most dogs were adults (58%) and of a large size (68%). Vaccine prophylaxis and parasitic prevention were regular in 13% and 76% of dogs, respectively. Few dogs showed lameness, evidence of pain, other clinical problems, or thermal discomfort. Overall, 82% of dogs did not show fear or aggression to unfamiliar people. The level of agreement between the two assessors was quite high, ranging from substantial (0.61–0.80) to perfect (1) for the majority of measures. This study highlighted some critical issues in TNR implementation and the suitability of the protocol as a tool for animal welfare assessment.

Download Full-text

Test−Retest Reliability of the ‘Welfare Quality® Animal Welfare Assessment Protocol for Sows and Piglets’. Part 1. Assessment of the Welfare Principle of ‘Appropriate Behavior’

Animals ◽

10.3390/ani9070398 ◽

2019 ◽

Vol 9 (7) ◽

pp. 398 ◽

Cited By ~ 3

Author(s):

Friedrich ◽

Krieter ◽

Kemper ◽

Czycholl

Keyword(s):

Animal Welfare ◽

Correlation Coefficient ◽

Retest Reliability ◽

Welfare Assessment ◽

Appropriate Behavior ◽

Assessment Protocol ◽

Smallest Detectable Change ◽

Welfare Quality ◽

Animal Welfare Assessment ◽

Test Retest Reliability

The present study’s aim was to assess the test−retest reliability (TRR) of the ‘Welfare Quality® animal welfare assessment protocol for sows and piglets’ focusing on the welfare principle ‘appropriate behavior’. TRR was calculated using Spearman’s rank correlation coefficient (RS), intraclass correlation coefficient (ICC), smallest detectable change (SDC), and limits of agreement (LoA). Principal component analysis (PCA) was used for deeper analysis of the Qualitative Behavior Assessment (QBA). The study was conducted on thirteen farms in Northern Germany, which were visited five times by the same observer. Farm visits 1 (F1; day 0) were compared to farm visits 2 to 5 (F2–F5). The QBA indicated no TRR when applying the statistical parameters introduced above (e.g., ‘playful‘ (F1–F4) RS 0.08 ICC 0.00 SDC 0.50 LoA [−0.62, 0.38]). The PCA detected contradictory TRR. Acceptable TRR could be found for parts of the instantaneous scan sampling (e.g., negative social behavior (F1–F3) RS 0.45 ICC 0.37 SDC 0.02 LoA [−0.03, 0.02]). The human−animal relationship test solely achieved poor TRR, whereas scans for stereotypies showed sufficient TRR (e.g., floor licking (F1–F4) RS 0.63 ICC 0.52 SDC 0.05 LoA [−0.08, 0.04]). Concluding, the principle ‘appropriate behavior’ does not represent TRR and further investigation is needed before implementation on-farm.

Download Full-text

Test-retest reliability of the Welfare Quality® animal welfare assessment protocol for growing pigs

Animal Welfare ◽

10.7120/09627286.25.4.447 ◽

2016 ◽

Vol 25 (4) ◽

pp. 447-459 ◽

Cited By ~ 17

Author(s):

I Czycholl ◽

C Kniese ◽

K Büttner ◽

E Grosse Beilage ◽

L Schrader ◽

...

Keyword(s):

Animal Welfare ◽

Growing Pigs ◽

Retest Reliability ◽

Welfare Assessment ◽

Assessment Protocol ◽

Welfare Quality ◽

Animal Welfare Assessment ◽

Test Retest Reliability

Download Full-text

Measuring “Language Access Profiles” in Deaf and Hard-of-Hearing Children With the DHH Language Exposure Assessment Tool

Journal of Speech Language and Hearing Research ◽

10.1044/2020_jslhr-20-00439 ◽

2020 ◽

pp. 1-25

Author(s):

Matthew L. Hall ◽

Stephanie De Anda

Keyword(s):

Cluster Analysis ◽

Exposure Assessment ◽

Hard Of Hearing ◽

Assessment Tool ◽

Communication Mode ◽

Developmental Potential ◽

Language Access ◽

Retest Reliability ◽

Language Exposure ◽

Test Retest Reliability

Purpose The purposes of this study were (a) to introduce “language access profiles” as a viable alternative construct to “communication mode” for describing experience with language input during early childhood for deaf and hard-of-hearing (DHH) children; (b) to describe the development of a new tool for measuring DHH children's language access profiles during infancy and toddlerhood; and (c) to evaluate the novelty, reliability, and validity of this tool. Method We adapted an existing retrospective parent report measure of early language experience (the Language Exposure Assessment Tool) to make it suitable for use with DHH populations. We administered the adapted instrument (DHH Language Exposure Assessment Tool [D-LEAT]) to the caregivers of 105 DHH children aged 12 years and younger. To measure convergent validity, we also administered another novel instrument: the Language Access Profile Tool. To measure test–retest reliability, half of the participants were interviewed again after 1 month. We identified groups of children with similar language access profiles by using hierarchical cluster analysis. Results The D-LEAT revealed DHH children's diverse experiences with access to language during infancy and toddlerhood. Cluster analysis groupings were markedly different from those derived from more traditional grouping rules (e.g., communication modes). Test–retest reliability was good, especially for the same-interviewer condition. Content, convergent, and face validity were strong. Conclusions To optimize DHH children's developmental potential, stakeholders who work at the individual and population levels would benefit from replacing communication mode with language access profiles. The D-LEAT is the first tool that aims to measure this novel construct. Despite limitations that future work aims to address, the present results demonstrate that the D-LEAT represents progress over the status quo.

Download Full-text

The Development and Psychometric Validation of the Diabetes Impact and Device Satisfaction Scale for Individuals with Type 1 Diabetes

Journal of Diabetes Science and Technology ◽

10.1177/1932296819897976 ◽

2020 ◽

Vol 14 (2) ◽

pp. 309-317

Author(s):

Michelle L. Manning ◽

Harsimran Singh ◽

Keaton Stoner ◽

Steph Habif

Keyword(s):

Type 1 Diabetes ◽

Assessment Tool ◽

Rapid Development ◽

Insulin Delivery ◽

Psychometric Validation ◽

Specific Information ◽

Retest Reliability ◽

Test Retest Reliability ◽

Post Assessment

Background: With the rapid development of new insulin delivery technology, measuring patient experience has become especially pertinent. The current study reports on item development, psychometric validation, and intended use of the newly developed Diabetes Impact and Device Satisfaction (DIDS) Scale. Method: The DIDS Scale was informed by a comprehensive literature review, and field tested as part of two focus groups. The finalized measure was used at baseline and 6 months post-assessment with a large US cohort. Exploratory factor analyses (EFAs) were conducted to determine and confirm factor structure and item selection. Internal reliability, test–retest reliability, and convergent/divergent validity of the emerged factors were tested with demographics, diabetes-specific information, and diabetes behavioral and satisfaction measures. Results: In all, 778 participants with type 1 diabetes (66% female, mean age 47.13 ± 17.76 years, 74% insulin pump users) completed surveys at both baseline and post-assessment. EFA highlighted two factors—Device Satisfaction (seven items, Cronbach’s α = 0.85-0.90) and Diabetes Impact (four items, Cronbach’s α = 0.71-0.75). DIDS Scale demonstrated good concurrent validity and test–retest reliability. Conclusion: The DIDS Scale is a novel and a brief assessment tool with robust psychometric properties. It is recommended for use across all insulin delivery devices and is considered appropriate for use in longitudinal studies. Future studies are recommended to evaluate the performance of DIDS Scale in diverse populations with diabetes.

Download Full-text

A Novel Tool for the Assessment of Sport Climbers’ Movement Performance

International Journal of Sports Physiology and Performance ◽

10.1123/ijspp.2019-0311 ◽

2020 ◽

Vol 15 (6) ◽

pp. 795-800 ◽

Cited By ~ 1

Author(s):

Nicola Taylor ◽

David Giles ◽

Micha Panáčková ◽

James Mitchell ◽

Joel Chidley ◽

...

Keyword(s):

Performance Assessment ◽

Assessment Tool ◽

Intraclass Correlation ◽

Validity And Reliability ◽

Consultation Process ◽

Retest Reliability ◽

Performance Factors ◽

Sport Climbing ◽

Movement Performance ◽

Test Retest Reliability

Purpose: To assess the validity and reliability of a novel movement-performance assessment tool for climbing/sport climbing. Methods: First, salient climbing movement-performance factors were identified through an iterative consultation process with 10 expert climbing coaches; the resulting Climber’s Movement Performance Assessment Tool (CM-PAT) contained 14 items in 5 categories. Second, 61 intermediate to advanced climbers ascended a single route, which was video recorded. Subsequently, 4 experienced (>10 y coaching) coaches used the CM-PAT to observe and score the climbers’ performance. Interrater reliability and comparisons with existing measures of climbing performance (6-mo self-reported ability, success and failure, climbing pace [m·min−1], and geometric entropy) were made. Results: Intraclass correlation coefficient (2,k) for the 4 raters demonstrated excellent reliability (>.81) between observers and good to excellent test–retest reliability (.71–.91). Pearson correlations between self-reported ability and CM-PAT scores explained 61% of the variance in self-reported climbing performance compared with 16% for geometric entropy and 52% for climbing pace. Considering differences in successful and unsuccessful climbers, the CM-PAT (P < .0005; d = 2.14), geometric entropy (P = .014; d = 0.67), and pace (P < .0005; d = 1.88) were able to differentiate between groups. Conclusions: The CM-PAT is the first sport climbing performance observational instrument to be developed through a thorough iterative process with expert coaches. Excellent interrater and test–retest reliability and excellent agreement with self-reported ability and with existing quantitative measures of performance support its recommendation for use in coaching and research contexts. Notably, a key advantage over existing measures is the identification of coachable elements of performance.

Download Full-text

Validation of a Hidradenitis Suppurativa Self-Assessment Tool

Journal of Cutaneous Medicine and Surgery ◽

10.1177/1203475419839965 ◽

2019 ◽

Vol 23 (4) ◽

pp. 388-390 ◽

Cited By ~ 4

Author(s):

Aditi Senthilnathan ◽

Sree S. Kolli ◽

Leah A. Cardwell ◽

Irma Richardson ◽

Steven R. Feldman ◽

...

Keyword(s):

Hidradenitis Suppurativa ◽

Assessment Tool ◽

Multiple Scales ◽

Weighted Kappa ◽

Self Administration ◽

Self Assessment ◽

Good Test ◽

Retest Reliability ◽

Dermatology Clinic ◽

Test Retest Reliability

Background: Hidradenitis suppurativa (HS) is a debilitating dermatologic condition presenting with recurrent abscesses. While there are multiple scales to determine HS severity, none are designed for self-administration. A validated severity self-assessment tool may facilitate survey research and improve communication by allowing patients to objectively report their HS severity between clinic visits. Objectives: The purpose of this study was to assess a self-administered HS measure. Methods: An HS self-assessment tool (HSSA) with 10 photographs of different Hurley stages was developed. The tool was administered to patients diagnosed with HS who visited the Wake Forest Baptist Health dermatology clinic over a span of 2 months. Physician-administered Hurley stage was recorded to determine criterion validity. To assess test-retest reliability of the measure, patients completed the HSSA again at least 30 minutes after the first completion. Results: Twenty-four patients completed the measure, and 20 of these patients completed it twice. Agreement between physician-determined Hurley stage and self-determined Hurley stage was 66.7% with a weighted kappa of 0.57 (95% confidence interval [CI]: 0.30-0.84). The weighted kappa for agreement between patients’ initial and second completion of the HSSA was 0.81 (95% CI: 0.64-0.99). Conclusions: The self-administered measure provides moderate agreement with physician-determined Hurley stage and good test-retest reliability.

Download Full-text

Psychometric evaluation of the Chinese version of the Person-Centred Care Assessment Tool

BMJ Open ◽

10.1136/bmjopen-2019-031580 ◽

2020 ◽

Vol 10 (7) ◽

pp. e031580

Author(s):

Cai Le ◽

Ke Ma ◽

Pingfen Tang ◽

David Edvardsson ◽

Lina Behm ◽

...

Keyword(s):

Assessment Tool ◽

Intraclass Correlation ◽

Psychometric Evaluation ◽

Quality Of Healthcare ◽

Hospital Staff ◽

Chinese Version ◽

Validity And Reliability ◽

Retest Reliability ◽

Care Assessment ◽

Test Retest Reliability

ObjectiveThis study aims to test a Chinese cross-cultural adaptation of the English version of the Person-Centred Care Assessment Tool (P-CAT) and evaluate its psychometric properties.DesignP-CAT was translated/back-translated using established procedures before the psychometric evaluation of the Chinese version was made.SettingTwo hospitals covering urban and suburban areas of Kunming in the Yunnan province of China.Participants152 female hospital staff completed the survey.Main outcome measure(s)Construct validity and reliability, including internal consistency and test–retest reliability, were assessed among a sample of hospital staff.ResultsThe factor analysis resulted in a two-component solution that consisted of two subscales. The corrected item-total correlations for all of the items ranged from 0.14 to 0.44, with six items not meeting the cut-off level for item-total correlation (>0.3). The Chinese P-CAT demonstrated strong reliability, with a Cronbach’s alpha of 0.91–0.94 for the scales and a test–retest reliability coefficient of 0.88 for the overall scale scores. The intraclass correlation was 0.92 (95% CI 0.90 to 0.95).ConclusionP-CAT appears to be a promising measure for evaluating staff perceptions of person-centredness in Chinese hospital environments. The results show that P-CAT can be a useful tool for improving the quality of healthcare in terms of person-centred care in the Chinese context.

Download Full-text

Reliability testing of a portfolio assessment tool for postgraduate family medicine training in South Africa

African Journal of Primary Health Care & Family Medicine ◽

10.4102/phcfm.v5i1.577 ◽

2013 ◽

Vol 5 (1) ◽

Cited By ~ 1

Author(s):

Louis Jenkins ◽

Bob Mash ◽

Anselme Derese

Keyword(s):

South Africa ◽

Family Medicine ◽

Assessment Tool ◽

Portfolio Assessment ◽

Validity And Reliability ◽

Rater Reliability ◽

Retest Reliability ◽

Competency Based ◽

The Difference ◽

Test Retest Reliability

Background: Competency-based education and the validity and reliability of workplace-based assessment of postgraduate trainees have received increasing attention worldwide.Family medicine was recognised as a speciality in South Africa six years ago and a satisfactory portfolio of learning is a prerequisite to sit the national exit exam. A massive scaling up of the number of family physicians is needed in order to meet the health needs of the country.Aim: The aim of this study was to develop a reliable, robust and feasible portfolio assessment tool (PAT) for South Africa.Methods: Six raters each rated nine portfolios from the Stellenbosch University programme, using the PAT, to test for inter-rater reliability. This rating was repeated three months later to determine test–retest reliability. Following initial analysis and feedback the PAT was modified and the inter-rater reliability again assessed on nine new portfolios. An acceptable intra-classcorrelation was considered to be > 0.80.Results: The total score was found to be reliable, with a coefficient of 0.92. For test–retest reliability, the difference in mean total score was 1.7%, which was not statistically significant. Amongst the subsections, only assessment of the educational meetings and the logbook showed reliability coefficients > 0.80.Conclusion: This was the first attempt to develop a reliable, robust and feasible national portfolio assessment tool to assess postgraduate family medicine training in the South African context. The tool was reliable for the total score, but the low reliability of several sections in the PAT helped us to develop 12 recommendations regarding the use of the portfolio, the design of the PAT and the training of raters.

Download Full-text

Development, reliability and validity of the Safe Use of Mobility Aids Checklist (SUMAC) for 4-wheeled walker use in people living with dementia

10.21203/rs.2.22127/v2 ◽

2020 ◽

Author(s):

Susan Hunter ◽

Alison Divine ◽

Humberto Omana ◽

Edward Madou ◽

Jeffrey Holmes

Keyword(s):

Focus Groups ◽

Physical Function ◽

Assessment Tool ◽

Rank Order ◽

Criterion Validity ◽

Minimal Detectable Change ◽

Retest Reliability ◽

People With Dementia ◽

Mobility Aid ◽

Test Retest Reliability

Abstract Background: Balance and gait problems are common and progressive in dementia. Mobility aid use in people with dementia increases the likelihood of falls three-fold. An assessment tool to quantify mobility aid safety in people with dementia does not currently exist. The objectives of this study were: 1) to develop a tool for the evaluation of physical function and safety with use of a 4-wheeled walker in people with dementia, and 2) to evaluate its construct and criterion validity, inter-rater and test-retest reliability and minimal detectable change.Methods: Healthcare professionals (HCP) experienced in care and rehabilitation of people with dementia participated in focus groups for item generation of the new tool, entitled The Safe Use of Mobility Aid Checklist (SUMAC). The SUMAC evaluates physical function (PF) and safe use of the equipment (EQ) on nine tasks of daily life. Ten individuals with dementia using a 4-wheeled walker were recorded performing the tasks within the SUMAC. Reliability was evaluated by having five HCP score participant videos on two occasions. Inter-rater and test-retest reliability was assessed using intra-class correlation coefficients (ICC). Construct validity was evaluated using scores determined by a consensus panel compared to the individual HCP scores using Spearman’s rank-order correlations. Criterion validity was assessed between SUMAC-PF and the Performance-Oriented Mobility Assessment (POMA) gait subscale using Spearman’s rank-order correlations. Results: Three focus groups (n=17) were completed and these generated a tool comprised of nine tasks and the components within each task that the clinicians would observe for physical function and safe use. Inter-rater reliability was statistically significant for the SUMAC-PF (ICC=0.92, 95%CI (0.81, 0.98), p<0.001) and SUMAC-EQ (ICC=0.82, 95%CI (0.54, 0.95), p<0.001). Test-retest reliability was also statistically significant for the SUMAC-PF (ICC=0.89, 95%CI (0.81, 0.94), p<0.001) and SUMAC-EQ (ICC=0.88, 95%CI (0.79, 0.93), p<0.001). The POMA gait subscale correlated strongly with the SUMAC-PF (r=0.84), but not EQ (r=0.39).Conclusions: Each of the scores of the SUMAC, physical function and safe use of the equipment, demonstrated content validity, strong inter-rater and test-retest reliability and strong criterion and concurrent validity for the assessment of mobility aid safety in people with dementia.

Download Full-text