STREET: Swedish Tool for Risk/Resource Estimation at EvenTs. Part two, resource assessment – face validity and inter–rater reliability

ObjectiveTo describe the clinimetric validation of the I-DECIDED tool for peripheral intravenous catheter assessment and decision-making.Design and settingI-DECIDED is an eight-step tool derived from international vascular access guidelines into a structured mnemonic for device assessment and decision-making. The clinimetric evaluation process was conducted in three distinct phases.MethodsInitial face validity was confirmed with a vascular access working group. Next, content validity testing was conducted via online survey with vascular access experts and clinicians from Australia, the UK, the USA and Canada. Finally, inter-rater reliability was conducted between 34 pairs of assessors for a total of 68 peripheral intravenous catheter (PIVC) assessments. Assessments were timed to ensure feasibility, and the second rater was blinded to the first’s findings. Content validity index (CVI), mean item-level CVI (I-CVI), internal consistency, mean proportion of agreement, observed and expected inter-rater agreements, and prevalence-adjusted bias-adjusted kappas (PABAK) were calculated. Ethics approvals were obtained from university and hospital ethics committees.ResultsThe I-DECIDED tool demonstrated strong content validity among international vascular access experts (n=7; mean I-CVI=0.91; mean proportion of agreement=0.91) and clinicians (n=11; mean I-CVI=0.93; mean proportion of agreement=0.94), and high inter-rater reliability in seven adult medical-surgical wards of three Australian hospitals. Overall, inter-rater reliability was 87.13%, with PABAK for each principle ranging from 0.5882 (‘patient education’) to 1.0000 (‘document the decision’). Time to complete assessments averaged 2 min, and nurse-reported acceptability was high.ConclusionThis is the first comprehensive, evidence-based, valid and reliable PIVC assessment and decision tool. We recommend studies to evaluate the outcome of implementing this tool in clinical practice.Trial registration number12617000067370

Download Full-text

The Adult Attachment Projective Picture System: a pilot study of inter-rater reliability and face validity with adults with intellectual disabilities

Advances in Mental Health and Intellectual Disabilities ◽

10.1108/amhid-11-2017-0036 ◽

2018 ◽

Vol 12 (2) ◽

pp. 57-66 ◽

Cited By ~ 1

Author(s):

Deanna Gallichan ◽

Carol George

Keyword(s):

Intellectual Disabilities ◽

Adult Attachment ◽

Clinical Utility ◽

Reliability And Validity ◽

Clinical History ◽

Face Validity ◽

Rater Reliability ◽

Content Type ◽

Adult Attachment Projective ◽

Adults With Intellectual Disabilities

Purpose The purpose of this paper is to assess whether the Adult Attachment Projective (AAP) Picture System is a reliable and face valid measure of internal working models of attachment in adults with intellectual disabilities (ID). Design/methodology/approach The AAPs of 20 adults with ID were coded blind by two reliable judges and classified into one of four groups: secure, dismissing, preoccupied, or unresolved. Inter-rater reliability was calculated using κ. Six participants repeated the assessment for test-retest reliability. Two independent experts rated ten cases on the links between the AAP analysis and the clinical history. Findings There was significant agreement between AAP judges, κ=0.677, p<0.001. Five out of six participants showed stability in their classifications over time. The majority of expert ratings were “good” or “excellent”. There was a significant inter-class correlation between raters suggesting good agreement between them r=0.51 (p<0.05). The raters’ feedback suggested that the AAP had good clinical utility. Research limitations/implications The inter-rater reliability, stability, face validity, and clinical utility of the AAP in this population is promising. Further examination of these findings with a larger sample of individuals with ID is needed. Originality/value This is the first study attempting to investigate the reliability and validity of the AAP in this population.

Download Full-text

Assessing the Validity and Reliability of the Persian Version of the Interpersonal Problem Solving Skills Assessment Tool in Schizophrenia

Journal of Rehabilitation ◽

10.32598/rj.22.1.3214.1 ◽

2021 ◽

Vol 22 (1) ◽

pp. 118-131

Author(s):

Milad Abolhasani ◽

◽

Ashraf Karbalaee Nouri ◽

Enayatollah Bakhshi ◽

Milad Abolhasani ◽

...

Keyword(s):

Content Validity ◽

Face Validity ◽

Validity And Reliability ◽

Alpha Coefficient ◽

Rater Reliability ◽

Problem Solving Skills ◽

Retest Reliability ◽

Persian Version ◽

Interpersonal Problem Solving ◽

Test Retest Reliability

Objective: This study aimed to translate the Assessment of Interpersonal Problem-Solving Skills (AIPSS) into Persian and to evaluate the validity and reliability of the Persian version of AIPSS to use for adults with schizophrenia. Materials & Methods: In this methodological study, the translation process was performed according to the International Quality of Life Assessment (IQOLA) protocol. The face validity of the translated AIPSS was determined based on the opinions of experts and The Content Validity Index (CVI) and Content Validity Ratio (CVR) were also calculated for each item. The Persian version of the test was performed on 52 patients with schizophrenia disorders at Tehran’s Razi Mental Hospital; they were selected using a convenience sampling method. Cronbach’s alpha coefficient was used to evaluate internal consistency. Inter-rater reliability was determined by the Intraclass Correlation Coefficient (ICC). A retest was complete on 15 patients with 2 weeks interval and ICC was used to determine the test-retest reliability. Results: Face validity was confirmed by the experts’ opinions. The Content Validity Ratio (CVR) and the Content Validity Index (CVI) were equal to one for all scenes. Cronbach's alpha coefficient for all scales was ranged between 0.511 and 0.821. The ICC in all scales were more than 0.98 for inter-rater reliability. In calculating test-retest reliability, the ICC for all scales ranged 0.733-0.893. Conclusion: Results show that the Persian version of AIPSS has acceptable face validity, content validity, internal consistency, inter-rater reliability test-retest reliability. Therefore, this instrument can be used in clinical fields and research studies to assess the social skills of Iranian patients with schizophrenia.

Download Full-text

Comparison of three probability distributions and techno-economic analysis of wind energy production along the coastal belt of Pakistan

Energy Exploration & Exploitation ◽

10.1177/0144598720931587 ◽

2020 ◽

pp. 014459872093158 ◽

Cited By ~ 1

Author(s):

Muhammad Sumair ◽

Tauseef Aized ◽

Syed Asad Raza Gardezi ◽

Muhammad Mahmood Aslam Bhutta ◽

Syed Muhammad Sohail Rehman ◽

...

Keyword(s):

Wind Speed ◽

Energy Production ◽

Probability Distributions ◽

Resource Assessment ◽

Wind Data ◽

Coefficient Of Determination ◽

Resource Estimation ◽

Wind Resource Assessment ◽

Coastal Belt ◽

Wind Resource

Continuous probability distributions have long been used to model the wind data. No single distribution can be declared accurate for all locations. Therefore, a comparison of different distributions before actual wind resource assessment should be carried out. Current work focuses on the application of three probability distributions, i.e. Weibull, Rayleigh, and lognormal for wind resource estimation at six sites along the coastal belt of Pakistan. Four years’ (2015–2018) wind data measured each 60-minutes at 50 m height for six locations were collected from Pakistan Meteorological Department. Comparison of these distributions was done based on coefficient of determination ( R2), root mean square error, and mean absolute percentage deviation. Comparison showed that Weibull distribution is the most accurate followed by lognormal and Rayleigh, respectively. Wind power density ( PD) was evaluated and it was found that Karachi has the highest wind speed and PD as 5.82 m/s and 162.69 W/m2, respectively, while Jiwani has the lowest wind speed and PD as 4.62 m/s and 76.76 W/m2, respectively. Furthermore, feasibility of annual energy production (AEP) was determined using six turbines. It was found that Vestas V42 shows the worst performance while Bonus 1300/62 is the best with respect to annual energy production and Bonus 600/44 is the most economical. Finally, sensitivity analysis was carried out.

Download Full-text

Inter-rater reliability and face validity of the Executive Function Performance Test (EFPT)

British Journal of Occupational Therapy ◽

10.1177/0308022615575744 ◽

2015 ◽

Vol 78 (9) ◽

pp. 563-569

Author(s):

Marie Cederfeldt ◽

Gunnel Carlsson ◽

Synneve Dahlin–Ivanoff ◽

Gunilla Gosman–Hedstrom

Keyword(s):

Executive Function ◽

Performance Test ◽

Face Validity ◽

Rater Reliability

Download Full-text

Construction and Validation of a Video Coding Tool for an Intervention to Improve Parental Feeding

Current Developments in Nutrition ◽

10.1093/cdn/nzaa059_063 ◽

2020 ◽

Vol 4 (Supplement_2) ◽

pp. 1346-1346

Author(s):

Margaret Samson ◽

Sarah Amin ◽

Karen McCurdy ◽

Amy Moore ◽

Alison Tovar

Keyword(s):

Video Coding ◽

Nutrition Education ◽

Online Survey ◽

Feeding Practices ◽

Family Meals ◽

Test Reliability ◽

Face Validity ◽

Child Feeding ◽

Rater Reliability ◽

Blood Institute

Abstract Objectives Home-based interventions have used video-recorded meals to assess feeding practices, yet no studies have used videos to directly provide feedback on the practices observed. This study describes the development and initial validation of a video coding tool to assess feeding practices observed in video-recorded family meals in order to provide feedback to caregivers. Methods The tool with operational definitions was developed based on the previous literature and other tools that capture caregiver feeding practices. To assess face validity, a sample of child feeding experts (n = 6) reviewed the tool and completed an 8-item online survey. Usability and content were assessed on a scale of 0–100, with 100 representing high usability and importance, respectively. The tool was modified based on expert feedback and used by trained research assistants to code 10 video-recorded family meals. Inter-rater reliability (IRR) was calculated and 3 videos were then randomly selected and coded by each research assistant at two different time points to assess test re-test reliability. Results Expert ratings of tool usability (81.83 ± 11.67) and content (87.67 ± 13.98) included feedback regarding the need to expand the operational definitions to better code practices, in particular with regards to pressure and encouragement. They also suggested merging practices, such as nutrition education and reasoning, which may be difficult to discern in an observational setting. Average IRR was 86.4% with pressure (75.1%) and encouragement (66.9%) having the lowest rates of agreement between coders. Limited choices (100%) and restriction (95.9%) had the highest rates of agreement. For test re-test reliability, the average agreement between the two timepoints was 80.0%. Conclusions Following the feedback from experts, the face validity of the developed tool improved, and the inter-rater reliability and test re-test reliability of the tool was acceptable. Future studies should focus on the expansion of operational definitions and training efforts to further improve inter-rater and test re-test reliabilities. Funding Sources This work was supported by the National Institutes on Health National Heart, Lung and Blood Institute [Tovar/R34HL 140,229].

Download Full-text