scholarly journals Introduction, development, and evaluation of the miniclinical evaluation exercise in postgraduate education of chiropractors

2015 ◽  
Vol 29 (1) ◽  
pp. 22-28 ◽  
Author(s):  
Inga Paravicini ◽  
Cynthia K. Peterson
Keyword(s):  
Correlation Coefficient ◽  
Clinical Judgment ◽  
Rating Scales ◽  
Rating Scale ◽  
Clinical Performance ◽  
Intraclass Correlation ◽  
Postgraduate Education ◽  
Clinical Encounter ◽  
Formative Feedback ◽  
Evaluation Exercise

Objective To determine if the clinical evaluation exercise (CEX) format is reliable, applicable and useful for evaluating clinical competency in the postgraduate chiropractic program as formative feedback. Methods Twelve mini-CEX clinical encounters were evaluated by 2 assessors per clinical encounter (7 assessors per session) in 23 chiropractic residents over a 12-month period. Two different rating scales (9 point and 5 point) were used, and the 2 assessors completed the forms independently. Individual competencies assessed consisted of history taking, physical examination, organization/efficiency, clinical judgment, professionalism/communication, counseling, and overall clinical performance. Interassessor reliability was calculated using κ and intraclass correlation coefficient statistics. Cronbach α assessed internal consistency of the mini-CEX. Spearman correlation coefficient evaluated correlation between the various competencies. The Mann-Whitney U test evaluated differences between the assessors' median numerical scores. Results The κ value for the 9-point rating scale was 0.31 (fair) and for the 5-point scale was 0.42 (moderate) with statistically significant intraclass correlation values (p < .05) for 4 of the 6 competencies. High correlation coefficients (p = .0001) were found when comparing the various competencies at each clinical encounter. There were no significant differences between the 2 assessors per clinical encounter for the scores awarded to the residents. Conclusions The mini-CEX is a reliable and useful tool to provide valuable formative feedback to postgraduate chiropractic residents. The 5-point grading scale was more user-friendly with better reliability.

scholarly journals Using Differential Item Functioning to Test for Interrater Reliability in Constructed Response Items

2020 ◽  
Vol 80 (4) ◽  
pp. 808-820
Author(s):  
Cindy M. Walker ◽  
Sakine Göçer Şahin
Keyword(s):  
Differential Item Functioning ◽  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
Kappa Statistic ◽  
Promising Alternative ◽  
Constructed Response ◽  
Polytomous Item ◽  
Item Functioning

The purpose of this study was to investigate a new way of evaluating interrater reliability that can allow one to determine if two raters differ with respect to their rating on a polytomous rating scale or constructed response item. Specifically, differential item functioning (DIF) analyses were used to assess interrater reliability and compared with traditional interrater reliability measures. Three different procedures that can be used as measures of interrater reliability were compared: (1) intraclass correlation coefficient (ICC), (2) Cohen’s kappa statistic, and (3) DIF statistic obtained from Poly-SIBTEST. The results of this investigation indicated that DIF procedures appear to be a promising alternative to assess the interrater reliability of constructed response items, or other polytomous types of items, such as rating scales. Furthermore, using DIF to assess interrater reliability does not require a fully crossed design and allows one to determine if a rater is either more severe, or more lenient, in their scoring of each individual polytomous item on a test or rating scale.

Measurement of dyspnea in patients treated with mechanical ventilation

1999 ◽  
Vol 8 (4) ◽  
pp. 254-261 ◽  
Author(s):  
J Powers ◽  
SJ Bennett
Keyword(s):  
Mechanical Ventilation ◽  
Correlation Coefficient ◽  
Rating Scales ◽  
Rank Order ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Criterion Validity ◽  
Spearman Rank Order Correlation ◽  
Spearman Rank ◽  
Test Retest Reliability

BACKGROUND: Dyspnea, or difficult breathing, is common in patients receiving mechanical ventilation; however, dyspnea is not routinely or systematically measured. OBJECTIVE: The primary purpose of this methodological study was to evaluate the test-retest reliability of 5 dyspnea rating scales and the criterion validity of 4 dyspnea rating scales in patients receiving mechanical ventilation. The secondary purpose was to examine the correlations between each of these 5 rating scales and physiological measures of respiratory function. METHODS: The convenience sample consisted of 28 patients on mechanical ventilation during their hospitalization in the intensive care units of a large, inner-city hospital. Patients rated their dyspnea twice at 30-minute intervals on the visual analogue scale, the vertical analogue dyspnea scale, the modified Borg scale, the numerical scale, and the faces scale. Test-retest reliability was computed by using the intraclass correlation coefficient. Criterion validity was evaluated by using the Spearman rank-order correlation coefficient. RESULTS: The 5 rating scales had acceptable test-retest reliabilities, with intraclass correlation coefficients ranging from 0.81 to 0.97. Criterion validity of the 4 scales also was acceptable, with Spearman rank-order correlation coefficients from 0.76 to 0.96. The rating scales were not correlated with most of the physiological variables. At least half of the patients reported moderate to severe dyspnea. CONCLUSION: The scales showed acceptable reliability and validity, and they will be useful in quantifying dyspnea experienced by patients receiving mechanical ventilation. Further work is needed to evaluate the extent and the severity of dyspnea in such patients in order to evaluate the effectiveness of interventions.

scholarly journals Visual assessment of movement quality in the single leg squat test: a review and meta-analysis of inter-rater and intrarater reliability

2019 ◽  
Vol 5 (1) ◽  
pp. e000541 ◽  
Author(s):  
John Ressman ◽  
Wilhelmus Johannes Andreas Grooten ◽  
Eva Rasmussen Barr
Keyword(s):  
Rating Scales ◽  
Rating Scale ◽  
Meta Analysis ◽  
Intraclass Correlation ◽  
Cochrane Library ◽  
Intrarater Reliability ◽  
Rater Reliability ◽  
Movement Quality ◽  
Step Down ◽  
Single Leg Squat

Single leg squat (SLS) is a common tool used in clinical examination to set and evaluate rehabilitation goals, but also to assess lower extremity function in active people.ObjectivesTo conduct a review and meta-analysis on the inter-rater and intrarater reliability of the SLS, including the lateral step-down (LSD) and forward step-down (FSD) tests.DesignReview with meta-analysis.Data sourcesCINAHL, Cochrane Library, Embase, Medline (OVID) and Web of Science was searched up until December 2018.Eligibility criteriaStudies were eligible for inclusion if they were methodological studies which assessed the inter-rater and/or intrarater reliability of the SLS, FSD and LSD through observation of movement quality.ResultsThirty-one studies were included. The reliability varied largely between studies (inter-rater: kappa/intraclass correlation coefficients (ICC) = 0.00–0.95; intrarater: kappa/ICC = 0.13–1.00), but most of the studies reached ‘moderate’ measures of agreement. The pooled results of ICC/kappa showed a ‘moderate’ agreement for inter-rater reliability, 0.58 (95% CI 0.50 to 0.65), and a ‘substantial’ agreement for intrarater reliability, 0.68 (95% CI 0.60 to 0.74). Subgroup analyses showed a higher pooled agreement for inter-rater reliability of ≤3-point rating scales while no difference was found for different numbers of segmental assessments.ConclusionOur findings indicate that the SLS test including the FSD and LSD tests can be suitable for clinical use regardless of number of observed segments and particularly with a ≤3-point rating scale. Since most of the included studies were affected with some form of methodological bias, our findings must be interpreted with caution.PROSPERO registration numberCRD42018077822.

scholarly journals Signs and Symptoms of Psychotic Illness (SSPI): A rating scale

2002 ◽  
Vol 180 (1) ◽  
pp. 45-50 ◽  
Author(s):  
Peter F. Liddle ◽  
Elton T. C. Ngan ◽  
Gary Duffield ◽  
King Kho ◽  
Anthony J. Warren
Keyword(s):  
Factor Structure ◽  
Rating Scales ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
Signs And Symptoms ◽  
Antipsychotic Treatment ◽  
Sensitivity To Change ◽  
Psychotic Illness ◽  
Score Change ◽  
Reliability Sensitivity

BackgroundIn the rating scales commonly used for assessing response to antipsychotic treatment, individual items embrace symptoms that apparently arise from distinguishable pathophysiological processes and might be expected to respond differently to treatment.AimsTo test the reliability sensitivity to change and factor structure of a new scale for the assessment of the Signs and Symptoms of Psychotic Illness (the SSPI).MethodInterrater reliability was evaluated by determining the intraclass correlation for the ratings of 63 patients. Sensitivity to change was assessed in a longitudinal study of 33 patients. Factor structure was determined from scores for 155 patients.ResultsThe intraclass correlation was satisfactory for all individual items and excellent for the total score. Scores were sensitive to change. A change in Clinical Global Impression of one unit corresponded to an SSPI total score change of 31%. Factor analysis revealed five clusters of symptoms.ConclusionsThe SSPI provides a sensitive and reliable measure of the five major clusters of symptoms that occur commonly in psychotic illness.

scholarly journals Recognizing depression in patients with Parkinson’s disease: accuracy and specificity of two depression rating scale

2006 ◽  
Vol 64 (2b) ◽  
pp. 407-411 ◽  
Author(s):  
Cláudia Débora Silberman ◽  
Jerson Laks ◽  
Cláudia Figueiredo Capitão ◽  
Cláudia Soares Rodrigues ◽  
Irene Moreira ◽  
...  
Keyword(s):  
Clinical Judgment ◽  
Rating Scales ◽  
Rating Scale ◽  
Kappa Statistic ◽  
Roc Curves ◽  
Operating Characteristics ◽  
Predictive Values ◽  
Clinical Diagnoses ◽  
Similar Accuracy ◽  
Kappa Agreement

This study aimed to find cut-off scores for the Montgomery-Asberg rating scale (MADRS) and the Beck depression inventory (BDI) that can relate to specific clinical diagnoses of depression in Parkinson´s disease (PD). Mild and moderate PD patients (n=46) were evaluated for depression according to the DSM IV criteria. All patients were assessed with the MADRS and the BDI. A "receiver operating characteristics" (ROC) curve was obtained and the sensibility, specificity, positive and the negative predictive values were calculated for different cut-off scores of the MADRS and the BDI. The Kappa statistic was calculated for different cut-off scores to assess the agreement between the clinical judgment and both scales. Depression was present in 18 patients. MADRS cut-off scores of 6 and 10 showed Kappa 0.5 and 0.56, respectively. Specificity of cut-off score of 6 was 78.6% and of cut-off score of 10 was 96.4%. Kappa agreement of BDI cut-off scores of 10 and 18 were 0.36 and 0.62, respectively. Specificity was 60.7% for 10 and 92.9% for 18. Both rating scales show similar accuracy within the ROC curves (84.3% for MADRS and 79.7% for BDI). The MADRS and the BDI show a good accuracy and correlation to the clinical diagnosis when a cut-off score of 10 is used to MADRS and a cut-off score of 18 is used to BDI to recognize depression in mild to moderate PD patients. This may help clinicians to recognize depression in PD.

Assessment of Occupational Functioning for Screening of Patients to Occupational Therapy in General Psychiatric Care

1998 ◽  
Vol 18 (4) ◽  
pp. 193-206 ◽  
Author(s):  
Lena Haglund ◽  
Lars-Hakan Thorell ◽  
Jan Walinder
Keyword(s):  
Occupational Therapy ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Interrater Reliability ◽  
Psychiatric Care ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
Case Analysis ◽  
Occupational Therapist ◽  
Swedish Version

A Swedish version of the Occupational Case Analysis Interview and Rating Scale (OCAIRS-S) has been tested earlier for interrater reliability. The present study, using the second version of OCAIRS-S and including a sample of 145 patients, showed interrater correlations between .88 and .96 (Intraclass Correlation Coefficient). The results indicate that OCAIRS-S predicts which patients should be included in and excluded from occupational therapy and identifies patients who should be observed more before making such decisions. The study indicates a need for further investigations regarding which components in OCAIRS-S influence the occupational therapist in judging the patient's need for occupational therapy.

Iowa's Severity Rating Scales for Communication Disabilities

1982 ◽  
Vol 13 (3) ◽  
pp. 156-162 ◽  
Author(s):  
Kenneth D. Barker ◽  
Robert A. Baldes ◽  
Phillip H. Jenkinson ◽  
Kenyon D. Wilson ◽  
J. Joseph Freilinger
Keyword(s):  
Service Delivery ◽  
Clinical Judgment ◽  
Rating Scales ◽  
Rating Scale ◽  
Language Fluency ◽  
State Laws ◽  
Severity Rating ◽  
Speech Language Pathologist ◽  
Restrictive Environment ◽  
The Continuum

PL 94-142 mandates that all pupils who are handicapped receive an appropriate program in the least restrictive environment. In implementing both federal and state laws, it was necessary to depart from the traditional methods of service delivery, and so a continuum of services concept was adopted. Attached to the continuum of services are severity rating scales that indicate milestones along the continuum. A severity rating scale was developed for articulation, language, fluency, and voice. Although providing direction for the speech-language pathologist, clinical judgment was not compromised. The severity rating scale assists the clinician in case selection and suggests the intensity of the service delivery model. Data obtained from use of the scale can be employed to appropriately assign staff. Each scale is reproduced in an appendix to this article.

scholarly journals Cross-cultural adaptation and validation of the Romanian International Knee Documentation Committee—subjective knee form

PeerJ ◽  
2020 ◽  
Vol 8 ◽  
pp. e8448 ◽  
Author(s):  
Adrian Todor ◽  
Dinu Vermesan ◽  
Horia Haragus ◽  
Jenel M. Patrascu Jr ◽  
Bogdan Timar ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Internal Consistency ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
International Knee Documentation Committee ◽  
Repeated Administration ◽  
Data Sets ◽  
Cronbach’S Alpha ◽  
Female Ratio ◽  
Cronbach's Alpha

Aim We aimed to translate and cross-culturally adapt the International Knee Documentation Committee—subjective knee form (IKDC) in Romanian. Method The original (US) IKDC—subjective knee form was translated according to recommended guidelines. Validity was tested using Spearmans’s correlation coefficient between score sand test-retest reproducibility. Reliability and internal consistency were determined using Cronbach’s alpha coefficient and intraclass correlation coefficient (ICC). Results A total of 106 data sets were available for processing. The average age was 52 years and the male to female ratio was 40:66. Fifty-five subjects repeated the form after an average of 4 days. There were no floor or ceiling effects (range 3.4–74.7). There was a strong correlation between the first and repeated administration of the IKDC—subjective knee form (r = 0.816, n = 50) and moderate compared to Tegner-Lysholm knee rating scale (r = 0.506, n = 102), KOOSJR (Knee disability and Osteoarthritis Outcome Score for Joint Replacement, r =  − 0.622, n = 96), EuroqolEQ-5D-5L Index (r = 0.633, n = 100) and visual analogue scale VAS (r = 0.484, n = 99). Internal consistency was moderate with Cronbach’s alpha 0.611 (n = 102) and ICC 0.611 for average measures (95% CI 0.493–0.713). Conclusion The Romanian translation of the IKDC—subjective knee form is a valid, consistent and reproducible outcome measure in patients with knee pain and dysfunction.

A comparison of Numeric Pain Rating Scale (NPRS) and the Visual Analog Scale (VAS) in patients with chronic cancer-associated pain.

2017 ◽  
Vol 35 (31_suppl) ◽  
pp. 217-217 ◽  
Author(s):  
Shagufta Firdous ◽  
Zankhana Mehta ◽  
Carlos Fernandez ◽  
Bertarnd Behm ◽  
Mellar Davis
Keyword(s):  
Correlation Coefficient ◽  
Visual Analog Scale ◽  
Rating Scales ◽  
Rating Scale ◽  
Metastatic Cancer ◽  
Pearson Correlation ◽  
Analog Scale ◽  
Full Data ◽  
Number Of Patients ◽  
The Mean

217 Background: Though numerical rating scales (NRS) are frequently used, a subset of patients do not understand the abstract nature on NRS and use quality descriptors of severity. We used a visual analog scale (VAS) using descriptors of “good day”, “average day” and “bad day” along a 10-cm line and compared it to a NRS in patients with cancer diagnosis and pain 1 month or longer. Methods: Eligible patients were verbally consented and completed the two scales. We asked patients thoughts about which they felt was the more appropriate scale to gauge their pain. The scales were compared by the distance from the extreme left of the line to the mark the patient placed on the VAS. Additional data included demographic and diagnosis information. The VAS and NRS were compared using two-sample t-tests or Wilcoxon rank-sum tests and Pearson’s chi-square or Fisher’s exact tests, as appropriate. The relationship between the numeric pain scale and the visual analog scale is described using Pearson’s correlation coefficient. Results: Full data was provided from 94 patient surveys who completed surveys. The mean age was 61 years old, 56.4% were female, and 78.0% had metastatic cancer. Numerous cancer types were reported with gastrointestinal and lung cancers being the most common (30.1% and 23.7%, respectively). The mean NRS rating reported by patients was 5.9 and for the VAS 4.9. Sixty patients (63.8%) preferred the NRS, 32 (34.0%) preferred the VAS, and 2 (2.1%) reported no preference. Patients who preferred the NRS reported a higher NRS rating than patients who preferred the VAS (mean NRS of 6.3 compared to 5.2, p = 0.0409). VAS ratings were higher among patients who preferred the NRS but the difference was not statistically significant (mean rating of 5.2 vs. 4.4, p = 0.1894). There were no differences in patient characteristics between the two groups. There was a moderate association between patients’ NRS and VAS ratings. The Pearson correlation coefficient was 0.653 (p < 0.0001). Conclusions: Majority of patients of all ages and both genders prefer NRS. Future studies involving larger number of patients and adding a question about the reason of scale preference may provide better understanding for scale preference.

scholarly journals Continuous Assessment of Labour Pain using Handgrip Force

2015 ◽  
Vol 20 (3) ◽  
pp. 159-163
Author(s):  
Nadine Wickboldt ◽  
Georges Savoldelli ◽  
Benno Rehberg-Klug
Keyword(s):  
Pain Intensity ◽  
Correlation Coefficient ◽  
Rating Scales ◽  
Rating Scale ◽  
Numerical Rating Scale ◽  
Area Under The Curve ◽  
Labour Pain ◽  
Voluntary Action ◽  
Continuous Assessment ◽  
Obstetric Analgesia

BACKGROUND: Assessment of dynamic changes in painful experiences, such as labour, using conventional rating scales (eg, numerical rating scale [NRS]) has limitations. An alternative for continuous pain evaluation could be a signal generated by voluntary action of the parturient. Remifentanil administration for obstetric analgesia could be improved by these dynamic measures of labour pain. In the present study, handgrip force was measured by a dynamometer to signal labour pain.OBJECTIVES: To evaluate: whether continuous monitoring of labour pain using handgrip force allows for determination of pain measurement during contractions; and the correlation between handgrip force and pain intensity on NRS.METHODS: The present observational, single-centre study included 43 parturients. After calibration of the dynamometer for individual hand muscle strength, pain was recorded during early and late labour using a dynamometer and an NRS. The primary end point was the correlation coefficient between NRS ratings and peak intensity recorded by the dynamometer.RESULTS: All dynamometer-registered readings were also registered by the external tocogram. All contractions recorded by external tocogram were also registered by the dynamometer. Handgrip force was moderately correlated with pain scores on the NRS. Mean handgrip force during contractions had the highest correlation coefficient (Pearson’s r=0.67) compared with peak handgrip force (r=0.56) and area under the curve of handgrip force (r=0.55).CONCLUSIONS: Pain intensity and duration can be assessed continuously using handgrip force measured via a dynamometer. The feedback of intensity and duration of pain could optimize patient-controlled remifentantil application for obstetric analgesia and other situations of highly variable pain intensity.

Sign in / Sign up

Export Citation Format

Share Document