scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Items

2020 ◽  
Author(s):  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour ◽  
Suha Almuhaissen
Keyword(s):  
Psychometric Properties ◽  
Academic Programs ◽  
Assessment Tools ◽  
Pharmacy Students ◽  
Psychometric Characteristics ◽  
Traditional Assessment ◽  
Clinical Clerkships ◽  
Response Choices ◽  
Case Based

Abstract Background Examinations are the traditional assessment tools. In addition to measurement of learning, exams are used to guide the improvement of academic programs. The current study attempted to evaluate the quality of assessment items of sixth year clinical clerkships examinations as a function of assessment items format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items . Methods. A total of 173 assessment items used in the examinations of sixth year clinical clerkships of a PharmD program were included. Items were classified as case based or noncase based and as MCQs or open-ended. The psychometric characteristics of the items were studied as a function of the Bloom’s levels addressed, item format, and number of choices in MCQs. Results . Items addressing analysis skills were more difficult. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the latter . Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as noncase based items. Conclusion. Item formats, structure, and number of options in MCQs significantly affected the psychometric properties of the studied items. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. Examination items should be prepared considering the above characteristics to improve their psychometric properties and maximize their usefulness.

scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Items

2020 ◽  
Author(s):  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour ◽  
Suha Almuhaissen
Keyword(s):  
Psychometric Properties ◽  
Academic Programs ◽  
Assessment Tools ◽  
Pharmacy Students ◽  
Psychometric Characteristics ◽  
Traditional Assessment ◽  
Clinical Clerkships ◽  
Response Choices ◽  
Case Based

Abstract Background Examinations are the traditional assessment tools. In addition to measurement of learning, exams are used to guide the improvement of academic programs. The current study attempted to evaluate the quality of assessment items of sixth year clinical clerkships examinations as a function of assessment items format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods A total of 173 assessment items used in the examinations of sixth year clinical clerkships of a PharmD program were included. Items were classified as case based or noncase based and as MCQs or open-ended. The psychometric characteristics of the items were studied as a function of the Bloom’s levels addressed, item format, and number of choices in MCQs. Results Items addressing analysis skills were more difficult. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the latter . Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier. Open-ended questions were significantly more discriminative in comparison to MCQs as case based items while they were more discriminative as noncase based items. Conclusion Item formats, structure, and number of options in MCQs significantly affected the psychometric properties of the studied items. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively. Examination items should be prepared considering the above characteristics to improve their psychometric properties and maximize their usefulness.

scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Tools

2019 ◽  
Author(s):  
Suha Al Muhaissen ◽  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour
Keyword(s):  
Assessment Tool ◽  
Academic Programs ◽  
Assessment Tools ◽  
Item Format ◽  
Traditional Assessment ◽  
Cognitive Levels ◽  
Examination Quality ◽  
Standardized Examinations ◽  
Case Based

Abstract Background Examination is the traditional assessment tool, and are used to guide the improvement of academic programs. Accreditation committees’ emphasis on the implementation of standardized examinations. The aims of the current study are to evaluate the quality of assessment tools of sixth year PharmD students as a function of assessment item format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods. A total of 173 assessment items used in the examinations of sixth year rotations of PharmD program were included. Items were classified as case based or non-case based and as MCQs or open-ended. The psychometric characteristics of examinations were studied as a function of the level of the Bloom’s levels addressed by an item, item format, and number of choices in MCQs. Results. Items addressing intellectual and analysis skills were more difficult, while items associated with multiple cognitive levels were more discriminative. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the letter. Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as non-case based items. Conclusion. Item formats, structure, and number of options in MCQs, affected students’ performance and overall examination quality. The discrimination of items associated with multiple Bloom’s levels was significantly higher than those associated with a single level. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively.

scholarly journals Measuring test-retest reliability (TRR) of AMSTAR provides moderate to perfect agreement – a contribution to the discussion of the importance of TRR in relation to the psychometric properties of assessment tools

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Stefanie Bühn ◽  
Peggy Ober ◽  
Tim Mathes ◽  
Uta Wegewitz ◽  
Anja Jacobs ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Assessment Tools ◽  
Measurement Properties ◽  
Perfect Agreement ◽  
Retest Reliability ◽  
Test Retest Reliability ◽  
The Impact

Abstract Background Systematic Reviews (SRs) can build the groundwork for evidence-based health care decision-making. A sound methodological quality of SRs is crucial. AMSTAR (A Measurement Tool to Assess Systematic Reviews) is a widely used tool developed to assess the methodological quality of SRs of randomized controlled trials (RCTs). Research shows that AMSTAR seems to be valid and reliable in terms of interrater reliability (IRR), but the test retest reliability (TRR) of AMSTAR has never been investigated. In our study we investigated the TRR of AMSTAR to evaluate the importance of its measurement and contribute to the discussion of the measurement properties of AMSTAR and other quality assessment tools. Methods Seven raters at three institutions independently assessed the methodological quality of SRs in the field of occupational health with AMSTAR. Between the first and second ratings was a timespan of approximately two years. Answers were dichotomized, and we calculated the TRR of all raters and AMSTAR items using Gwet’s AC1 coefficient. To investigate the impact of variation in the ratings over time, we obtained summary scores for each review. Results AMSTAR item 4 (Was the status of publication used as an inclusion criterion?) provided the lowest median TRR of 0.53 (moderate agreement). Perfect agreement of all reviewers was detected for AMSTAR-item 1 with a Gwet’s AC1 of 1, which represented perfect agreement. The median TRR of the single raters varied between 0.69 (substantial agreement) and 0.89 (almost perfect agreement). Variation of two or more points in yes-scored AMSTAR items was observed in 65% (73/112) of all assessments. Conclusions The high variation between the first and second AMSTAR ratings suggests that consideration of the TRR is important when evaluating the psychometric properties of AMSTAR.. However, more evidence is needed to investigate this neglected issue of measurement properties. Our results may initiate discussion of the importance of considering the TRR of assessment tools. A further examination of the TRR of AMSTAR, as well as other recently established rating tools such as AMSTAR 2 and ROBIS (Risk Of Bias In Systematic reviews), would be useful.

Critical appraisal of questionnaires to assess functional impairment in individuals with mild cognitive impairment

2016 ◽  
Vol 28 (9) ◽  
pp. 1425-1439 ◽  
Author(s):  
Navaldeep Kaur ◽  
Patricia Belchior ◽  
Isabelle Gelinas ◽  
Nathalie Bier
Keyword(s):  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Activities Of Daily Living ◽  
Psychometric Properties ◽  
Daily Living ◽  
Functional Independence ◽  
Assessment Tools ◽  
Cosmin Checklist ◽  
Psychometric Studies

ABSTRACTBackground:Mild deficits in instrumental activities of daily living (IADLs) have consistently been reported in the individuals with mild cognitive impairment (MCI). A variety of functional assessment tools, including self-and informant report questionnaires and performance-based measures, have been employed in MCI. Previously, a limited focus has been directed at appraising the quality of questionnaires. The goal of this study was to identify the questionnaires that have been validated in the MCI population. Additionally, the quality of validation studies and psychometric attributes of these questionnaires were appraised.Methods:Relevant articles were systematically searched in PsychINFO, Ovid MEDLINE, and CINAHL against specific eligibility criteria. To evaluate the methodology of the psychometric studies, the COSMIN checklist was employed. Also, the psychometric properties of the assessment tools were evaluated based upon Terwee's criteria.Results:A total of five psychometric studies and questionnaires were critically evaluated. Varying psychometric properties were available for the chosen tools. None of the studies received the best possible rating for their methodological quality. It was found that questionnaires with high discriminative ability to distinguish MCI from other diagnostic groups were: Disability Assessment in Dementia-6 (DAD-6), Functional Activity Questionnaire (FAQ), and Alzheimer's Disease Cooperative Study/Activities of Daily Living scale adapted for MCI patients (ADCS-MCI-ADL-24).Conclusion:Psychometric studies with strong methodological rigor are required in the future. Considering the fact that IADL decline has been associated with dementia, early detection of functional difficulties in MCI needs to be encouraged as it will allow suitable and timely interventions to prolong functional independence of affected individuals.

Screening for Distress and Unmet Needs in Patients With Cancer: Review and Recommendations

2012 ◽  
Vol 30 (11) ◽  
pp. 1160-1177 ◽  
Author(s):  
Linda E. Carlson ◽  
Amy Waller ◽  
Alex J. Mitchell
Keyword(s):  
Quality Of Life ◽  
Psychometric Properties ◽  
Unmet Needs ◽  
Assessment Tools ◽  
Screening Tools ◽  
Distress Screening ◽  
Screening Programs ◽  
Patients With Cancer ◽  
Screening For Distress

Purpose This review summarizes the need for and process of screening for distress and assessing unmet needs of patients with cancer as well as the possible benefits of implementing screening. Methods Three areas of the relevant literature were reviewed and summarized using structured literature searches: psychometric properties of commonly used distress screening tools, psychometric properties of relevant unmet needs assessment tools, and implementation of distress screening programs that assessed patient-reported outcomes (PROs). Results Distress and unmet needs are common problems in cancer settings, and programs that routinely screen for and treat distress are feasible, particularly when staff are supported and links with specialist psychosocial services exist. Many distress screening and unmet need tools have been subject to preliminary validation, but few have been compared head to head in independent centers and in different stages of cancer. Research investigating the overall effectiveness of screening for distress in terms of improved recognition and treatment of distress and associated problems is not yet conclusive, but screening seems to improve communication between patients and clinicians and may enhance psychosocial referrals. Direct effects on quality of life are uncertain, but screening may help improve discussion of quality-of-life issues. Conclusion Involving all stakeholders and frontline clinicians when planning screening for distress programs is recommended. Training frontline staff to deliver screening programs is crucial, and continuing to rigorously evaluate outcomes, including PROs, process of care, referrals, and economic costs and benefits is essential.

scholarly journals A protocol for a systematic review and meta-analysis to identify measures of breakthrough pain and evaluate their psychometric properties

BMJ Open ◽  
2020 ◽  
Vol 10 (3) ◽  
pp. e035541
Author(s):  
Katie Greenfield ◽  
Simone Holley ◽  
Daniel Eric Schoth ◽  
Julie Bayliss ◽  
Anna-Karenia Anderson ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Full Text ◽  
Breakthrough Pain ◽  
Meta Analysis ◽  
Assessment Tools ◽  
Assessment Process ◽  
Measurement Properties ◽  
Cochrane Library ◽  
Health Measurement

IntroductionBreakthrough pain is common in children and adults with cancer and other conditions, including those approaching end-of-life, although it is often poorly managed, possibly partly due to a lack of validated assessment tools. This review aims to (1) identify all available instruments measuring breakthrough pain in infants, children, adolescents or adults and (2) critically appraise, compare and summarise the quality of the psychometric properties of the identified instruments using COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria.Methods and analysisTwo searches will be carried out between October 2019 and January 2020, one for each aim of the review. The Cochrane Library, International Prospective Register of Systematic Reviews, Embase, Cumulative Index of Nursing and Allied Health Literature, Medical Literature Analysis and Retrieval System Online (MEDLINE), PsycINFO, Web of Science Core Collection, Google Scholar, the ProQuest Dissertations & Theses Database, Evidence Search and OpenGrey databases will be searched from database inception until the date the search is conducted. Reference lists of eligible articles will be screened and authors in the field contacted. For search 1, articles will be screened by two reviewers by abstract, and full-text where necessary, to identify if a breakthrough pain assessment was used. Search 2 will then be conducted to identify studies evaluating measurement properties of these assessments. Two reviewers will screen articles from search 2 by title and abstract. All potentially relevant studies will be screened by full text by both reviewers. For search 2, data will be extracted in parallel with the quality assessment process, as recommended by COSMIN. Two reviewers will assess methodological quality using the COSMIN Risk of Bias checklist and the COSMIN updated criteria for good measurement properties. Findings will be summarised and, if possible, data will be pooled using meta-analysis. The quality of the evidence will be graded and summarised using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) guidelines.Ethics and disseminationResults of this review will be submitted for publication in a peer review journal and presented at conferences.PROSPERO registration numberCRD42019155583.

Psychometric properties of home accessibility assessment tools: A systematic review

2019 ◽  
Vol 86 (3) ◽  
pp. 172-184 ◽  
Author(s):  
Alexandre Patry ◽  
Claude Vincent ◽  
Christian Duval ◽  
Emmanuelle Careau
Keyword(s):  
Systematic Review ◽  
Psychometric Properties ◽  
Home Environment ◽  
Critical Appraisal ◽  
Reliability And Validity ◽  
Occupational Therapists ◽  
Assessment Tools ◽  
Good Reliability ◽  
Standardized Measures

Background. Although home environment assessments are commonly performed by occupational therapists working in home care, use of nonstandardized measures created in-house or lack of measure use can cast a shadow over the quality of these assessments for people with disabilities. To ensure quality of home environment assessments, occupational therapists need standardized measures with demonstrated psychometric properties. Purpose. This study provides a critical appraisal of objective accessibility measures of the home environment. Method. A systematic review was undertaken for which three databases—CINAHL, PubMed, and Embase—were searched to identify accessibility measures of the home environment and evaluate their psychometric properties. Two authors independently assessed the quality of selected studies using the critical appraisal form for psychometric articles. Findings. Ten studies discussing seven accessibility measures were identified and selected for this review. No measures showed strong evidence of both good reliability and validity. Only one study addressed the responsiveness of a measure of accessibility. Implications. As occupational therapists are specialists of the person–environment relationship, the lack of evidence of the psychometric properties of objective accessibility measures of the home environment harms evidence-based occupational therapy practice. This review identified the most promising assessment tools, but further research is needed.

scholarly journals School health assessment tools: a systematic review of measurement in primary schools

PeerJ ◽  
2020 ◽  
Vol 8 ◽  
pp. e9459
Author(s):  
Maryam Kazemitabar ◽  
Ali Moghadamzadeh ◽  
Mojtaba Habibi ◽  
Rezvan Hakimzadeh ◽  
Danilo Garcia
Keyword(s):  
Systematic Review ◽  
Psychometric Properties ◽  
Primary Schools ◽  
School Health ◽  
Health Assessment ◽  
Risk Of Bias ◽  
Assessment Tools ◽  
Measurement Properties ◽  
Cosmin Checklist

Background This systematic review aimed to investigate the psychometric properties of the school health’s assessment tools in primary schools through COSMIN Risk of Bias checklist. We examined the studies that have addressed the measurement properties of school-health instruments to give a clear overview of the quality of all available tools measuring school health in primary schools. This systematic review was registered in PROPERO with the Registration ID: CRD42020158158. Method Databases of EBSCOhost, PubMed, ProQuest, Wily, PROSPERO, and OpenGrey were systematically searched without any time limitation to find all full-text English journal articles studied at least one of the COSMIN checklist measurement properties of a school-health assessment tool in primary schools. The instruments should be constructed based on a school health model. The eligible studies were assessed by COSMIN Risk of Bias checklist to report their quality of methodology for each measurement property and for the whole study by rating high, moderate or low quality. Results At the final screening just seven studies remained for review. Four studies were tool development, three of them were rated as “adequate” and the other study as “very good”; five studies examined the content validity, three of them were appraised as “very good”, and the two remaining as “inadequate”. All seven studies measured structural validity, three of them were evaluated as “very good”, three other were scored as “adequate”, and the last study as “inadequate”. All the seven studies investigated the internal consistency, five of them were assessed as “very good”, one was rated as “doubtful”, and the last one as “inadequate”. Just one study examined the cross-cultural validity and was rated as “adequate”. Finally, all seven studies measured reliability, two of them were rated as “very good” and the rest five studies were appraised as “doubtful”. All rating was based on COSMIN checklist criteria for quality of measurement properties assessment. Conclusion The number of studies addressing school health assessment tools was very low and therefore not sufficient. Hence, there is a serious need to investigate the psychometric properties of the available instruments measuring school health at primary schools. Moreover, the studies included in the present systematic review did not fulfill all the criteria of the COSMIN checklist for assessing measurement properties. We suggest that future studies consider these criteria for measuring psychometric properties and developing school health assessment tools.

scholarly journals Psychometric Properties of the Spanish Version of Psychosocial Impact of Assistive Devices Scale in a Large Sample of People With Neuromuscular, Neurological, or Hearing Disabilities

2021 ◽  
Vol 12 ◽  
Author(s):  
Emiliano Díez ◽  
Estíbaliz Jiménez-Arberas ◽  
Thais Pousada
Keyword(s):  
Assistive Technology ◽  
Psychometric Properties ◽  
Psychosocial Impact ◽  
Assistive Devices ◽  
Assessment Tools ◽  
Spanish Version ◽  
Assistive Device ◽  
Measurement Properties ◽  
Large Sample

Assistive technology (AT) is any device, software, or equipment designed for and used by individuals with disabilities to engage in everyday activities and achieve independence. However, the usefulness of those technology-based or supported treatments is a complex issue that has led to the development of various conceptual models for assistive technology outcomes research and practice as well as different assessment tools that help to explore the effect of technology on people's lives. One of those instruments is the Psychosocial Impact of Assistive Device Scale (PIADS), a 26-item questionnaire that measures the psychosocial impact of interventions, using assistive devices in three quality-of-life domains: competence, adaptability, and self-esteem. PIADS scale has been translated and adapted to several languages, and it has been successfully used to measure AT outcomes in different disability profiles to predict abandonment or even as a relevant determinant of future adoption of eHealth. Quinteiro (2010) adapted PIADS to Spanish for the first time, although no studies have yet been published to systematically study its psychometric properties. Therefore, the purpose of this study was to evaluate measurement properties of the Spanish version of PIADS scale by means of a dataset obtained from its application to a large sample (n = 417) of people with neuromuscular, neurological, or hearing disabilities that used different assistive devices. The results will provide valuable indicators about the measurement quality of the Spanish PIADS scale and will help to promote the use of reliable and valid AT outcome assessment tools for research and clinical purposes.

scholarly journals A Psychometric Analysis Update of Behavioral Pain Assessment Tools for Noncommunicative, Critically Ill Adults

2019 ◽  
Vol 30 (4) ◽  
pp. 365-387 ◽  
Author(s):  
Céline Gélinas ◽  
Aaron M. Joffe ◽  
Paul M. Szumita ◽  
Jean-Francois Payen ◽  
Mélanie Bérubé ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Critically Ill ◽  
Pain Assessment ◽  
Pain Scale ◽  
Assessment Tools ◽  
Process Reliability ◽  
Behavioral Pain Scale ◽  
Critically Ill Adults ◽  
Ill Adults

This is an updated, comprehensive review of the psychometric properties of behavioral pain assessment tools for use with noncommunicative, critically ill adults. Articles were searched in 5 health databases. A total of 106 articles were analyzed, including 54 recently published papers. Nine behavioral pain assessment tools developed for noncommunicative critically ill adults and 4 tools developed for other non-communicative populations were included. The scale development process, reliability, validity, feasibility, and clinical utility were analyzed using a 0 to 20 scoring system, and quality of evidence was also evaluated. The Behavioral Pain Scale, the Behavioral Pain Scale-Nonintubated, and the Critical-Care Pain Observation Tool remain the tools with the strongest psychometric properties, with validation testing having been conducted in multiple countries and various languages. Other tools may be good alternatives, but additional research on them is necessary.

Sign in / Sign up

Export Citation Format

Share Document