scholarly journals Designing and verifying a tool for diagnosing scientific misconceptions in genetics topic

2020 ◽  
Vol 9 (3) ◽  
pp. 564
Author(s):  
Sasithorn Kantahan ◽  
Putcharee Junpeng ◽  
Sompong Punturat ◽  
Keow Ngang Tang ◽  
Perman Gochyyev ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Item Analysis ◽  
Multinomial Logit Model ◽  
Two Dimensions ◽  
Sufficient Evidence ◽  
Evidence Based ◽  
Multidimensional Model ◽  
Outcome Space ◽  
Best Fit

The main purpose of this study was to design and verify the quality of an assessment aimed at diagnosing scientific misconceptions among students enrolled in 10th-grade biology subjects. A sample consisted of N=200 students from schools under the administration of the Office of Secondary Educational Service 31, Nakhon Ratchasima province. We employed the design-based approach, which consists of four phases, namely, construct map, item design, outcome space, and Wright map. Multidimensional Random Coefficient Multinomial Logit Model (MRCMLM) was used to evaluate the quality of the assessment tool. The assessment tool consists of two dimensions, namely, knowledge and reasoning. The assessment tool is comprised of 40 items, with 20 items tapping each of the dimensions. Findings form item analysis and modeling revealed sufficient evidence on the internal structure and validity of the instrument. It can be concluded that the unidimensional model is more appropriate than the multidimensional model for diagnosing scientific misconceptions in the genetics topic for dichotomously scored with four options in each item. The polytomous scored type should be further investigated to determine the best fit model in the items relating to evidence-based reasoning for supporting students’ answers.

scholarly journals Designing Standards-Setting for Levels of Mathematical Proficiency in Measurement and Geometry: Multidimensional Item Response Model

2021 ◽  
Vol 10 (6) ◽  
pp. 103
Author(s):  
Sudarat Phaniew ◽  
Putcharee Junpeng ◽  
Keow Ngang Tang
Keyword(s):  
Assessment Tool ◽  
Multinomial Logit Model ◽  
Measurement Model ◽  
Two Dimensions ◽  
Item Response Model ◽  
Multidimensional Item Response ◽  
Item Fit ◽  
Mathematical Proficiency ◽  
Mathematical Process

This study intends to design and verify the quality of a model that measures mathematical proficiency and aims to set the standards in measuring levels of proficiency in the subjects of measurement and geometry. Construct modeling was employed to design a mathematical proficiency measurement model which consists of the mathematical process and the dimensions of a conceptual structure. A total of 517 Secondary Year 1 students were selected from the big data to participate as test-takers. Design-based research encompassing four phases was used to verify the quality of the mathematical proficiency measurement model. A Multidimensional Random Coefficient Multinomial Logit model was used to examine the standards-setting of the mathematical proficiency measurement model. The results indicated that the two dimensions of mathematical proficiency can be further divided into five levels, from non-response/irrelevance to strategic/extended thinking and extended abstract structure for mathematical process and conceptual structural dimensions, respectively. The assessment tool covers 18 items with 15 multiple-choice items and three subjective items in measurement and geometry. Moreover, the results also demonstrated that the validity evidence associated with the internal structure of the multidimensional model is fit. Besides, reliability evidence, as well as item fit, is compliance with the quality of the mathematical proficiency measurement model as illustrated in analysis of the standard error of measurement and infit and outfit of the items. Finally, the researchers managed to set standards for the mathematical proficiency measurement model based on the assessment criterion results from the Wright Map. In conclusion, the standards-setting of the mathematical proficiency measurement model provides substantial information, particularly for measuring those students who are above the lowest level of mathematical proficiency because the error for estimating proficiency was low.

Validation evidence of the paediatric Objective Structured Assessment of Debriefing (OSAD) tool

2016 ◽  
Vol 2 (3) ◽  
pp. 61-67 ◽  
Author(s):  
Jane Runnacles ◽  
Libby Thomas ◽  
James Korndorffer ◽  
Sonal Arora ◽  
Nick Sevdalis
Keyword(s):  
Assessment Tool ◽  
Learning Experience ◽  
Pearson Correlation ◽  
Phase 1 ◽  
Evidence Based ◽  
Validity Evidence ◽  
Multiple Sources ◽  
Simulation Based ◽  
Expert Input

IntroductionDebriefing is essential to maximise the simulation-based learning experience, but until recently, there was little guidance on an effective paediatric debriefing. A debriefing assessment tool, Objective Structured Assessment of Debriefing (OSAD), has been developed to measure the quality of feedback in paediatric simulation debriefings. This study gathers and evaluates the validity evidence of OSAD with reference to the contemporary hypothesis-driven approach to validity.MethodsExpert input on the paediatric OSAD tool from 10 paediatric simulation facilitators provided validity evidence based on content and feasibility (phase 1). Evidence for internal structure validity was sought by examining reliability of scores from video ratings of 35 postsimulation debriefings; and evidence for validity based on relationship to other variables was sought by comparing results with trainee ratings of the same debriefings (phase 2).ResultsSimulation experts’ scores were significantly positive regarding the content of OSAD and its instructions. OSAD's feasibility was demonstrated with positive comments regarding clarity and application. Inter-rater reliability was demonstrated with intraclass correlations above 0.45 for 6 of the 7 dimensions of OSAD. The internal consistency of OSAD (Cronbach α) was 0.78. Pearson correlation of trainee total score with OSAD total score was 0.82 (p<0.001) demonstrating validity evidence based on relationships to other variables.ConclusionThe paediatric OSAD tool provides a structured approach to debriefing, which is evidence-based, has multiple sources of validity evidence and is relevant to end-users. OSAD may be used to improve the quality of debriefing after paediatric simulations.

scholarly journals Development of an assessment tool for mathematical reading, analytical thinking and mathematical writing

2020 ◽  
Vol 9 (4) ◽  
pp. 955
Author(s):  
Nuttida Kesorn ◽  
Putcharee Junpeng ◽  
Metta Marwiang ◽  
Kissadapan Pongboriboon ◽  
Keow Ngang Tang ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Fourth Grade ◽  
Two Dimensions ◽  
Random Coefficients ◽  
Ratio Test ◽  
Item Response Model ◽  
Mathematical Writing ◽  
Analytical Thinking ◽  
Fourth Grade Students

<p>The main objective of this research was to develop and validate the quality of an assessment tool for evaluating the mathematical reading, analytical thinking, and mathematical writing skills of fourth-grade students. We randomly selected 222 fourth grade students across multiple schools of varying sizes to take the assessment. Multidimensional Random Coefficients Multinomial Item Response Model was applied to validate the quality of the developed assessment tool. A design-based research methodology was adopted to develop the assessment tool encompassing four phases as follows: 1) analyze how students solve mathematical problems; 2) develop the assessment tool; 3) validation of the tool; and 4) reflection. The results of this research indicate that the assessment tool consisting of 19 items and two dimensions is a reliable and valid metric to measure mathematical reading, mathematical writing and analytical ability of fourth graders. The Likelihood-Ratio test showed that the multidimensional model fits better in comparison to the unidimensional model. It can be concluded that each item is qualified to assess the students and relevant to the developed dimensional examination structure.</p>

Protecting Musicians from Hearing Damage: A Review of Evidence-based Research

2009 ◽  
Vol 24 (3) ◽  
pp. 103-107
Author(s):  
Michael Santucci
Keyword(s):  
At Risk ◽  
Hearing Loss ◽  
Sufficient Evidence ◽  
Evidence Based ◽  
Loss Prevention ◽  
Sound Levels ◽  
Protection Devices ◽  
Hearing Damage

The great irony of being a musician is that the sound produced—the very essence of the musical experience—represents a long-term health risk to the artist. This overview examines the lack of sufficient evidence-based studies on this at-risk population, which is particularly important in light of the low level of compliance to hearing-loss prevention programs among musicians. The review explores the number of musicians at risk, the five most common types of hearing loss affecting them, and the necessary components of a hearing-loss prevention program, including measurement, education, and acoustic modifications to the work environment. Hearing protection devices designed specifically for performing musicians are explored in depth, including the proper use of spectrum-neutral high-fidelity earplugs with in-ear monitoring systems as tools to control sound levels without detracting from the quality of musical performance.

scholarly journals An evidence assessment tool for ecosystem services and conservation studies

2014 ◽  
Author(s):  
Anne-Christine Mupepele ◽  
Jessica C. Walsh ◽  
William J. Sutherland ◽  
Carsten F. Dormann
Keyword(s):  
Environmental Management ◽  
Ecosystem Services ◽  
Assessment Tool ◽  
Evidence Based Practice ◽  
Evidence Based ◽  
Research Implementation ◽  
Hierarchy Of Evidence ◽  
Research Findings ◽  
Quality Of Research

Reliability of scientific findings is important, especially if they directly impact decision making, such as in environmental management. In the 1990s, assessments of reliability in the medical field resulted in the development of evidence-based practice. Ten years later, evidence-based practice was translated into conservation, but so far no guidelines exist on how to assess the evidence of individual studies. Assessing the evidence of individual studies is essential to appropriately identify and summarize the confidence in research findings. We develop a tool to assess the strength of evidence of ecosystem services and conservation studies. This tool consists of (1) a hierarchy of evidence, based on the experimental design of studies and (2) a critical-appraisal checklist that identifies the quality of research implementation. The application is illustrated with 13 examples and we suggest further steps required to move towards more evidence-based environmental management.

scholarly journals Do we know how scabies outbreaks in residential and nursing care homes for the elderly should be managed? A systematic review of interventions using a novel approach to assess evidence quality

2019 ◽  
Vol 147 ◽  
Author(s):  
E. J. Morrison ◽  
J. Middleton ◽  
S. Lanza ◽  
J. E. Cowen ◽  
K. Hewitt ◽  
...  
Keyword(s):  
Drug Treatment ◽  
Outcome Measures ◽  
Nursing Care ◽  
Assessment Tool ◽  
The Elderly ◽  
Evidence Based ◽  
Quality Assessment Tool ◽  
Homes For The Elderly ◽  
Optimal Drug

Abstract Currently no national guidelines exist for the management of scabies outbreaks in residential or nursing care homes for the elderly in the United Kingdom. In this setting, diagnosis and treatment of scabies outbreaks is often delayed and optimal drug treatment, environmental control measures and even outcome measures are unclear. We undertook a systematic review to establish the efficacy of outbreak management interventions and determine evidence-based recommendations. Four electronic databases were searched for relevant studies, which were assessed using a quality assessment tool drawing on STROBE guidelines to describe the quality of observational data. Nineteen outbreak reports were identified, describing both drug treatment and environmental management measures. The quality of data was poor; none reported all outcome measures and only four described symptom relief measures. We were unable to make definitive evidence-based recommendations. We draw on the results to propose a framework for data collection in future observational studies of scabies outbreaks. While high-quality randomised controlled trials are needed to determine optimal drug treatment, evidence on environmental measures will need augmentation through other literature studies. The quality assessment tool designed is a useful resource for reporting of outcome measures including patient-reported measures in future outbreaks.

scholarly journals Development of Evidence-based Nursing Practice Guidelines for Peripheral Intravenous Catheter Management in Hospitalized Children and Adult

2017 ◽  
Vol 3 (1) ◽  
pp. 82 ◽  
Author(s):  
Young-Ju Kim ◽  
Sun-Mi Lee ◽  
Ho-Ran Park ◽  
Kyeong-Yae Sohng ◽  
Seok-Jung Kim
Keyword(s):  
Search Strategy ◽  
Assessment Tool ◽  
Developmental Process ◽  
Evidence Based ◽  
Intravenous Catheter ◽  
Peripheral Intravenous Catheter ◽  
Clinical Questions ◽  
Evidence Based Nursing ◽  
Intravenous Catheterization

Aims: The purpose of this study was to report on the development of evidence-based practice guidelines.Design: Developmental research for practice guidelines.Methods: The guidelines developmental process was designed according to a procedure provided by Scottish Intercollegiate Guidelines Network (SIGN). A first step, key clinical questions were selected. Next, 74 studies were selected from studies of 719 according to a search strategy, and then methodological quality of those studies was evaluated using assessment tool of SIGN. After the evaluation of draft guidelines including recommendations and their grades, the contents were modified. Last, definitive guidelines were evaluated using Appraisal of Guidelines for Research and Evaluation (AGREE) tool.Result: The guidelines consist of three categories and 64 recommendations, i.e. nine recommendations for user before peripheral intravenous catheterization (PIVC), 26 recommendations during PIVC, and 29 recommendations after PIVC. Content validity was revealed to 70-78% by experts’ agreement. Conclusion: These guidelines were completed throughout systematic reviews and evaluations by clinical experts. Their contents are also included about overall managements for PIV therapy. Therefore these guidelines could help PIV practitioners to make evidence based decision.Relevance to clinical practice: The method and result of this study are described specifically in figures, tables and appendix, which could give guidance to nurses who develop guidelines regarding other subjects.

scholarly journals Chatbots to Support People with Dementia and Their Caregivers: A Systematic Review of Functions and Quality (Preprint)

2020 ◽  
Author(s):  
Nicole Ruggiano ◽  
Ellen L. Brown ◽  
Lisa Roberts ◽  
C. Victoria Framil Suarez ◽  
Yan Luo ◽  
...  
Keyword(s):  
Information Technologies ◽  
Assessment Tool ◽  
Healthcare Delivery ◽  
Evidence Based ◽  
End User ◽  
The Public ◽  
Healthcare Settings ◽  
People With Dementia ◽  
Google Play

BACKGROUND Over the past decade, there has been an increase in the use of information technologies to educate and support persons with dementia and their family caregivers. At the same time, chatbot technologies have become increasingly popular for use by the public and have been identified as having benefits for healthcare delivery. However, little remains known about how chatbot technologies may benefit persons with dementia and their caregivers. OBJECTIVE Our aim was to identify the types of currently available chatbots that are designed for use by persons with dementia and their caregivers and to assess their quality of features and content. METHODS Chatbots were identified through a systematic search of Google Play, the IOS store, Alexa skills, and the web. An evidence-based assessment tool was used to evaluate the features and content of the identified apps. The assessment was conducted through inter-rater agreement among four separate reviewers. RESULTS Of the 505 initial chatbots identified, six were included in the review. The chatbots assessed varied significantly in content and scope. While chatbots were generally found to be easy to use, there were noted limitations to their performance and programmed content for dialogue. CONCLUSIONS While chatbot technologies have become well-established and commonly used by the public, their development for persons with dementia and their caregivers is in its infancy. Given the successful use of chatbots for other healthcare settings and applications, there are opportunities for integrating this technology into dementia care. However, more evidence-based chatbots that have undergone end-user evaluation are needed to evaluate their potential for adequately educating and supporting these populations.

scholarly journals Quality of top webpages providing abortion pill information for Google searches in the USA: An evidence-based webpage quality assessment

PLoS ONE ◽  
2021 ◽  
Vol 16 (1) ◽  
pp. e0240664
Author(s):  
Elizabeth Pleasants ◽  
Sylvia Guendelman ◽  
Karen Weidert ◽  
Ndola Prata
Keyword(s):  
Quality Assessment ◽  
Assessment Tool ◽  
Pregnancy Termination ◽  
The United States ◽  
Evidence Based ◽  
Online Information ◽  
Search Results ◽  
Link Type ◽  
Google Search

Background In the United States, the internet is widely used to seek health information. Despite an estimated 18 million Google searches on abortion per year and the demonstrated importance of the abortion pill as an option for pregnancy termination, the top webpage search results for abortion pill searches, as well as the content and quality of those webpages, are not well understood. Methods We used Google’s Custom Search Application Programming Interface (API) to identify the top 10 webpages presented for “abortion pill” searches on August 06, 2018. We developed a comprehensive, evidence-based Family Planning Webpage Quality Assessment Tool (FPWQAT), which was used to assess webpage quality for the five top webpages presenting text-based educational content. Results Of the top webpages for “abortion pill” searches, a plannedparenthood.com page was the top result and scored highest on our assessment (81%), providing high-quality and useable information. The other four webpages, a Wikipedia.com page and three anti-abortion information webpages, scored much lower on our assessment (14%-43%). These four webpages had lower quality of information in less useable formats. The anti-abortion pages also presented a variety of disinformation about the abortion pill. Conclusions Both the lack of accurate clinical content on the majority of top webpages and the concerning disinformation they contained raise concerns about the quality of online abortion pill information, while underlining challenges posed by Google search results to informed choice for consumers. Healthcare providers and consumers must be informed of online abortion pill content that is not based in current clinical evidence, while advocates and policymakers should push for online information that is credible and useable. These changes are imperative given the importance of sound abortion pill information for reproductive decision-making at a time when in-person abortion services are further challenged in the US.

scholarly journals ANALYSIS OF THE PERFORMANCE OF MCQs AS A PART OF FORMATIVE ASSESSMENT FOR IST MBBS STUDENTS IN BIOCHEMISTRY

2019 ◽  
Vol 3 (9) ◽  
Author(s):  
Suryakar Vrushali Prabhunath ◽  
Surekha T. Nemade ◽  
Ganesh D. Ghuge
Keyword(s):  
Formative Assessment ◽  
Assessment Tool ◽  
Item Analysis ◽  
Multiple Choice ◽  
Discrimination Index ◽  
Multiple Choice Questions ◽  
Test Items ◽  
Valid Test ◽  
Difficulty Index

Introduction: Multiple Choice Questions (MCQs) is one of the most preferred tool of assessment in medical education as a part of formative as well as summative assessment. MCQ performance as an assessment tool can be statistically analysed by Item analysis. Thus, aim of this study is to assess the quality of MCQs by item analysis and identify the valid test items to be included in the question bank for further use. Materials and methods: Formative assessment of Ist MBBS students was carried out with 40 MCQs as a part of internal examination in Biochemistry. Item analysis was done by calculating Difficulty index (P), Discrimination index (d) and number of Non- functional distractors. Results: Difficulty index (P) of 65% (26) items was well within acceptable range, 7.5% (3) items were too difficult whereas 27.5% (11) items were in the category of too easy. Discrimination Index (d) of 70% (28) items fell in recommended category whereas 10% (4) items were with acceptable, and 20% (8) were with poor Discrimination index. Out of 120 distractors 88.33% (106) were functional distractors and 11.66% (14) were non-functional distractors. After considering difficulty index, discrimination index and distractor effectiveness, 42.5% (17) items were found ideal to be included in the question bank. Conclusion: Item analysis remains an essential tool to be practiced regularly to improve the quality of the assessment methods as well as a tool for obtaining feedback for the instructors. Key Words: Difficulty index, Discrimination index, Item analysis, Multiple choice questions, Non-functional distractors

Sign in / Sign up

Export Citation Format

Share Document