Test Validation Analysis Based on Fuzzy Assessment

2012 ◽  
Vol 433-440 ◽  
pp. 2749-2754
Author(s):  
Tie Wang ◽  
Li Qing Wang ◽  
Cao Dai ◽  
Jing Shang
Keyword(s):  
Small Sample ◽  
Assessment Tools ◽  
Test Validation ◽  
Distribution Model ◽  
Fuzzy Evaluation ◽  
Student Culture ◽  
Downhill Simplex ◽  
Fuzzy Assessment ◽  
Examination Quality

In the student culture work, the examination has turned into a recognized assessment tools in the society and different professional certificate authority depend on the examination in many institutions. While the strength and weakness of the examination quality will affect the fairness objectivity and effectiveness of the test results. The study of index and evaluation of the examination is not only a scale that evaluate the effectiveness of the examination but also an important tool that improve the quality of the examination. This paper adopts standard establishment distribution model of employer by the use of extended fuzzy evaluation and will use downhill simplex optimization to establish distribution model results of the examination and calculate index by Aitken interpolation and composite trapezoidal integration method. When the multiple targets are evaluated, COSINE similarity need be used. This method can realize the test validation analysis of small sample.

scholarly journals Evaluation of the Effect of Items’ Format and Type on Psychometric Properties of Sixth Year Pharmacy Students Clinical Clerkship Assessment Tools

2019 ◽  
Author(s):  
Suha Al Muhaissen ◽  
Hatim S. AlKhatib ◽  
Gayle Brazeau ◽  
Amal Akour
Keyword(s):  
Assessment Tool ◽  
Academic Programs ◽  
Assessment Tools ◽  
Item Format ◽  
Traditional Assessment ◽  
Cognitive Levels ◽  
Examination Quality ◽  
Standardized Examinations ◽  
Case Based

Abstract Background Examination is the traditional assessment tool, and are used to guide the improvement of academic programs. Accreditation committees’ emphasis on the implementation of standardized examinations. The aims of the current study are to evaluate the quality of assessment tools of sixth year PharmD students as a function of assessment item format and type/structure and to assess the effect of the number of response choices on the characteristics of MCQs as assessment items. Methods. A total of 173 assessment items used in the examinations of sixth year rotations of PharmD program were included. Items were classified as case based or non-case based and as MCQs or open-ended. The psychometric characteristics of examinations were studied as a function of the level of the Bloom’s levels addressed by an item, item format, and number of choices in MCQs. Results. Items addressing intellectual and analysis skills were more difficult, while items associated with multiple cognitive levels were more discriminative. No differences were found between case based and noncase based items in terms of their difficulty, with a slightly better discrimination in the letter. Open-ended items were easier, yet more discriminative. MCQs with higher number of options were easier and more discriminative. Open-ended questions were significantly easier and more discriminative in comparison to MCQs as case based items while they were more difficult and more discriminative as non-case based items. Conclusion. Item formats, structure, and number of options in MCQs, affected students’ performance and overall examination quality. The discrimination of items associated with multiple Bloom’s levels was significantly higher than those associated with a single level. Noncase based items and open-ended items were easier and more discriminative than case based items and MCQs, respectively.

Assessing Medical Evidence

2018 ◽  
Author(s):  
Jacob Stegenga
Keyword(s):  
Quality Assessment ◽  
Randomized Trials ◽  
Empirical Studies ◽  
Allocation Concealment ◽  
Assessment Tools ◽  
Medical Evidence ◽  
Empirical Methods ◽  
First Order ◽  
First Order Methods

Medical scientists employ ‘quality assessment tools’ to assess evidence from medical research, especially from randomized trials. These tools are designed to take into account methodological details of studies, including randomization, subject allocation concealment, and other features of studies deemed relevant to minimizing bias. There are dozens of such tools available. They differ widely from each other, and empirical studies show that they have low inter-rater reliability and low inter-tool reliability. This is an instance of a more general problem called here the underdetermination of evidential significance. Disagreements about the quality of evidence can be due to different—but in principle equally good—weightings of the methodological features that constitute quality assessment tools. Thus, the malleability of empirical research in medicine is deep: in addition to the malleability of first-order empirical methods, such as randomized trials, there is malleability in the tools used to evaluate first-order methods.

Evaluating Qualitative Research

2017 ◽  
Author(s):  
Jeasik Cho
Keyword(s):  
Qualitative Research ◽  
Large Scale ◽  
Academic Research ◽  
Research Community ◽  
Assessment Tools ◽  
Journal Articles ◽  
Journal Editors ◽  
The Past ◽  
Grant Proposals

This book provides the qualitative research community with some insight on how to evaluate the quality of qualitative research. This topic has gained little attention during the past few decades. We, qualitative researchers, read journal articles, serve on masters’ and doctoral committees, and also make decisions on whether conference proposals, manuscripts, or large-scale grant proposals should be accepted or rejected. It is assumed that various perspectives or criteria, depending on various paradigms, theories, or fields of discipline, have been used in assessing the quality of qualitative research. Nonetheless, until now, no textbook has been specifically devoted to exploring theories, practices, and reflections associated with the evaluation of qualitative research. This book constructs a typology of evaluating qualitative research, examines actual information from websites and qualitative journal editors, and reflects on some challenges that are currently encountered by the qualitative research community. Many different kinds of journals’ review guidelines and available assessment tools are collected and analyzed. Consequently, core criteria that stand out among these evaluation tools are presented. Readers are invited to join the author to confidently proclaim: “Fortunately, there are commonly agreed, bold standards for evaluating the goodness of qualitative research in the academic research community. These standards are a part of what is generally called ‘scientific research.’ ”

scholarly journals Evaluation of the efficacy of hearing aids in older adults: a multiparametric longitudinal study protocol

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Domenico Cuda ◽  
Sara Ghiselli ◽  
Alessandra Murri
Keyword(s):  
Quality Of Life ◽  
Older Adults ◽  
Speech Perception ◽  
Hearing Aids ◽  
Assessment Tools ◽  
Elderly Subjects ◽  
Single Subject ◽  
Measurement Tools ◽  
First Time

Abstract Background Prevalence of hearing loss increases with age. Its estimated prevalence is 40–50 % in people over 75 years of age. Recent studies agree that declinein hearing threshold contribute to deterioration in sociality, sensitivity, cognition, and quality of life for elderly subjects. The aim of the study presented in this paper is to verify whether or not rehabilitation using first time applied Hearing Aids (HA) in a cohort of old people with hearing impairment improves both speech perception in a noisy environment over time and the overall health-related quality of life. Methods The monocentric, prospective, repeated measurements, single-subject, clinical observational study is to recruit 100 older adults, first-time HA recipients (≥ 65 years).The evaluation protocol is designed to analyze changes in specific measurement tools a year after the first HA usage in comparison with the evaluation before HA fitting. Evaluations will consist of multiparametric details collected through self-report questionnaires completed by the recipients and a series of commonly used audiometric measures and geriatric assessment tools. The primary indicator of changes in speech perception in noise to be used is the Italian version of Oldenburg Satz (OLSA) test whereas the indicator of changes in overall quality of life will be the Assessment of Quality of Life (AQoL) and Hearing Handicap Inventory for the Elderly (HHIE) questionnaires. The Montreal Cognitive Assessment (MoCA) will help in screening the cognitive state of the subjects. Discussion The protocol is designed to make use of measurement tools that have already been applied to the hearing-impaired population in order to compare the effects of HA rehabilitation in the older adults immediately before first HA usage (Pre) and after 1 year of experience (Post). This broad approach will lead to a greater understanding of how useful hearing influences the quality of life in older individuals, and therefore improves potentials for healthy aging. The data is to be analyzed by using an intrasubject endpoint comparison. Outcomes will be described and analyzed in detail. Trial registration This research was retrospectively registered underno. NCT04333043at ClinicalTrials.gov (http://www.clinicaltrials.gov/) on the 26 March 2020. This research has been registered with the Ethics Committee of the Area Vasta Emilia Nord under number 104, date of approval 17/07/2017.

scholarly journals An AI-Based System for Formative and Summative Assessment in Data Science Courses

2020 ◽  
Author(s):  
Pierpaolo Vittorini ◽  
Stefano Menini ◽  
Sara Tonelli
Keyword(s):  
Data Science ◽  
Ad Hoc ◽  
Online Courses ◽  
Assessment Tools ◽  
Science Courses ◽  
Summative Assessments ◽  
Assessment Activity ◽  
Automated Grading ◽  
Automated Feedback

AbstractMassive open online courses (MOOCs) provide hundreds of students with teaching materials, assessment tools, and collaborative instruments. The assessment activity, in particular, is demanding in terms of both time and effort; thus, the use of artificial intelligence can be useful to address and reduce the time and effort required. This paper reports on a system and related experiments finalised to improve both the performance and quality of formative and summative assessments in specific data science courses. The system is developed to automatically grade assignments composed of R commands commented with short sentences written in natural language. In our opinion, the use of the system can (i) shorten the correction times and reduce the possibility of errors and (ii) support the students while solving the exercises assigned during the course through automated feedback. To investigate these aims, an ad-hoc experiment was conducted in three courses containing the specific topic of statistical analysis of health data. Our evaluation demonstrated that automated grading has an acceptable correlation with human grading. Furthermore, the students who used the tool did not report usability issues, and those that used it for more than half of the exercises obtained (on average) higher grades in the exam. Finally, the use of the system reduced the correction time and assisted the professor in identifying correction errors.

scholarly journals The effects of intensive home treatment on self-efficacy in patients recovering from a psychiatric crisis

2021 ◽  
Vol 15 (1) ◽  
Author(s):  
Ansam Barakat ◽  
Matthijs Blankers ◽  
Jurgen E Cornelis ◽  
Nick M Lommerse ◽  
Aartjan T F Beekman ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Mental Disorders ◽  
Self Efficacy ◽  
Psychotic Disorders ◽  
Rating Scale ◽  
Small Sample ◽  
Home Treatment ◽  
Differential Effects ◽  
Psychiatric Crisis

Abstract Background This study evaluated whether providing intensive home treatment (IHT) to patients experiencing a psychiatric crisis has more effect on self-efficacy when compared to care as usual (CAU). Self-efficacy is a psychological concept closely related to one of the aims of IHT. Additionally, differential effects on self-efficacy among patients with different mental disorders and associations between self-efficacy and symptomatic recovery or quality of life were examined. Methods Data stem from a Zelen double consent randomised controlled trial (RCT), which assesses the effects of IHT compared to CAU on patients who experienced a psychiatric crisis. Data were collected at baseline, 6 and 26 weeks follow-up. Self-efficacy was measured using the Mental Health Confidence Scale. The 5-dimensional EuroQol instrument and the Brief Psychiatric Rating Scale (BPRS) were used to measure quality of life and symptomatic recovery, respectively. We used linear mixed modelling to estimate the associations with self-efficacy. Results Data of 142 participants were used. Overall, no difference between IHT and CAU was found with respect to self-efficacy (B = − 0.08, SE = 0.15, p = 0.57), and self-efficacy did not change over the period of 26 weeks (B = − 0.01, SE = 0.12, t (103.95) = − 0.06, p = 0.95). However, differential effects on self-efficacy over time were found for patients with different mental disorders (F(8, 219.33) = 3.75, p < 0.001). Additionally, self-efficacy was strongly associated with symptomatic recovery (total BPRS B = − 0.10, SE = 0.02, p < 0.00) and quality of life (B = 0.14, SE = 0.01, p < 0.001). Conclusions Although self-efficacy was associated with symptomatic recovery and quality of life, IHT does not have a supplementary effect on self-efficacy when compared to CAU. This result raises the question whether, and how, crisis care could be adapted to enhance self-efficacy, keeping in mind the development of self-efficacy in depressive, bipolar, personality, and schizophrenia spectrum and other psychotic disorders. The findings should be considered with some caution. This study lacked sufficient power to test small changes in self-efficacy and some mental disorders had a small sample size. Trial registration This trial is registered at Trialregister.nl, number NL6020.

scholarly journals Fuzzy Evaluation Model of Process Improvement Capability with Costs Consideration

2021 ◽  
Vol 11 (10) ◽  
pp. 4344
Author(s):  
Kuen-Suan Chen ◽  
Shui-Chuan Chen ◽  
Ting-Hsin Hsu ◽  
Min-Yi Lin ◽  
Chih-Feng Wu
Keyword(s):  
Process Improvement ◽  
Quality Evaluation ◽  
Evaluation Model ◽  
Fuzzy Evaluation ◽  
Sampling Errors ◽  
Capability Index ◽  
Testing Method ◽  
Improvement Potential ◽  
Testing Accuracy

The Taguchi capability index, which reflects the expected loss and the yield of a process, is a useful index for evaluating the quality of a process. Several scholars have proposed a process improvement capability index based on the expected value of the Taguchi loss function as well as the corresponding cost of process improvement. There have been a number of studies using the Taguchi capability index to develop suppliers’ process quality evaluation models, whereas models for evaluating suppliers’ process improvement potential have been relatively lacking. Thus, this study applies the process improvement capability index to develop an evaluation model of the supplier’s process improvement capability, which can be provided to the industry for application. Besides, owing to the current need to respond quickly, coupled with cost considerations and the limits of technical capabilities, the sample size for sampling testing is usually not large. Consequently, the evaluation model of the process improvement capability developed in this study adopts a fuzzy testing method based on the confidence interval. This method reduces the risk of misjudgment due to sampling errors and improves the testing accuracy because it can incorporate experts and their accumulated experiences.

scholarly journals An assessment of the Armenian national midwifery curriculum as per international standards

2020 ◽  
Vol 30 (Supplement_5) ◽  
Author(s):  
L Aslanyan ◽  
S Sahakyan ◽  
V Petrosyan
Keyword(s):  
Poor Quality ◽  
International Standards ◽  
Assessment Tools ◽  
Global Competency ◽  
Midwifery Education ◽  
Study Team ◽  
Competency Based ◽  
Document Review ◽  
Provision Of Care

Abstract Background Literature suggests increasing evidence that incompetent midwifery education results in poor quality of care. The study compared and evaluated the level of compliance of the Armenian national criterion on midwifery with the global competency standards and developed recommendations for improvement. Methods The study team conducted a document review using a standardized checklist, which was adapted from the International Confederation of Midwives (ICM) core assessment tools. The document review explored if the main knowledge and skills/ability related elements of each ICM competency were separately covered by different modules of the national criterion using a scoring system with categories: not met, partially met, fully met and unspecified. Results Although the national criterion did not have specifically defined competencies, most of the elements required by the ICM were present as specific learning outcomes under different modules of the criterion. Midwifery program curriculum was described as intensive with unnecessarily heavy workload. Additionally, the document review reviled that most of the learning outcome defined in modules of the national criterion focused more on theoretical knowledge rather than practical skills and abilities. Overall, the ICM required competency in provision of care during pregnancy was the most comprehensively covered one in the national criterion, while the competency in facilitation of abortion related care was the lowest covered. National criterion did not highlight the importance of topics such as women's rights and health, principles of epidemiology, statistical methods of research, cultural, local and ethical beliefs. Conclusions The systematic comparison of the national criterion with internationally recognized essential competencies demonstrates significant gaps. The study team recommends revisions to the national criterion to make it competency based. Key messages Midwifery education criteria in Armenia did not reflect internationally accepted midwifery competency’s fundamental philosophy and values. Given the shortcomings of the current national midwifery education criterion, a comprehensive revision of the competencies of midwives in Armenia should be considered.

scholarly journals Quality of life in primary sclerosing cholangitis: a systematic review

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Elena Marcus ◽  
Paddy Stone ◽  
Anna-Maria Krooupa ◽  
Douglas Thorburn ◽  
Bella Vivat
Keyword(s):  
Quality Of Life ◽  
Systematic Review ◽  
Clinical Research ◽  
Disease Progression ◽  
Primary Sclerosing Cholangitis ◽  
Sclerosing Cholangitis ◽  
Small Sample ◽  
Limited Evidence ◽  
Research Studies

Abstract Background Primary sclerosing cholangitis (PSC) is a rare bile duct and liver disease which can considerably impact quality of life (QoL). As part of a project developing a measure of QoL for people with PSC, we conducted a systematic review with four review questions. The first of these questions overlaps with a recently published systematic review, so this paper reports on the last three of our initial four questions: (A) How does QoL in PSC compare with other groups?, (B) Which attributes/factors are associated with impaired QoL in PSC?, (C) Which interventions are effective in improving QoL in people with PSC?. Methods We systematically searched five databases from inception to 1 November 2020 and assessed the methodological quality of included studies using standard checklists. Results We identified 28 studies: 17 for (A), ten for (B), and nine for (C). Limited evidence was found for all review questions, with few studies included in each comparison, and small sample sizes. The limited evidence available indicated poorer QoL for people with PSC compared with healthy controls, but findings were mixed for comparisons with the general population. QoL outcomes in PSC were comparable to other chronic conditions. Itch, pain, jaundice, severity of inflammatory bowel disease, liver cirrhosis, and large-duct PSC were all associated with impaired QoL. No associations were found between QoL and PSC severity measured with surrogate markers of disease progression or one of three prognostic scoring systems. No interventions were found to improve QoL outcomes. Conclusion The limited findings from included studies suggest that markers of disease progression used in clinical trials may not reflect the experiences of people with PSC. This highlights the importance for clinical research studies to assess QoL alongside clinical and laboratory-based outcomes. A valid and responsive PSC-specific measure of QoL, to adequately capture all issues of importance to people with PSC, would therefore be helpful for clinical research studies.

Sign in / Sign up

Export Citation Format

Share Document