scholarly journals ITEM ANALYSIS OF READING COMPREHENSION TEST FOR POST-GRADUATE STUDENTS

2018 ◽  
Vol 7 (1) ◽  
pp. 29
Author(s):  
Ari Arifin Danuwijaya
Keyword(s):  
Reading Comprehension ◽  
Item Difficulty ◽  
Item Analysis ◽  
Classical Test Theory ◽  
Test Development ◽  
Test Theory ◽  
Difficulty Level ◽  
Classical Test ◽  
Comprehension Test

Developing a test is a complex and reiterative process which subject to revision even if the items were developed by skilful item writers. Many commercial test publishers need to conduct test analysis, rather than trusting the item writers� judgement and skills to improve the quality of items that need to be proven statistically after trying out was performed. This study is a part of test development process which aims to analyse the reading comprehension test items. One hundred multiple choice questions were pilot tested to 50 postgraduate students in one university. The pilot testing was aimed to investigate item quality which can further be developed better. The responses were then analysed using Classical Test Theory and using psychometric software called Lertap. The results showed that item difficulty level was mostly average. In terms of item discrimination, more than half of the total items were categorized marginal which required further modifications. This study suggests some recommendation that can be useful to improve the quality of the developed items.��Keywords: reading comprehension; item analysis; classical test theory; item difficulty; test development.

scholarly journals Summative Test Items Analysis Using Classical Test Theory (CTT)/ Analisis Item Kertas Peperiksaan Sumatif Menggunakan Teori Ujian Klasik (TUK)

2020 ◽  
Vol 12 (2-2) ◽  
Author(s):  
Nor Aisyah Saat
Keyword(s):  
Item Difficulty ◽  
Item Analysis ◽  
Classical Test Theory ◽  
Test Theory ◽  
Difficulty Level ◽  
Test Items ◽  
Classical Test ◽  
Difficulty Index ◽  
The Given ◽  
Examination Question

Item analysis is the process of examining student responses to test items individually in order to get clear picture on the quality of the item and the overall test. Teachers are encouraged to perform item analysis for each administered test in order to determine which items should be retained, modified, or discarded in the given test. This study aims to analyse items in 2 summative examination question papers by using classical test theory (CTT). The instruments used were the SPM Mathematics Trial Examination Questions 1 2019 which involved 50 students in form 5 students and the SPM Mathematics Trial Examination Question 1 2019 which involved 20 students. The SPM Mathematics Trial Examination Question paper 1 contains 40 objective questions while the SPM Mathematics Trial Examination paper 1 contains 25 subjective questions. The data obtained were analysed using Microsoft Excel software based on the formulas of item difficulty index and discrimination index. This analysis can help teachers for better understanding about the difficulty level of the items used. Finally, based on the analysis items obtained, the items were classified as good, good but improved, marginal or weak items.

scholarly journals The Role of Classical Test Theory to Determine the Quality of Classroom Teaching Test Items

2020 ◽  
Vol 9 (1) ◽  
pp. 5-34
Author(s):  
Wong Vincent ◽  
S.Kanageswari Suppiah Shanmugam
Keyword(s):  
Item Difficulty ◽  
Classical Test Theory ◽  
Test Theory ◽  
Difficulty Level ◽  
Item Discrimination ◽  
Test Items ◽  
Classical Test ◽  
English Competence

The purpose of this study is to describe the use of Classical Test Theory (CTT) to investigate the quality of test items in measuring students' English competence. This study adopts a research method with a mixed methods approach. The results show that most items are within acceptable range of both indexes, with the exception of items in synonyms. Items that focus on vocabulary are more challenging. What is surprising is that the short answer items have an excellent item difficulty level and item discrimination index. General results from data analysis of items also support the hypothesis that items that have an ideal item difficulty value between 0.4 and 0.6 will have the same ideal item discrimination value. This paper reports part of a larger study on the quality of individual test items and overall tests.

scholarly journals Analisis Butir Soal IPA Try Out USBN Tahun Ajaran 2018/2019 dalam Kaitannya dengan Level Kognitif

MADRASAH ◽  
2020 ◽  
Vol 12 (1) ◽  
pp. 29-39
Author(s):  
Nuril Huda ◽  
Tutik Sri Wahyuni
Keyword(s):  
Classical Test Theory ◽  
Answer Sheet ◽  
Test Theory ◽  
Difficulty Level ◽  
Cognitive Level ◽  
Descriptive Research ◽  
Classical Test ◽  
Discriminating Power ◽  
Academic Year

This research aims to: 1) find out the characteristics of the science items try out National Standar School Exams (USBN) in the academic year 2018/2019 based on Classical Test Theory (CTT); 2) find out the number of the science items try out USBN in the academic year 2018/2019 in relation to cognitive level. This type of research is a descriptive research with a quantitative approach. The data obtained was a computer answer sheet of 5022 students who took USBN try out of Elementary School 2019 on February 21, 2019 in Tulungagung Regency. The results showed that: 1) The characteristics of the science items try out USBN in the academic year 2018/2019 based on Classical Test Theory (CTT) in aspects of: a) validity of 35 items valid; b) the reliability value of 0.818 is very high; c) the level of difficulty level: 4 items (11.43%) are difficult, 9 items (25.71%) are moderate, 16 items (45.71%) are easy and 6 items (7.140%) are very easy; d) discriminating power: 3 items (8.57%) are bad, 12 items (34.29%) are good enough, 15 items (42.86%) are moderate, and 5 items (14.29%) are good; e) the quality of options: 17 items (48.57%) without revision, 9 items (25.71%) one option revision, 5 items (14.29%) 2 option revisions, and 4 items (11.43% ) wrong revision of 3 options; f) 13 items (37.14%) about the science try out USBN in the academic year 2018/2019 have quite good and good characteristics, so they can be included in the question bank; 2) items the science of  try out USBN in the academic year 2018/2019 in relationship with cognitive level, 11 items (31.43%) category L1 (knowledge), 10 items (28.57%) category L1 (understanding), 4 items (11.43%) category L2 (application), and 10 items (28.57%) category L3 (reasoning). Of the 13 items entered in the question bank with cognitive level, the science try out USBN in the academic year 2018/2019 was dominated at the cognitive level L1 (knowledge and understanding).

scholarly journals EXAMINING THE QUALITY OF ENGLISH TEST ITEMS USING PSYCHOMETRIC AND LINGUISTIC CHARACTERISTICS AMONG GRADE SIX PUPILS

2020 ◽  
Vol 17 (Number 2) ◽  
pp. 63-101
Author(s):  
S. Kanageswari Suppiah Shanmugam ◽  
Vincent Wong ◽  
Murugan Rajoo
Keyword(s):  
Item Difficulty ◽  
Classroom Teacher ◽  
Item Analysis ◽  
Classical Test Theory ◽  
Test Theory ◽  
Psychometric Analysis ◽  
Cognitive Interviews ◽  
Test Items ◽  
Item Quality

Purpose - This study examined the quality of English test items using psychometric and linguistic characteristics among Grade Six pupils. Method - Contrary to the conventional approach of relying only on statistics when investigating item quality, this study adopted a mixed-method approach by employing psychometric analysis and cognitive interviews. The former was conducted on 30 Grade Six pupils, with each item representing a different construct commonly found in English test papers. Qualitative input was obtained through cognitive interviews with five Grade Six pupils and expert judgements from three teachers. Findings - None of the items were found to be too easy or difficult, and all items had positive discrimination indices. The item on idioms was most ideal in terms of difficulty and discrimination. Difficult items were found to be vocabulary-based. Surprisingly, the higher-order-thinking subjective items proved to be excellent in difficulty, although improvements could be made on their ability to discriminate. The qualitative expert judgements agreed with the quantitative psychometric analysis. Certain results from the item analysis, however, contradicted past findings that items with the ideal item difficulty value between 0.4 and 0.6 would have equally ideal item discrimination index. Significance -The findings of the study can serve as a reminder on the significance of using Classical Test Theory, a non-complex psychometric approach in assisting classroom teacher practitioners during the meticulous process of test design and ensuring test item quality.

scholarly journals The transition to digital presentation of the diagnostic imaging domain of the Part IV examination of the National Board of Chiropractic Examiners

2020 ◽  
Vol 34 (1) ◽  
pp. 52-67 ◽  
Author(s):  
Igor Himelfarb ◽  
Margaret A. Seron ◽  
John K. Hyland ◽  
Andrew R. Gow ◽  
Nai-En Tang ◽  
...  
Keyword(s):  
Diagnostic Imaging ◽  
Item Difficulty ◽  
Item Analysis ◽  
Classical Test Theory ◽  
Test Theory ◽  
High Stakes ◽  
Radiographic Images ◽  
National Board ◽  
Classical Statistics ◽  
Chiropractic Students

Objective: This article introduces changes made to the diagnostic imaging (DIM) domain of the Part IV of the National Board of Chiropractic Examiners examination and evaluates the effects of these changes in terms of item functioning and examinee performance. Methods: To evaluate item function, classical test theory and item response theory (IRT) methods were employed. Classical statistics were used for the assessment of item difficulty and the relation to the total test score. Item difficulties along with item discrimination were calculated using IRT. We also studied the decision accuracy of the redesigned DIM domain. Results: The diagnostic item analysis revealed similarity in item function across test forms and across administrations. The IRT models found a reasonable fit to the data. The averages of the IRT parameters were similar across test forms and across administrations. The classification of test takers into ability (theta) categories was consistent across groups (both norming and all examinees), across all test forms, and across administrations. Conclusion: This research signifies a first step in the evaluation of the transition to digital DIM high-stakes assessments. We hope that this study will spur further research into evaluations of the ability to interpret radiographic images. In addition, we hope that the results prove to be useful for chiropractic faculty, chiropractic students, and the users of Part IV scores.

scholarly journals Lessons Efficiency Paradox: The Unexpected Empirical Findings about the Amount of Homework and the Teacher’s Strictness

Pedagogika ◽  
2017 ◽  
Vol 127 (3) ◽  
pp. 104-118
Author(s):  
Gediminas Merkys ◽  
Daiva Bubelienė
Keyword(s):  
Social Relations ◽  
Classical Test Theory ◽  
Educational Process ◽  
Test Theory ◽  
Dimensional Structure ◽  
Classical Test ◽  
Integrated Index ◽  
Different Types ◽  
Metrological Quality

In the article a newly created questionnaire intended for older schoolchildren – “evaluate the teacher and his lessons” is introduced. The theoretical and practical context of the instrument based on 87 primary questions is named, the dimensional structure and metrological quality of the formed integrated scales and sub-scales is presented. The scales and sub-scales were formed following the classical test theory, combining logical and factorial validation. The secondary sub-scale factorization has indicated that it is expedient to distinguish between two integrated lesson dimensions (scales). The first integrated scale reflects the quality of social relations and teacher-centered orientation. The second scale reflects the management and didactics of the educational process. High correlation between the evaluations of integrated scales (r = 0.86) indicates that a generalized integrated index of evaluation of the teacher and his lesson can be derived by aggregating even 81 primary variables defining the most various aspects of the lesson. In the article the basis of statistic norming of the questionnaire possessed at present is described: Nschool children = 4024 and Nteachers = 200 which encompasses schools of different types from various regions of the country. The wide coverage of the content of created questionnaire, quite good quality of the scales opens good opportunities for its application in both the practice of schools evaluation and research. First of all, the methodical purpose of the article has been to introduce a new standardized instrument of survey. Secondly, the question why such indicators as “abundance of homework” and “level of the requirements set by the teacher” practically do not correlate with all the remaining scales, although the latter intercorrelate very significantly, is set. In the paper the question (and hypotheses) whether the mentioned variables can truly affect the didactic quality of the lesson counterproductively is elaborated.

scholarly journals Karakteristik Butir Soal Penilaian Akhir Semester Mata Pelajaran Sejarah Kelas XI SMA Negeri 1 Pangkalpinang

2021 ◽  
Vol 5 (2) ◽  
pp. 210-221
Author(s):  
Anis Faridah
Keyword(s):  
Social Sciences ◽  
Classical Test Theory ◽  
Reliability Coefficient ◽  
Test Theory ◽  
Difficulty Level ◽  
Theory Approach ◽  
Final Exam ◽  
Classical Test ◽  
The Subject ◽  
Quantitative Descriptive

This research is a study of quantitative descriptive. The purpose of this research is to describe the characteristics of final semester exam items for grade XI in the History subject at SMA Negeri 1 Pangkalpinang using the classical test theory approach. The research of the subject was 138 students of class XI in Social Sciences Major. The result of the research shows that final exam questions in the history subject class XI of SMA Negeri 1 Pangkalpinang are proper to use. This shows that from the validity of the items which there are 39 items of questions (97.5%) which are proven empirically valid with a 0.818 reliability coefficient. Other than that, there are 27 items of questions (67,5%) that can fulfill the criteria for the difficulty level, distinguishing power, and distractor function so it can be used directly to measure the student's ability without correction. While 12 items of questions (30%) need to be fixed and 1 item of question (2,5%) is declared to be invalid so it can't be used to measure the student's ability in History Subject. Permasalahan yang melatarbelakangi penelitian ini adalah pengembangan soal penilaian akhir semester mata pelajaran sejarah yang tidak melalui tahapan analisis butir soal sehingga kualitas butir soal tidak diketahui. Penelitian ini merupakan penelitian deskriptif kuantitatif. Tujuan penelitian ini adalah untuk mendeskripsikan karakteristik butir soal penilaian akhir semester mata pelajaran sejarah kelas XI SMA Negeri 1 Pangkalpinang menggunakan pendekatan teori tes klasik. Subjek penelitian berjumlah 138 peserta didik kelas XI jurusan IPS. Hasil penelitian menunjukkan bahwa soal PAS mata pelajaran sejarah kelas XI SMA Negeri 1 Pangkalpinang telah layak digunakan. Hal ini dibuktikan dari validitas butir soal yang mana terdapat 39 butir soal (97,5%) terbukti valid secara empirik dengan koefisien reliabilitas sebesar 0,818. Selain itu terdapat 27 butir soal (67,5%) yang memenuhi kriteria tingkat kesukaran, daya beda, dan keberfungsian distraktor sehingga dapat digunakan langsung untuk mengukur kemampuan peserta didik tanpa perbaikan. Sedangkan sebanyak 12 butir soal (30%) perlu dilakukan perbaikan dan 1 butir soal (2,5%) dinyatakan gugur sehingga tidak dapat digunakan untuk mengukur kemampuan peserta didik pada mata pelajaran sejarah.

scholarly journals Applicability of Item Response Theory to the Korean Nurses' Licensing Examination

2005 ◽  
Vol 2 ◽  
pp. 23 ◽  
Author(s):  
Geum-Hee Jeong ◽  
Mi Kyoung Yim
Keyword(s):  
Rasch Model ◽  
Logistic Model ◽  
Goodness Of Fit ◽  
Item Analysis ◽  
Classical Test Theory ◽  
Test Theory ◽  
Classical Test ◽  
Licensing Examination ◽  
The Rasch Model ◽  
Two Parameter

To test the applicability of item response theory (IRT) to the Korean Nurses' Licensing Examination (KNLE), item analysis was performed after testing the unidimensionality and goodness-of-fit. The results were compared with those based on classical test theory. The results of the 330-item KNLE administered to 12,024 examinees in January 2004 were analyzed. Unidimensionality was tested using DETECT and the goodness-of-fit was tested using WINSTEPS for the Rasch model and Bilog-MG for the two-parameter logistic model. Item analysis and ability estimation were done using WINSTEPS. Using DETECT, Dmax ranged from 0.1 to 0.23 for each subject. The mean square value of the infit and outfit values of all items using WINSTEPS ranged from 0.1 to 1.5, except for one item in pediatric nursing, which scored 1.53. Of the 330 items, 218 (42.7%) were misfit using the two-parameter logistic model of Bilog-MG. The correlation coefficients between the difficulty parameter using the Rasch model and the difficulty index from classical test theory ranged from 0.9039 to 0.9699. The correlation between the ability parameter using the Rasch model and the total score from classical test theory ranged from 0.9776 to 0.9984. Therefore, the results of the KNLE fit unidimensionality and goodness-of-fit for the Rasch model. The KNLE should be a good sample for analysis according to the IRT Rasch model, so further research using IRT is possible.

Conditional Reasoning: An Integrated Approach to Item Analysis

2019 ◽  
Vol 23 (1) ◽  
pp. 124-153 ◽  
Author(s):  
Daniel R. Smith ◽  
Michael E. Hoffman ◽  
James M. LeBreton
Keyword(s):  
Item Analysis ◽  
Classical Test Theory ◽  
Integrated Approach ◽  
Conditional Reasoning ◽  
Test Theory ◽  
Analytic Framework ◽  
Test Items ◽  
Classical Test ◽  
Reasoning Test ◽  
Unique Nature

This article provides a review of the approach that James used when conducting item analyses on his conditional reasoning test items. That approach was anchored in classical test theory. Our article extends this work in two important ways. First, we offer a set of test development protocols that are tailored to the unique nature of conditional reasoning tests. Second, we further extend James’s approach by integrating his early test validation protocols (based on classical test theory) with more recent protocols (based on item response theory). We then apply our integrated item analytic framework to data collected on James’s first test, the conditional reasoning test for relative motive strength. We illustrate how this integrated approach furnishes additional diagnostic information that may allow researchers to make more informed and targeted revisions to an initial set of items.

The Validation Study of Both the Modified Barthel and Barthel Index, and Their Comparison Based on Rasch Analysis in the Hospitalized Acute Stroke Elderly

2020 ◽  
pp. 009141502098177
Author(s):  
Reyhaneh Aminalroaya ◽  
Fatemeh Sadat Mirzadeh ◽  
Kazem Heidari ◽  
Mahtab Alizadeh-Khoei ◽  
Farshad Sharifi ◽  
...  
Keyword(s):  
Acute Stroke ◽  
Validation Study ◽  
Barthel Index ◽  
Rasch Analysis ◽  
Item Difficulty ◽  
Classical Test Theory ◽  
Test Theory ◽  
Stair Climbing ◽  
Theory Approach ◽  
Classical Test

A validation study the Iranian Modified Barthel Index (MBI) in hospitalized acute stroke elderly by classical test theory approach and investigate Rasch analysis for both Iranian version MBI and BI and compare the hierarchical item difficulty of them. Face-to-face interview with 100 geriatric stroke inpatients 60+ or their caregivers was done in a cross-sectional study. First, construct validity of MBI analyzed by the classical test theory, then Rasch analysis were done for BI and MBI. The reliability of the Iranian MBI was significant at 0.955. One factor achieved by the variance of 83.2%. In Rasch analysis for MBI, the most difficult item was stair climbing, whereas the simplest items were bowel and bladder control. In BI, the most difficult items were toilet use and ambulation. The Iranian MBI is very accurate and reliable; therefore the use of MBI to measure better outcomes in stroke elderly inpatients is recommended comparing with BI.

Sign in / Sign up

Export Citation Format

Share Document