scholarly journals Psychometric Analysis of Economics Achievement Test Using Item Response Theory

2020 ◽  
Vol 10 (2) ◽  
pp. 59
Author(s):  
Roseline Ifeoma Ezechukwu ◽  
Basil Chinecherem, E. Oguguo ◽  
Catherine U. Ene ◽  
Clifford O. Ugorji
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Research Question ◽  
Achievement Test ◽  
Choice Test ◽  
Response Theory ◽  
Test Items ◽  
Significant Difference ◽  
Research Questions ◽  
Two Parameter

This study determined the psychometric properties of the Economics Achievement Test (EAT) using Item Response Theory (IRT). Two popular IRT models namely, one-parameter logistics (1PL) and two-parameter logistics (2PL) models were utilized. The researcher adopted instrumentation research design. Four research questions and two hypotheses were formulated to guide the study. The population size is five thousand, three hundred and sixty-two (5,362) from thirty-seven (37) schools. The sample for the study was 1,180 senior secondary school students (SSS3) drawn using multi-stage sampling procedure. The 1,180 students were stratified according to gender which resulted to 885 females and 295 males. The instrument for the study consisted of 50 multiple-choice test items on the economics achievement test, developed by the researchers. Reliability and validity for each item and for the whole test were established according to the one-parameter and two-parameter logistic models. Research question one was answered using 1PLM, while research questions two and three were answered using 2PLM IRT model. Hypothesis one was tested using t-test analysis of difference between the difficulty parameters estimated using 1PLM and 2PLM while hypothesis two was tested using Chi-square. The finding of the study revealed significant difference between the item difficulties estimated using 1PLM and 2PLM. Also the observed scores of the testees on the test items fit the 1PL 2PL models.

scholarly journals Implementation of Item Response Theory for Analysis of Test Items Quality and Students’ Ability in Chemistry

2017 ◽  
Vol 2 (1) ◽  
pp. 1 ◽  
Author(s):  
Rizki Nor Amelia ◽  
Kriswantoro Kriswantoro
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Repeated Measure ◽  
Choice Test ◽  
Response Theory ◽  
First Semester ◽  
Test Items ◽  
Descriptive Research ◽  
Measurement Tools ◽  
The Mean

<p>This first aim of this study is to describe the quality of chemistry test item made by teacher. The test was developed for 11<sup>th</sup> grade students’ science class in the first semester on academic year 2015/2016. The second aim of this study is to describe the characteristic of measurement’s result for students’ ability in chemistry. This is descriptive research design with the 101 student’s responses patterns from multiple choice test device with 5 answer alternatives. The responses patterns were collected by documentation technique and analyzed quantitatively using Item Response Theory software such as BILOG MG V3.0 with 1-PL, 2-PL, and 3-PL models. The differences of students’ ability in chemistry in model 1-PL, 2-PL, dan 3-PL were analyzed using One-Way Anova Repeated Measure. The result showed that the mean of item difficulties level (b), item differentiate (a), and pseudo-guessing (c) are good. The measurement tools arranged by teacher were suitable for students who have the ability from -1.0 to +1.7. The maximum score of item information function is 68.83 (SEM =0.121) with ability in 0.2 logit. The highest ability’s estimation score was showed by Model 2-PL. The mean of students’ ability for 11<sup>th</sup> grade students is -0.0185 logit and consider as moderate category.</p><p> </p>

scholarly journals ANALYSIS AND EVALUATION OF MULTIPLE-CHOICE TEST ITEMS AND TEST DESIGN: A STUDY ON APPLICATION OF ITEM RESPONSE THEORY

2021 ◽  
Vol 226 (13) ◽  
pp. 72-81
Author(s):  
Nguyễn Văn Cảnh ◽  
Phạm Văn Tặc
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Multiple Choice ◽  
Choice Test ◽  
Test Design ◽  
Multiple Choice Test ◽  
Response Theory ◽  
Analysis And Evaluation ◽  
Test Items

Bài viết trình bày kết quả ứng dụng lý thuyết ứng đáp câu hỏi với mô hình 2 tham số vào việc phân tích, đánh giá câu hỏi và xây dựng đề thi trắc nghiệm khách quan. Dựa trên việc phân tích dữ liệu kết quả thi của sinh viên (không thuộc chuyên ngành Tiếng Anh) đối với các đề thi Tiếng Anh 1 được sử dụng tại Trường Đại học Đồng Tháp từ năm 2017 đến 2021 bằng phần mềm R (gói ltm), nghiên cứu đã chỉ ra những câu hỏi đạt yêu cầu, đủ điều kiện để sử dụng trong các đề thi và những cầu hỏi chưa đạt yêu cầu, cần phải được xem xét lại để điều chỉnh, cải tiến. Trong đó, một số câu hỏi được sử dụng trong các đề thi có dấu hiệu bất thường, vi phạm nghiệm trọng về giới hạn giá trị các tham số độ khó, độ phân biệt cần phải được loại bỏ ra khỏi đề thi. Ngoài ra, kết quả nghiên cứu còn cho thấy các đề thi trên đều có ý nghĩa đo lường mức năng lực khá thấp (dưới 0.0 theo thang đo năng lực). Bên cạnh đó, nghiên cứu đã giới thiệu cách vận dụng công cụ đường cong thông tin vào việc xây dựng các đề thi giúp đo lường chính xác năng lực của người học dựa trên các tham số đặc trưng của các câu hỏi.

scholarly journals Employment of Item Response Theory to measure change in Children's Analogical Thinking Modifiability Test

2013 ◽  
Vol 30 (4) ◽  
pp. 479-486
Author(s):  
Odoisa Antunes de Queiroz ◽  
Ricardo Primi ◽  
Lucas de Francisco Carvalho ◽  
Sônia Regina Fiorim Enumo
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Dynamic Testing ◽  
Intermediate Phase ◽  
Cognitive Testing ◽  
Difficulty Level ◽  
Response Theory ◽  
Analogical Thinking ◽  
Test Items ◽  
Post Test

Dynamic testing, with an intermediate phase of assistance, measures changes between pretest and post-test assuming a common metric between them. To test this assumption we applied the Item Response Theory in the responses of 69 children to dynamic cognitive testing Children's Analogical Thinking Modifiability Test adapted, with 12 items, totaling 828 responses, with the purpose of verifying if the original scale yields the same results as the equalized scale obtained by Item Response Theory in terms of "changes quantifying". We followed the steps: 1) anchorage of the pre and post-test items through a cognitive analysis, finding 3 common items; 2) estimation of the items' difficulty level parameter and comparison of those; 3) equalization of the items and estimation of "thetas"; 4) comparison of the scales. The Children's Analogical Thinking Modifiability Test metric was similar to that estimated by the TRI, but it is necessary to differentiate the pre and post-test items' difficulty, adjusting it to samples with high and low performance.

Postexamination Analysis of Objective Tests Using the Three-Parameter Item Response Theory

2014 ◽  
Vol 22 (1) ◽  
pp. 94-105
Author(s):  
Mohsen Tavakol ◽  
Mohammad Rahimi-Madiseh ◽  
Reg Dennick
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Item Parameter ◽  
Response Theory ◽  
Characteristic Curves ◽  
Test Characteristic ◽  
Test Items ◽  
Test Information ◽  
Objective Tests

Background and Purpose: Although the importance of item response theory (IRT) has been emphasized in health and medical education, in practice, few psychometricians in nurse education have used these methods to create tests that discriminate well at any level of student ability. The purpose of this study is to evaluate the psychometric properties of a real objective test using three-parameter IRT. Methods: Three-parameter IRT was used to monitor and improve the quality of the test items. Results: Item parameter indices, item characteristic curves (ICCs), test information functions, and test characteristic curves reveal aberrant items which do not assess the construct being measured. Conclusions: The results of this study provide useful information for educators to improve the quality of assessment, teaching strategies, and curricula.

scholarly journals Characteristics and equation of accounting vocational theory trial test items for vocational high schools by subject-matter teachers' forum

2017 ◽  
Vol 3 (2) ◽  
pp. 152
Author(s):  
Dian Normalitasari Purnama
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Sampling Technique ◽  
Response Theory ◽  
Test Items ◽  
Descriptive Research ◽  
Vocational Theory ◽  
Trial Test ◽  
The Mean ◽  
B Test

This study is aimed at: (1) understanding the characteristics of Accounting Vocational Theory trial test items using the Item Response Theory and (2) determining the horizontal equation of Accounting Vocational Theory trial exam instruments. This was explorative-descriptive research, observing the subject of the eleventh-grade students. The research objects were test instruments and responses of students from six schools selected through the stratified random sampling technique. The data analysis employed review sheets and BILOG program for the Item Response Theory 2PL. The findings were as follows. (1) The test item review of test packages A and B found 37 good quality items, the Item Response Theory using 2PL showed that Package A Test generated 27 good questions, Package B Test contained 24 good questions. (2) The question equating using the Mean/Sigma method resulted in the equation of = 1.168bx + 0.270, with the Mean/Mean method resulting in the equation of  = 0.997bx - 0.250, the Mean/Mean method at 0.250, while Mean/Sigma method at 0.320. 

scholarly journals Analysis of Socio-Economic Status and Gender Related Differential Item Functioning Using Item Response Theory Approach

2021 ◽  
pp. 42-55
Author(s):  
Stella Eteng-Uket
Keyword(s):  
Item Response Theory ◽  
Differential Item Functioning ◽  
Item Response ◽  
English Language ◽  
Economic Status ◽  
Objective Test ◽  
Response Theory ◽  
Socio Economic Status ◽  
Item Functioning ◽  
Research Questions

The study investigated detecting differential item functioning using item response theory in West African Senior School Certificate English language test in south-south Nigeria. 2 research questions were formulated to guide the study. Using descriptive research survey design for the study, study population was 117845 Senior Secondary 3 students in Edo, Delta, Rivers and Bayelsa state. A sample of 1309 (604 males, 705 females) drawn through multi stage sampling technique was used for the study. Two valid instruments titled: Socio-economic status questionnaire (SSQ) and WASSCE/SSCE English language objective test (ELOT) were used to collect data for the study. The reliability indices of the instruments were estimated using the Cronbach Alpha method of internal consistency and Richard Kuderson 20 with coefficient values of .84 for the English Language objective test and .71 for the socio-economic status questionnaire respectively. Chi-square and Lord Wald test statistics statistical technique employed by Item Response Theory for Patient Reported Outcome (IRTPRO) was the technique used in data analysis which provided answers to the research questions at.05 level of significance. On analysis, the result revealed that 13 items functioned differently significant between the male and female group and significantly 23 items differentially functioned between High and low socio-economic status group. Thus, this shows 18% DIF based on gender and 32% based on socio-economic status indicating large DIF and items that are potentially biased. Based on the findings, recommendation were made and one among others was that Item Response theory should be used as DIF detection method by large scale public examination and test developers.

scholarly journals The Applicability of Item Response Theory Based Statistics to Detect Differential Item Functioning in Polytomous Tests

2020 ◽  
Vol 1 (1) ◽  
pp. 1-13
Author(s):  
Abdul Wahab Ibrahim
Keyword(s):  
Item Response Theory ◽  
Differential Item Functioning ◽  
Item Response ◽  
Undergraduate Students ◽  
Function Analysis ◽  
Response Theory ◽  
Test Items ◽  
Item Functioning ◽  
Logistic Discriminant Function ◽  
High Degree

The study used statistical procedures based on Item Response Theory to detect Differential Item Functioning (DIF) in polytomous tests. These were with a view to improving the quality of test items construction. The sample consisted of an intact class of 513 Part 3 undergraduate students who registered for the course EDU 304: Tests and Measurement at Sule Lamido University during 2017/2018 Second Semester. A self-developed polytomous research instrument was used to collect data. Data collected were analysed using Generalized Mantel Haenszel, Simultaneous Item Bias Test, and Logistic Discriminant Function Analysis. The results showed that there was no significant relationship between the proportions of test items that function differentially in the polytomous test when the different statistical methods are used.  Further, the three parametric and non-parametric methods complement each other in their ability to detect DIF in the polytomous test format as all of them have capacity to detect DIF but perform differently. The study concluded that there was a high degree of correspondence between the three procedures in their ability to detect DIF in polytomous tests. It was recommended that test experts and developers should consider using procedure based on Item Response Theory in DIF detection.

Understanding the Impact of Expertise on Design Outcome: An Approach Based on Concept Inventories and Item Response Theory

2016 ◽  
Author(s):  
Murtuza Shergadwala ◽  
Karthik N. Kannan ◽  
Jitesh H. Panchal
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Research Question ◽  
Behavioral Experiment ◽  
Response Theory ◽  
Solution Quality ◽  
Concept Inventories ◽  
Expert Novice ◽  
The Impact

Research on expertise in design has focused primarily on understanding expert-novice differences. Although it is well established that experts perform better than novices, there is a lack of formal methods to quantify the potential impact of expertise on the quality of design outcomes. The research question addressed in this paper is: How can the impact of expertise on the quality of design solutions be quantified? Quantifying such impacts can be of particular importance in product development, recruitment processes and design competitions. We utilize an approach based on Item Response Theory (IRT) and Concept Inventories (CI) for expertise quantification. We then investigate and validate the impact of expertise on solution quality through a behavioral experiment involving a track design problem. The results highlight the usefulness of the proposed approach and provide a functional relationship between expertise and solution quality. We also observe behavioral differences between participants with varying scores on a test taken in the behavioral experiment. The proposed approach could be used in the future work to quantify learning.

Sign in / Sign up

Export Citation Format

Share Document