APPLICATIONS OF ITEM RESPONSE THEORY MODELS TO ASSESS ITEM PROPERTIES AND STUDENTS’ ABILITIES IN DICHOTOMOUS RESPONSES ITEMS

A test is a tool meant to measure the ability level of the students, and how well they can recall the subject matter, but items making up a test may be defectives, and thereby unable to measure students’ ability or traits satisfactorily as intended if proper attention is not paid to item properties such as difficulty, discrimination, and pseudo guessing indices (power) of each item. This could be remedied by item analysis and moderation. It is a known fact that the absence or improper use of item analysis could undermine the integrity of assessment, certification and placement in our educational institutions. Both appropriateness and spread of items properties in accessing students’ abilities distribution, and the adequacy of information provided by dichotomous response items in a compulsory university undergraduate statistics course which was scored dichotomously, and analyzed with stata 16 SE on window 7 were focused here. In view of this, three dichotomous Item Response Theory (IRT) measurement models were used in the context of their potential usefulness in an education setting such as in determining these items properties. Ability, item discrimination, difficulty, and guessing parameters as unobservable characteristics were quantified with a binary response test, then discrete item response becomes an observable outcome variable which is associated with student’s ability level is thereby linked by Item Characteristic Curves that is defined by a set of item parameters that models the probability of observing a given item response by conditioning on a specific ability level. These models were used to assess each of the three items properties together with students’ abilities; then identified defective items that were needed to be discarded, moderated, and non-defectives items as the case may be while some of these chosen items were discussed based on underlining models. Finally, the information provided by these items was also discussed.

Citizen Science for Marine Litter Detection and Classification on Unmanned Aerial Vehicle Images

Unmanned aerial vehicles (UAV, aka drones) are being used for mapping macro-litter in the environment. As drone images require a manual processing task for detecting marine litter, it is of interest to evaluate the accuracy of non-expert citizen science operators (CSO) in performing this task. Students from Italian secondary schools (in this work, the CSO) were invited to identify, mark, and classify stranded litter items on a UAV orthophoto collected on an Italian beach. A specific training program and working tools were developed for the aim. The comparison with the standard in situ visual census survey returned a general underestimation (50%) of items. However, marine litter bulk categorisation was fairly in agreement with the in situ survey, especially for sources classification. The concordance level among CSO ranged between 60% and 91%, depending on the item properties considered (type, material, and colour). As the assessment accuracy was in line with previous works developed by experts, remote detection of marine litter on UAV images can be improved through citizen science programs, upon an appropriate training plan and provision of specific tools.

Development and Psychometric Evaluation of the Physical Resilience Instrument for Older Adults (PRIFOR)

Background: Physical resilience is known to minimize the adverse outcomes of health stressors for older people. However, validated instruments that assess physical resilience in older adults are rare. Therefore, we aimed to validate the Physical Resilience Instrument for Older Adults (PRIFOR) to fill the literature gap. Methods: Content analysis with content validity was first carried out to generate relevant items assessing physical resilience for older adults, and 19 items were developed. Psychometric evaluation of the 19 items was then tested on 200 older adults (mean [SD] age = 76.4 [6.6] years; 51.0% women) via item properties, factor structure, item fit, internal consistency, criterion-related validity, and known-group validity. Results: All 19 items had satisfactory item properties, as they were normally distributed (skewness = -1.03 to 0.38; kurtosis = -1.05 to 0.32). However, two items were removed due to substantial ceiling effects. The retained 17 items were embedded in three factors as suggested by the exploratory factor analysis (EFA). All items except one had satisfactory item fit statistics; thus, the unidimensionality was supported for the three factors on 16 items. The retained 16 items showed promising properties in known-group validity, criterion-related validity, and internal consistency (α = 0.94). Conclusions: The 16-item PRIFOR exhibits good psychometric properties. Using this instrument to measure physical resilience would be beneficial to identify factors that could protect older people from negative health consequence. With the use of the PRIFOR, intervention effects can also be evaluated. It is helpful to strengthen resilience and thereby facilitate successful aging.

Assessing Momentary States of Relaxation: The Relaxation State Questionnaire

Stress is ubiquitous in everyday life and hazardous for mental and physical health. To prevent or ameliorate stress-related disease, approaches such as relaxation exercises aim to counteract stress by inducing short-lasting states of relaxation. Critically, current assessments capture the mid- and long-term consequences of relaxation but are unable to quantify these states of relaxation. To address this problem, we developed the Relaxation State Questionnaire (RSQ). We assessed the item properties, reliability, and validity of the questionnaire in an online study. Construct validity was examined through correlations with the Perceived Stress Questionnaire (PSQ; Fliege et al., 2005). An exploratory factor analysis revealed four factors capturing the momentary state of muscle tension, sleepiness, cardiovascular activity, and general relaxation. The items had high loadings on their factors and the reliability and construct validity of the RSQ were both high. These findings establish the RSQ as a tool to measure states of relaxation. As such, the RSQ opens up investigations of the immediate effects and effectiveness of relaxation exercises.

The Organization of Political Belief Networks: A Cross-Country Analysis

Studies on mass opinion conceptualize political ideology as an interrelated network of attitudes, beliefs and values. Drawing from the emerging literature on belief network analysis, I formalize this conceptualization to investigate different ways of organizing political ideologies among 76 countries. Using the joint dataset of European Values Study and World Values Survey collected between 2017 and 2020, I construct networks of belief correlations and examine the variations in the organization of political beliefs in the cross-national context. Consistent with the theories of social constraint, I find that in countries with high institutionalization of political parties, the organization of political beliefs is more likely to be dense and consolidated. These patterns are robust to a variety of predictors between-countries, and the results are not sensitive to sampling variability, variable selection, or item properties.

Development and Validation of a New Measure of Mental Rotation for Preadolescent and Adolescent Groups

Measurement of mental rotation presents a serious challenge to cognitive researchers owing to the lack of a single comprehensive measure that can be applied across the developing age groups. Objective of the present study was to develop and validate a new measure of mental rotation for preadolescent and adolescent age groups. Items were conceptualized and constructed based on existing theories. Study I checked the suitability of these items among preadolescent and adolescent age groups. After revisions, Study II was done to evaluate the item properties using item response theory. Subsequently done Confirmatory Factor Analysis provided evidence for the construct validity of the new measure. Finally, Study III was done to develop the age wise and gender wise norms for preadolescent and adolescent age groups. The newly developed measure was found to have sufficient reliability and validity and hence can be widely applied for measuring mental rotation of preadolescents and adolescents.

Cross-linguistic transfer in bilingual reading is item specific

The grain size of orthographic representations prompted by a consistent orthography (like Spanish or Basque) increases if reading is simultaneously learned in another language with an inconsistent orthography (like French). Here, we aimed to identify item properties that trigger this grain-size accommodation in bilingual reading. Twenty-five French–Basque and 25 Spanish–Basque bilingual children attending Grade 3 read Basque words and pseudowords containing “complex” letter clusters mapping to one sound in French but several sounds in Basque or Spanish, and “simple” letter clusters mapping to the same sound structure in all three languages. Only French speaking children read “complex” Basque words faster than “simple” ones, suggesting that they accessed multi-letter “French” units to boost lexical processing. A negative complexity effect was found for pseudowords across groups. We discuss the existence of flexible cross-linguistic transfer in bilingual reading, proposing that the grain size of orthographic representations adjusts to item-specific characteristics during reading.

Ensemble Statistics Can Be Available before Individual Item Properties: Electroencephalogram Evidence Using the Oddball Paradigm

Behavioral studies have shown that statistical properties of object groups are perceived accurately with brief exposure durations. This finding motivated the hypothesis that ensemble perception occurs rapidly in vision. However, the precise timing of ensemble perception remains unclear. Here, we used the superior temporal resolution of electroencephalography to directly compare the timing of ensemble processing to that of individual object processing. The P3b was chosen as a particular component of interest, as it is thought to measure the latency of stimulus evaluation. Participants performed a simple “oddball” task in which sets of 51 lines with varied orientations sequentially flashed briefly on the display. In these sequences, there was a 20% chance of an individual oddball, wherein one marked object tilted clockwise, and a 20% chance of an ensemble oddball, wherein the average orientation of the set tilted 20% clockwise. In counterbalanced blocks, participants were instructed to respond to either individual or ensemble oddballs. ERP analysis was performed to test the timing of this processing. At parietal electrodes, P3b components were found for both individual and ensemble oddballs. Ensemble P3b components were found to occur significantly earlier than individual P3b components, as measured with both 50% area latency and 50% onset latency. Using multivariate pattern analysis, ensemble oddball trials were classifiable from standard trials significantly earlier in their timecourse than individual oddball trials. Altogether, these results provide compelling evidence that ensemble perception occurs rapidly and that ensemble properties can be available earlier than individual object properties.

You Can Play the Game Without Knowing the Rules – But You’re Better Off Knowing Them

Due to their high item difficulties and excellent psychometric properties, construction-based figural matrices tasks are of particular interest when it comes to high-stakes testing. An important prerequisite is that test preparation – which is likely to occur in this context – does not impair test fairness or item properties. The goal of this study was to provide initial evidence concerning the influence of test preparation. We administered test items to a sample of N = 882 participants divided into two groups, but only one group was given information about the rules employed in the test items. The probability of solving the items was significantly higher in the test preparation group than in the control group ( M = 0.61, SD = 0.19 vs. M = 0.41, SD = 0.25; t(54) = 3.42, p = .001; d = .92). Nevertheless, a multigroup confirmatory factor analysis, as well as a differential item functioning analysis, indicated no differences between the item properties in the two groups. The results suggest that construction-based figural matrices are suitable in the context of high-stakes testing when all participants are provided with test preparation material so that test fairness is ensured.

Gender Differences or Gender Bias?

Sadism, defined by the infliction of pain and suffering on others for pleasure or subjugation, has recently garnered substantial attention in the psychological research literature. The Assessment of Sadistic Personality (ASP) was developed to measure levels of everyday sadism and has been shown to possess excellent reliability and validity using classical test theory methods. However, it is not known how well ASP items discriminate between respondents of different trait levels, or which Likert categories are endorsed by persons of various trait levels. Additionally, individual items should be evaluated to ensure that men and women of similar levels of sadism have an equal probability of response endorsement. The purpose of this research was to apply item response theory (IRT) and differential item functioning (DIF) to investigate item properties of the ASP across its three translations: English, Polish, and Italian. Overall, the results of the IRT analysis showed that with the exception of Item 9, the ASP demonstrated sound item properties. The DIF rate analyses identified two items from each questionnaire that were of practical significance across gender. Implications of these results are discussed.