Modeling Faking in the Multidimensional Forced-Choice Format: The Faking Mixture Model

The multidimensional forced-choice (MFC) format has been proposed to reduce faking because items within blocks can be matched on desirability. However, the desirability of individual items might not transfer to the item blocks. The aim of this paper is to propose a mixture item response theory model for faking in the MFC format that allows to estimate the fakability of MFC blocks, termed the Faking Mixture model. Given current computing capabilities, within-subject data from both high- and low-stakes contexts are needed to estimate the model. A simulation showed good parameter recovery under various conditions. An empirical validation showed that matching was necessary but not sufficient to create an MFC questionnaire that can reduce faking. The Faking Mixture model can be used to reduce fakability during test construction.

The Precipitous Decline in Reasoning and Other Key Abilities with Age and Its Implications for Federal Judges

U. S. Supreme Court justices and other federal judges are, effectively, appointed for life, with no built-in check on their cognitive functioning as they approach old age. There is about a century of research on aging and intelligence that shows the vulnerability of processing speed, fluid reasoning, visual-spatial processing, and working memory to normal aging for men and women at all levels of education; even the maintained ability of crystallized knowledge declines in old age. The vulnerable abilities impact a person’s decision-making and problem solving; crystallized knowledge, by contrast, measures a person’s general knowledge. The aging-IQ data provide a rationale for assessing the key cognitive abilities of anyone who is appointed to the federal judiciary. Theories of multiple cognitive abilities and processes, most notably the Cattell-Horn-Carroll (CHC) model, provide a well-researched blueprint for interpreting the plethora of findings from studies of IQ and aging. Sophisticated technical advances in test construction, especially in item-response theory and computerized-adaptive testing, allow for the development of reliable and valid theory-based tests of cognitive functioning. Such assessments promise to be a potentially useful tool for evaluating federal judges to assess the impact of aging on their ability to perform at a level their positions deserve, perhaps to measure their competency to serve the public intelligently. It is proposed that public funding be made available to appoint a panel of experts to develop and validate an array of computerized cognitive tests to identify those justices who are at risk of cognitive impairment.

autoFC: An R Package for Automatic Item Pairing in Forced-Choice Test Construction

Recently, there has been increasing interest in adopting the forced-choice (FC) test format in non-cognitive assessments, as it demonstrates faking resistance when well-designed. However, traditional or manual pairing approaches to FC test construction are time- and effort- intensive and often involve insufficient considerations. To address these issues, we developed the new open-source autoFC R package to facilitate automated and optimized item pairing strategies. The autoFC package is intended as a practical tool for FC test constructions. Users can easily obtain automatically optimized FC tests by simply inputting the item characteristics of interest. Customizations are also available for considerations on matching rules and the behaviors of the optimization process. The autoFC package should be of interest to researchers and practitioners constructing FC scales with potentially many metrics to match on and/or many items to pair, essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods.

Assessing the Achievement Testing Practices of Teachers in Junior High Schools in the Sissala East Municipality of Ghana

The study sought to find out whether Junior High School teachers in the Sissala East Municipality of Ghana follow the basic prescribed principles in the construction, administration and scoring of classroom achievement tests. A descriptive survey design was adopted for the study. The study employed multistage sampling techniques (purposive, stratified, and simple random sampling procedures) to select a sample of 248 teachers from the Sissala East Municipality. A questionnaire was used for the data collection and its reliability was 0.8. Data were analysed using frequencies, means, and standard deviation. The results showed that generally, Junior High Schools teachers in the Sissala East Municipality adhere to most principles of test construction and test administration but had little knowledge of the principles in the scoring test. It was recommended that more workshops and in-service training should be organized for teachers in Junior High Schools concerning their testing practices (i.e., construction, administration and scoring of tests).

Application of Blue Filters Increases the Usefulness of Moreland Test in Anomaloscopic Color Vision Assessment for Blue–Green Color Range

The effect of blue light filters on the anomaloscopic examination was analyzed. Thirty subjects (18–43 y, 20 female, 10 male) without color vision disorders were examined in 4 filter conditions: no filter (F-0), Blue Control Hoya (F-BC), Med-1 JZO (F-Med1) and 450 Eschenbach (F-450). Both Rayleigh test (red–green axis) and Moreland test (blue–green axis) were performed. Application of F-BC filter shows negligible effect on color vision perception in both tests. Contrary to this, the application of strong F-450 filter causes significant shift in Moreland test towards tritanopy and the decrease in correlations of Moreland parameters with Rayleigh test parameters. The application of medium strong F-Med1 filter causes the slight shift in Moreland test towards the center of the Moreland scale and increases the Spearman correlations between Moreland and Rayleigh test parameters. This observation suggests that the about 15–40% reduction of blue diode intensity in the Moreland test may be beneficial in detecting mild changes in color vision perception in the blue-green axis and may improve its usefulness in evaluating the color vision perception disorders accompanying different illnesses, such as diabetes, glaucoma, neuritis optica, or cataract. The discussion concerning the modifications of Moreland test construction is also presented.

The Development of STEP, the CEFR-Based English Proficiency Test

This study aimed at developing the Silpakorn Test of English Proficiency (STEP), in alignment with the Common European Framework of Reference for Languages (CEFR), and in accordance with the theoretical framework established by Alderson et al. (2006). Four major steps were involved in the test construction. First, English language lecturers who served as content specialists were asked to design can-do statements presented in the CEFR. Then the specialists designed the test specification based on the can-do statements. Four skill areas: listening, semi-speaking, reading, and semi-writing were targeted as the test construct. At this juncture, the content specialists were required to write test items in accordance with the test specification. Next, the test items constructed were determined for their validity and reliability. Finally, a standard setting was carried out. The results demonstrated that the framework offered by Alderson et al. (2006) served as an effective reference document for developing the STEP. In terms of validity and reliability, the STEP was of statistical significance, that is, it could be aligned with the CEFR levels and measure test takers’ English proficiency at a specific CEFR level. The current findings provide useful insights for test developers or researchers who wish to design proficiency tests in alignment with the CEFR.

Analyzing Nested Experimental Designs - A User-Friendly Resampling Method to Determine Experimental Significance

While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. Resampling-based methods are a flexible strategy for performing these analyses but are difficult due to the lack of open-source software to automate test construction and execution. To address this, we report Hierarch, a Python package to perform hypothesis tests and compute confidence intervals on hierarchical experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest. Hierarch makes use of the Numba JIT compiler to reduce p-value computation times to under one second for typical datasets in biomedical research. Hierarch also enables researchers to construct user-defined resampling plans that take advantage of Hierarch's Numba-accelerated functions. Hierarch is freely available as a Python package at https://github.com/rishi-kulkarni/hierarch.