Identifying Informatively Easy and Informatively Hard Concepts

In this article, we leverage ideas from the theory of coevolutionary computation to analyze interactions of students with problems. We introduce the idea of informatively easy or hard concepts. Our approach is different from more traditional analyses of problem difficulty such as item analysis in the sense that we consider Pareto dominance relationships within the multidimensional structure of student–problem performance data rather than average performance measures. This method allows us to uncover not just the problems on which students are struggling but also the variety of difficulties different students face. Our approach is to apply methods from the Dimension Extraction Coevolutionary Algorithm to analyze problem-solving logs of students generated when they use an online software tutoring suite for introductory computer programming called problets . The results of our analysis not only have implications for how to scale up and improve adaptive tutoring software but also have the promise of contributing to the identification of common misconceptions held by students and thus, eventually, to the construction of a concept inventory for introductory programming.

Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines

A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this article, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.

Psychometric Evaluation of the Positive Mental Health (PMH) Scale Using Item Analysis According to the Rasch Model

Background: The investigation of patient-reported outcomes and psycho-oncological interventions mainly focuses on psychological distress or psychopathology. However, the recognition of the equal importance of positive mental health (PMH) has increased lately. The PMH-scale is a brief questionnaire allowing to assess well-being in individuals in the general population and in patients. Previous studies evaluated the psychometric properties of the PMH-scale using classical test theory (CTT). This study is the first to investigate the PMH-scale in patients with cancer using item analysis according to the Rasch model. Methods: In total, N = 357 cancer patients participated in the study. A Rasch analysis of the PMH-scale was conducted including testing of unidimensionality, local independence, homogeneity and differential item functioning (DIF) with regard to age, gender, type of cancer, the presence of metastases, psycho-oncological support, and duration of disease. Additionally, the ordering of the item thresholds as well as the targeting of the scale were investigated.Results: After excluding one misfitting item and accounting for local dependence by forming superitems, a satisfactory overall fit to the Rasch model was established (χ2 = 30.34, p = 0.21). The new PMH-8 scale proved to be unidimensional, and homogeneity of the scale could be inferred. All items showed ordered thresholds, there was no further item misfit. DIF was found for age, but as the impact of DIF was not substantial, no adjustment related to the age-DIF had to be made. The Person Separation Index (PSI = 0.89) was excellent, indicating excellent discriminatory power between different levels of positive mental health. Overall, the targeting of the PMH-8 was good for the majority of the present sample. However, at both ends of the scale item thresholds are missing as indicated by a slight floor effect (1.4%) and a considerable ceiling effect (9.8%). Conclusion: Overall, the results of the analysis according to the Rasch model support the use of the revised PMH-scale in a psycho-oncological context.

APPLICATIONS OF ITEM RESPONSE THEORY MODELS TO ASSESS ITEM PROPERTIES AND STUDENTS’ ABILITIES IN DICHOTOMOUS RESPONSES ITEMS

A test is a tool meant to measure the ability level of the students, and how well they can recall the subject matter, but items making up a test may be defectives, and thereby unable to measure students’ ability or traits satisfactorily as intended if proper attention is not paid to item properties such as difficulty, discrimination, and pseudo guessing indices (power) of each item. This could be remedied by item analysis and moderation. It is a known fact that the absence or improper use of item analysis could undermine the integrity of assessment, certification and placement in our educational institutions. Both appropriateness and spread of items properties in accessing students’ abilities distribution, and the adequacy of information provided by dichotomous response items in a compulsory university undergraduate statistics course which was scored dichotomously, and analyzed with stata 16 SE on window 7 were focused here. In view of this, three dichotomous Item Response Theory (IRT) measurement models were used in the context of their potential usefulness in an education setting such as in determining these items properties. Ability, item discrimination, difficulty, and guessing parameters as unobservable characteristics were quantified with a binary response test, then discrete item response becomes an observable outcome variable which is associated with student’s ability level is thereby linked by Item Characteristic Curves that is defined by a set of item parameters that models the probability of observing a given item response by conditioning on a specific ability level. These models were used to assess each of the three items properties together with students’ abilities; then identified defective items that were needed to be discarded, moderated, and non-defectives items as the case may be while some of these chosen items were discussed based on underlining models. Finally, the information provided by these items was also discussed.

Adaptation of Trait Emotional Intelligence Questionnaire (TEIQue) in Bangla

The Trait Emotional Intelligence Questionnaire (TEQue) is an instrument to measure emotional intelligence based on self-report. The instrument has increasingly drawing the attention in the organizations. The present study aimed at adapting the Trait Emotional Intelligence Questionnaire (TEQue) short form (K.V. Petrides, 2009) into Bangla version After careful checking by the Subject Matter Expert (SME), the translated form was administered over 200 adolescent-adult ranging from 17 to 21 years of age through convenience sampling during May 2017 to June 2017. Cronbach Alpha was found significant to be satisfactory. Validation of the Inventory was assured by content and construct validity where construct validity included item analysis. Satisfactory level of validity was found for the Bangla version of Trait Emotional Intelligence Questionnaire. The version of the test has found useful in the context of Bangladesh. This inventory can be used in Emotional Intelligence testing.

Development of Cognitive Learning Scale to Test the Knowledge Level of Veterinary Students on ICT

In the present study, to test the knowledge level of veterinary students on ICT, one hundred and seventy-one items were initially constructed on the basis of promoting thinking rather than rote memorization. It was designed in a manner that could differentiate the well-informed veterinary students from less informed ones. The scores of the respondents were subjects to item analysis to find the item difficulty index and item discrimination index. In the final selection, a total of 34 items with difficulty index between 30 and 80 and discrimination index ranging from 0.30 to 0.55 were selected. The reliability of the knowledge test developed was tested using split half technique. The coefficient of correlation value in split half test was 0.89, which was found to be significant at 1 per cent level of significance. It was found that, the developed knowledge test scale of Veterinary students on ICT was highly stable and can be used for measurement.