Challenges in addressing student difficulties with time-development of two-state quantum systems using a multiple-choice question sequence in virtual and in-person classes

Research-validated clicker questions as instructional tools for formative assessment are relatively easy to implement and can provide effective scaffolding when developed and implemented in a sequence. We present findings from the implementation of a research-validated Clicker Question Sequence (CQS) on student understanding of the time-development of two-state quantum systems. This study was conducted in an advanced undergraduate quantum mechanics course for two consecutive years in virtual and in-person classes. The effectiveness of the CQS discussed here in both modes of instruction was determined by evaluating students’ performance after traditional lecture-based instruction and comparing it to their performance after engaging with the CQS.

Test-Wiseness and Multiple-Choice Accounting Questions: Implications for Instructors

This study examines “test-wiseness” rules-of-thumb accounting students may use when they cannot answer a multiple-choice question. The effectiveness of the rules is poorly understood, but they rely largely on preventable flaws in question design. After identifying eleven relevant rules, we had graduate research assistants employ the rules on unfamiliar material, to test whether the rules can be effective when used by capable students who have not studied the material. Three of the rules demonstrate positive results, and participants are able to recognize the more valuable ones. Additionally, we survey accounting majors at two universities to assess general familiarity and belief in the rules. We find that they have well-formed ideas of the relative usefulness, which are consistent between universities. The results provide some insights into the issues question writers should consider, so as to avoid vulnerability to test-wiseness or even turn the rules to their advantage when composing questions.

Multiple-Choice Question Answering Models for Automatic Depression Severity Estimation

Depression is one of the most prevalent mental health diseases. Although there are effective treatments, the main problem relies on providing early and effective risk detection. Medical experts use self-reporting questionnaires to elaborate their diagnosis, but these questionnaires have some limitations. Social stigmas and the lack of awareness often negatively affect the success of these self-report questionnaires. This article aims to describe techniques to automatically estimate the depression severity from users on social media. We explored the use of pre-trained language models over the subject’s writings. We addressed the task “Measuring the Severity of the Signs of Depression” of eRisk 2020, an initiative in the CLEF Conference. In this task, participants have to fill the Beck Depression Questionnaire (BDI-II). Our proposal explores the application of pre-trained Multiple-Choice Question Answering (MCQA) models to predict user’s answers to the BDI-II questionnaire using their posts on social media. These MCQA models are built over the BERT (Bidirectional Encoder Representations from Transformers) architecture. Our results showed that multiple-choice question answering models could be a suitable alternative for estimating the depression degree, even when small amounts of training data are available (20 users).

Engagement and learning in an electronic spaced repetition curriculum companion for a paediatrics academic half-day curriculum

Postgraduate residencies utilize academic half-days to supplement clinical learning. Spaced repetition reinforces taught content to improve retention. We leveraged spaced repetition in a curriculum companion for a paediatric residency program’s half-day. One half-day lecture was chosen weekly for reinforcement (day 0). Participants received 3 key points on day 1 and a multiple-choice question (MCQ) on day 8. On day 29, they received two MCQs to test reinforced and unreinforced content from the same day 0. Thirty-one (79%) residents participated over 17 weeks, but only 14 (36%) completed more than half of the weekly quizzes. Of all quizzes, 37.4% were completed, with an average weekly engagement of 5.5 minutes. Helpfulness to learning was rated as 7.89/10 on a Likert-like scale. Reported barriers were missing related half-days and emails, or limited time. There was no significant difference in performance between reinforced (63.4%, [53.6–73.3]) and unreinforced (65.6%, [53.7–73.2]) questions. Spaced repetition is a proven strategy in learning science, but was not shown to improve performance. Operational barriers likely limited participation and underpowered our analysis, therefore future implementation must consider practical and individual barriers to facilitate success. Our results also illustrate that satisfaction alone is an inadequate marker of success.

Increased ability to arrange two tier multiple choice questions

SMP Muhammadiyah 2 Malang City is a school which is located on Jl Letjen Sutoyo No 68 Malang City. This school has great potential and has the opportunity to carry out two tier multiple choice question-making service activities because the school is the school's top priority (partner). The problem faced is that teachers have never compiled Two Tier Multiple Choice questions, especially science teachers. The science teacher said that he was always prepared questions for student evaluation instruments, either formative, sub-summative or summative, but the questions that were arranged were still conventional. The teacher said that he was not yet skilled in compiling HOTS questions, let alone the type of Two Tier Multiple Choice that he had never known. Activities to be carried out are socialization of teacher training programs, pre-test teacher assistance. Teachers have increased their abilities regarding the preparation of Two Tier Multiple Choice questions. The teacher-made question products have 30 sets or 60 items, and have been tested with the results of the questions that qualify as good questions

An Automated Multiple-Choice Question Generation using Natural Language Processing Techniques

Automatic multiple-choice question generation (MCQG) is a useful yet challenging task in Natural Language Processing (NLP). It is the task of automatic generation of correct and relevant questions from textual data. Despite its usefulness, manually creating sizeable, meaningful and relevant questions is a time-consuming and challenging task for teachers. In this paper, we present an NLP-based system for automatic MCQG for Computer-Based Testing Examination (CBTE).We used NLP technique to extract keywords that are important words in a given lesson material. To validate that the system is not perverse, five lesson materials were used to check the effectiveness and efficiency of the system. The manually extracted keywords by the teacher were compared to the auto-generated keywords and the result shows that the system was capable of extracting keywords from lesson materials in setting examinable questions. This outcome is presented in a user-friendly interface for easy accessibility.

Assessing the Quality of MCQs in the Final Examination of UUM Foundation Students

The multiple-choice question (MCQ) format is commonly used to assess student knowledge as it is able to accommodate a large number of participants. The advantages of MCQs are that they are easy to handle and the results are able to be obtained quickly. To ensure that students are effectively tested, it is important to analyse the quality of MCQs by assessing the questions or items using educational measurements. The aim of this study is to assess the quality of MCQs of final exam questions of foundation students by using the Difficulty Index and Discriminant Index. The sample for this study is the Introduction to Philosophy final exam paper which is one of the courses taken by foundation students. The analysis involves 33 graded final exams and 20 MCQ items. The findings showed that there are 3 items that are ‘good’ that can be retained, 9 items that are considered as ‘fair acceptable/not acceptable’ which can be retained or revised, and 8 items that should be revised or discarded. The results of this study show that it is important to assess the quality of items in the assessment as it is able to change the selection of items in the assessment.