STUDENT ACCEPTANCE LEVEL TOWARD THE USE OF ONLINE FACILITIES TO SUPPORT FLEXIBLE LEARNING ACTIVITIES BASED ON THE DESIGN ELEMENTS OF THE HEUTAGOGICAL APPROACH

Heutagogical learning approach is a flexible self-directive learning method aimed at forming highly motivated students thus enabling them to design their own learning approach according to their individual interests, whether in formal or informal situations. The main focus of this study is within the scope of the design elements of the heutagogy approach namely exploration, creation, collaboration, connection, sharing, and reflection. A total of 102 respondents involving a group of junior students (semester one) and senior students (semester five) from the Diploma in Information Technology program of three polytechnics; Politeknik Kuching Sarawak, Politeknik Muadzam Shah, and Politeknik Kuala Terengganu have provided feedback on the study conducted through a questionnaire that includes 24 items based on a five-point Likert scale. Item validation used the Cronbach Alpha method where the value obtained exceeds 0.80 for each heutagogy element (each element consists of four items). The findings of the study found students’ level of acceptance toward the use of online facilities to support their learning activities is good (mean score above 3.80) but there are significant differences between junior and senior student’s acceptance level in several elements of heutagogy, namely are collaboration, connection, and reflection elements. In conclusion, the heutagogical approach needs to be further strengthened to support joint efforts in producing holistic graduates.

Development of an Integrated Instrument About Critical Thinking Skills and Chemical Literacy Inhydrolysis

This research study aimed to analyse the characteristics of integrated assessment instruments to measure critical thinking and chemical literacy skills in salt hydrolysis material by meeting the criteria of validity, reliability, and level of difficulty. This research utilised the 4-D approach, i.e. define, design, develop, and disseminate. An experimental test was conducted on 175 students. Data analysis was done by analysing the results of item validation, assumption test analysis with Rasch modelling and analysis of an instrument test result. The results of this study showed that there were 30 multiple choice items in the instrument that had been vetted by two experts and four practitioners of education and which yielded 28 valid questions. Based on the result of Rasch modelling assumption, it was found that from 28 items, i.e. 14 questions of code A and 14 questions of code B, only 9 items from code A and 11 items from code B were found to be fit with Partial Credit Model 1-Parameter Logistic (PCM 1-PL). The cronbach alpha reliability coefficients were also computed and were found to be high. . The difficulty level of the items included in code A and code B were also established.

The Case for Cognitive Interviewing in Survey Item Validation: A Useful Approach for Improving the Measurement and Assessment of Substance Use Disorders

Background: In order to accurately identify substance use disorders, we must be confident of our ability to define and measure the construct itself. To date, research has demonstrated that the ways in which substance use disorder criteria are operationalized or assessed can significantly impact the information we obtain from these diagnoses. For example, differing operationalizations of the same construct (e.g., impaired control over substance use) can result in markedly different estimates of prevalence. This points to the need for approaches that aim to improve the validity of diagnostic assessments during the measure development phase. Methods: We performed a scoping review of the cognitive interviewing literature, a technique that aims to provide a systematic way to identify and reduce measurement error associated with the structure and content of interview items. Along with this, we present an applied example with alcohol tolerance. Findings: We argue that cognitive interviewing is well suited for improving the validity of substance use disorder assessment items. Conclusions: We suggest that cognitive interviewing be incorporated in the item generation stage of measure development for substance use disorder assessments.

Survey Item Validation

In the social sciences, validity refers to the adequacy of a survey (or other mode of assessment) for its intended purpose. Validation refers to the activities undertaken during and after the construction of the survey to evaluate and improve validity. Item validation refers here to procedures for evaluating and improving respondents’ understanding of the questions and response options included in a survey. Verbal probing techniques such as cognitive interviews can be used to understand respondents’ response process, that is, what they are thinking as they answer the survey items. Although cognitive interviews can provide evidence for the validity of survey items, they are time-consuming and thus rarely used in practice. The Response Process Evaluation (RPE) method is a newly-developed technique that utilizes open-ended meta-surveys to rapidly collect evidence of validity across a population of interest, make quick revisions to items, and immediately test these revisions across new samples of respondents. Like cognitive interviews, the RPE method focuses on how participants interpret the item and select a response. The chapter demonstrates the process of validating one survey item taken from the Inventory of Non-Ordinary Experiences (INOE).

Survey Item Validation

In the social sciences, validity refers to the adequacy of a survey (or other mode of assessment) for its intended purpose. Validation refers to the activities undertaken during and after the construction of the survey to evaluate and improve validity. Item validation refers here to procedures for evaluating and improving respondents’ understanding of the questions and response options included in a survey. Ver- bal probing techniques such as cognitive interviews can be used to understand respondents’ response process, that is, what they are thinking as they answer the survey items. Although cognitive interviews can provide evidence for the validity of survey items, they are time-consuming and thus rarely used in practice. The Response Process Evaluation (RPE) method is a newly-developed technique that utilizes open-ended meta-surveys to rapidly collect evidence of validity across a population of interest, make quick revisions to items, and immediately test these revisions across new samples of respondents. Like cognitive interviews, the RPE method focuses on how participants interpret the item and select a re- sponse. The chapter demonstrates the process of validating one survey item taken from the Inventory of Non-Ordinary Experiences (INOE).

The Development of an Integrated Assessment Instrument for Measuring Analytical Thinking and Science Process Skills

This research aims to develop instrument and determine the characteristics of an integrated assessment instrument. This research uses 4-D model, which includes define, design, develop, and disseminate. The primary product is validated by expert judgment, tested it’s readability by students, and assessed it’s feasibility by chemistry teachers. This research involved 246 students of grade XI of four senior high schools in Yogyakarta, Indonesia. Data collection techniques include interview, questionnaire, and test. Data collection instruments include interview guideline, item validation sheet, users’ response questionnaire, instrument readability questionnaire, and essay test. The results show that the integrated assessment instrument has Aiken validity value of 0.95. Item reliability was 0.99 and person reliability was 0.69. Teachers’ response to the integrated assessment instrument is very good. Therefore, the integrated assessment instrument is feasible to be applied to measure the students’ analytical thinking and science process skills.