21st Century Learner Self-Report Feedback Inventories: A review and defence of psychometric approaches

2021 ◽  
Author(s):  
Gavin T. L. Brown ◽  
Anran Zhao
Keyword(s):  
21St Century ◽  
Psychological Control ◽  
Self Regulation ◽  
Experimental Studies ◽  
Self Report ◽  
Current Evidence ◽  
Validity And Reliability ◽  
Effective Feedback ◽  
Self Reports ◽  
The Impact

Interest in how learners understand, respond to, and value educational feedback has increased notably in the last two decades. Much of the current evidence of the impact or influence of feedback arises from self-report inventories. Self-report respondents are presumed to be well situated to report on their own experiences, attitudes, and behaviours. However, considerable validity threats exist in this method, which may limit the meaningfulness of self-reports. Consequently, many researchers are concerned about the disproportionate role self-reports play in informing our understanding of effective feedback. The validity of claims about learner self-reports on feedback depend on the credibility of the measures used. Inventories developed within the psychometric tradition are expected to provide theoretical and empirical evidence for the validity and reliability of the measures to support subsequent interpretations and decisions. This review provides a defense of the potential value of psychometric self-report data in informing the psychology of effective feedback. A systematic review of 14 feedback self-report inventories published in the 21st century found 25 empirical studies that replicate one or more of these inventories. Thirteen survey and 12 experimental studies were reviewed to provide a thematic analysis of the state of research. Seven studies used latent theory modeling to test the robustness of the MIMIC-based inventories, relying instead on estimates of scale reliability. Eight of 25 studies reported full replication of the inventory. Just four studies had an independent measure (academic achievement), with the vast majority of studies relating feedback inventories to self-reported constructs. Most effects were small and used small samples, which weakens reproducibility. The data showed that feedback perceptions have complex overlap with psychological control and competence beliefs, largely consistent with self-regulation of learning theory. Psychometric methods can improve the quality of research into how students understand, experience, think about, and use feedback that is given to them.

scholarly journals Implications of Disagreement Between Self-Reporting and Objective Measures: A Scoping Review

2018 ◽  
Vol 10 (6) ◽  
pp. 199
Author(s):  
Hilal Al Shamsi ◽  
Abdullah Ghthaith Almutairi ◽  
Sulaiman Salim Al Mashrafi
Keyword(s):  
Risk Factors ◽  
Health Services ◽  
Scoping Review ◽  
Emotional Health ◽  
Self Report ◽  
Objective Measures ◽  
Utilization Of Health Services ◽  
Significant Difference ◽  
Self Reports ◽  
The Impact

INTRODUCTION: Researchers and health specialists generally collect data and information about chronic diseases from self-reports. However, the accuracy of self-reports has been questioned as they depend on the respondents' ability to recall information and their understanding of pathological conditions. Therefore, an objective diagnosis is usually regarded as a more accurate indication of the presence of diseases.OBJECTIVE: A scoping review will examine the extent of the disagreement between self- reports and objective measures, focusing on the implications of this disagreement in terms of indicators of physical and emotional health as well provision and planning of health services.METHOD: There are few publications on the impact of disagreements between self-reporting and objective measures. In this case, a scoping review was chosen as an efficient tool to explore the issue, due to the limited amount of available evidence. This review was conducted in two major research databases: Scopus and Medline databases. The criteria of the study included all genders, age groups, and geographic areas. The source of information for the scoping review included existing literature such as guidelines, letters, meta-analyses, systematic reviews, and primary research studies.RESULT: In the 12 studies, the total participants were 155,939 and each study’s sample size ranged from 77 to 118,553. Four out of twelve studies showed a significant difference between self-reported ailments and objective diagnosis for (kappa=0.17 to 0.3), whereas the agreement was moderate for the utilization of health services and quality of ambulatory care (kappa=0.43 to 0.5), however, the agreement on whether counselling and referrals were needed was low (kappa= 0.3, 95% CI [0.3-0.3]). The disagreements between self-report and objective measures had implications regarding prevalence of diseases (20% less by self-reported) or risk factors (such as physical activity [PA]), costs of treatments (15 EUR high by reports), risk factors such as car accidents for elderly (useful field of view in elderly drivers was a risk over four times larger than obtained from self-reported [OR= 13.7 vs OR=3.4]), and utilization of health services (34.1% higher by reported).CONCLUSION: In most health domains, we found there was low to moderate disagreement between self-reporting and objective measures for diagnosing illnesses and utilization of health services.  The prevalence of disease was lower when self-reported, while the utilization of health services and cost of health services were higher when self-reported than when objectively measured. This disagreement has implications regarding the increasing the cost of health services and provides a misleading basis for health planning.

The Effect of Item Order on Psychological Aggression Reporting: An Examination With the Multidimensional Measure of Emotional Abuse

Partner Abuse ◽  
2016 ◽  
Vol 7 (2) ◽  
pp. 125-139 ◽  
Author(s):  
Ryan C. Shorey ◽  
Catherine Strauss ◽  
William C. Woods ◽  
Tara L. Cornelius
Keyword(s):  
Partner Violence ◽  
Emotional Abuse ◽  
Reliability And Validity ◽  
Psychological Aggression ◽  
Self Report ◽  
Order Effects ◽  
Multidimensional Measure ◽  
Item Order ◽  
Self Reports ◽  
The Impact

Psychological aggression between intimate partners has been found to occur at high rates in college populations, but the reliability and validity of the self-report measures used to gather this information have been criticized. Some studies indicate item order may affect self-reports of victimization and perpetration of violent behaviors. In this study, we administered standardized and randomized formats of the Multidimensional Measure of Emotional Abuse (MMEA; Murphy & Hoover, 1999) to study the impact question order had on self-reports of psychological aggression victimization and perpetration. Results demonstrated that women reported more frequent perpetration and victimization than men on some MMEA subscales. Furthermore, those who took the standardized format of the MMEA reported more frequent perpetration and victimization on some MMEA subscales than those who took the randomized format. However, no significant interaction effects were found between gender and format. Overall, these mixed results warrant further research on item order effects with self-report measures of intimate partner violence.

scholarly journals Impact of teacher training on academic self-concept and educational outcomes

2019 ◽  
Vol 17 (49) ◽  
Author(s):  
Karla Lobos Peña ◽  
Claudio Bustos ◽  
Alejandro Díaz
Keyword(s):  
Teacher Training ◽  
Training Program ◽  
Experimental Studies ◽  
Self Report ◽  
Self Concept ◽  
Control Group ◽  
Teacher Training Program ◽  
Quasi Experimental ◽  
The Impact ◽  
Experimental Group

Introduction. Current research based on observational design studies characterized the teaching practices that stimulate the development of academic self-concept, but there is a need for experimental and quasi-experimental studies to validate those findings. The present research evaluate the impact of a teacher training program based on academic self-concept stimulation strategies on academic self-concept, attendance and grades of their students. Method. 36 teachers of primary level and 819 students from public schools of the Province of Concepción, Chile, participated. A quasi-experimental design with pre and post measures was applied. Information was collected through self-report and information available on official records. Results. After the intervention, teachers in the experimental group have greater capacity to stimulate the self-concept of their students, D(1, 41700.91)=17.09, p < 0.001, encouring self-concept related to capacity, work procedures and class participation. Also, the students on experimental group have a greater perception about their capacity to participate in classes after the intervention, dependent on the pre-test perceived capacity, D(1, 983-22)=6.10, p=0.014,  and higher attendance rates, D(1, 691.34)=82.50, p<0.001, than control group. Discussions and conclusions. The teacher training program is effective to improve the teaching strategies of stimulating the academic self-concept, the perception of competence to participate in classes of the students and to increase the levels of attendance.

scholarly journals A systematic review and meta-analysis of studies of reactivity to objective digital measurement of health behaviour

2021 ◽  
Author(s):  
Laura M König ◽  
Anila Allmeta ◽  
Nora Christlein ◽  
Miranda Van Emmenis ◽  
Stephen Sutton
Keyword(s):  
Systematic Review ◽  
Health Behaviour ◽  
Meta Analysis ◽  
Self Regulation ◽  
Experimental Studies ◽  
Health Behaviours ◽  
Self Report ◽  
Digital Measurement ◽  
Measurement Reactivity ◽  
Digital Measurements

Self-report measures of health behaviour have several limitations including measurement reactivity, i.e. changes in people’s behaviour, cognitions or emotions due to taking part in research. It is thus often recommended to use objective digital measurements instead; however, it is unclear whether they also induce reactivity. Therefore, this pre-registered systematic review and meta-analysis synthesised 31 studies, including 7 experimental studies with 18 effects. Most studies investigated reactivity to digital measurement of physical activity, reporting small but significant effects. Measurement reactivity may be amplified by (1) ease of changing the behaviour, (2) awareness of being measured and social desirability, and (3) resolving discrepancies between actual and desired behaviour through self-regulation. Measurement reactivity may be reduced by discarding the first days of measurement and by restricting visibility of the data. Studies on reactivity to measurement of other health behaviours were scarce, indicating the need for future rigorous experimental research.

scholarly journals Physiological, emotional and neural responses to visual stimuli in eating disorders: a review

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Victoria Burmester ◽  
Esme Graham ◽  
Dasha Nicholls
Keyword(s):  
Eating Disorders ◽  
Eating Disorder ◽  
Physiological Effect ◽  
Self Report ◽  
Main Body ◽  
Neural Responses ◽  
Physiological Reaction ◽  
Self Reports ◽  
The Impact ◽  
Selection Of

Abstract Background Overconcern with food and shape/weight stimuli are central to eating disorder maintenance with attentional biases seen towards these images not present in healthy controls. These stimuli trigger changes in the physiological, emotional, and neural responses in people with eating disorders, and are regularly used in research and clinical practice. However, selection of stimuli for these treatments is frequently based on self-reported emotional ratings alone, and whether self-reports reflect objective responses is unknown. Main body This review assessed the associations across emotional self-report, physiological, and neural responses to both food and body-shape/weight stimuli in people with anorexia nervosa (AN), bulimia nervosa (BN) and binge eating disorder (BED). For food stimuli, either an aversive or lack of physiological effect was generated in people with AN, together with a negative emotional response on neuroimaging, and high subjective anxiety ratings. People with BN showed a positive self-rating, an aversive physiological reaction, and a motivational neural response. In BED, an aversive physiological reaction was found in contrast to motivational/appetitive neural responses, with food images rated as pleasant. The results for shape/weight stimuli showed aversive responses in some physiological modalities, which was reflected in both the emotional and neural responses, but this aversive response was not consistent across physiological studies. Conclusions Shape/weight stimuli are more reliable for use in therapy or research than food stimuli as the impact of these images is more consistent across subjective and objective responses. Care should be taken when using food stimuli due to the disconnect reported in this review.

scholarly journals The Impact of Validity Screening on Associations Between Self-Reports of Bullying Victimization and Student Outcomes

2016 ◽  
Vol 78 (1) ◽  
pp. 80-102 ◽  
Author(s):  
Yuane Jia ◽  
Timothy R. Konold ◽  
Dewey Cornell ◽  
Francis Huang
Keyword(s):  
School Quality ◽  
National Policy ◽  
Self Report ◽  
Bullying Victimization ◽  
Student Adjustment ◽  
Adolescent Risk Behavior ◽  
Adolescent Risk ◽  
Self Reports ◽  
Additional Support ◽  
The Impact

Self-report surveys are widely used to measure adolescent risk behavior and academic adjustment, with results having an impact on national policy, assessment of school quality, and evaluation of school interventions. However, data obtained from self-reports can be distorted when adolescents intentionally provide inaccurate or careless responses. The current study illustrates the problem of invalid respondents in a sample ( N = 52,012) from 323 high schools that responded to a statewide assessment of school climate. Two approaches for identifying invalid respondents were applied, and contrasts between the valid and invalid responses revealed differences in means, prevalence rates of student adjustment, and associations among reports of bullying victimization and student adjustment outcomes. The results lend additional support for the need to screen for invalid responders in adolescent samples.

scholarly journals Personality Change Through Arts Education: A Review and Call for Further Research

2020 ◽  
Author(s):  
Michael P. Grosz ◽  
Julia Lemp ◽  
Beatrice Rammstedt ◽  
Clemens Lechner
Keyword(s):  
Personality Traits ◽  
Middle Childhood ◽  
Arts Education ◽  
Personality Development ◽  
Experimental Studies ◽  
Past Research ◽  
Personality Change ◽  
Current Evidence ◽  
Current Evidence Base ◽  
The Impact

Education involving active engagement in the arts, herein called arts education, is often believed to foster the development of desirable personality traits and skills in children and adolescents. Yet, the impact of arts education on personality development has rarely been systematically investigated. In the current paper, we reviewed the literature on personality change through arts education. We identified 36 suitable experimental and quasi-experimental studies. Evidence from these studies tentatively suggest arts education programs can foster personality traits such as extraversion and conscientiousness but not self-esteem. Also, the effects of arts education appeared to be stronger in early and middle childhood than in preadolescence and early adolescence. However, the evidence for the effectiveness of arts education was very limited among the few included true experiments. Furthermore, the reviewed studies were heterogenous and subject to content-related, methodological and statistical limitations. Thus, the current evidence base is inconclusive as to the effects of arts education on personality development. By identifying potential effects of arts education and limitations of past research, our review serves as a call for more research and guidepost for future studies on arts education and personality change.

scholarly journals A Longitudinal Examination of Perceived Parent Behavior and Positive Youth Development: Child-Driven Effects

2021 ◽  
Vol 10 (10) ◽  
pp. 369
Author(s):  
Goda Kaniušonytė ◽  
Mary Page Leggett-James ◽  
Brett Laursen
Keyword(s):  
Youth Development ◽  
Positive Youth Development ◽  
21St Century ◽  
Psychological Control ◽  
Positive Development ◽  
Self Reports ◽  
Role Of Parents ◽  
Longitudinal Examination ◽  
Insight Into

The advent of the 21st Century brought a new interest in promoting Positive Youth Development and a renewed emphasis on understanding transactional relations between parenting and adolescent development. The present study examined conventional parent-driven pathways, which describe the putative role of parents in the formation of positive characteristics in children, as well as the prospect of child-driven effects, which describe how parents respond to evidence of Positive Youth Development by potentially increasing support and reducing psychological control. We tested these pathways in a sample of 458 Lithuanian adolescents (52.2% girls; M = 15.14 years old at the outset) who completed surveys assaying perceptions of parent behaviors and self-reports of positive development (character, competence, connection, caring, and confidence) at annual intervals from ages 15–18. Across most lags, children’s perceptions of parenting changed in response to their own positive development with increased support and decreased psychological control. In contrast, there were no longitudinal associations from perceptions of parenting to subsequent Positive Youth Development. The results offer insight into parenting in the 21st Century, a time when youth are increasingly encouraged/required to acquire volunteer experiences designed to promote positive development. To the extent that these experiences are successful, one unexpected offshoot may be better relationships with parents.

The Development of a Measure of Self-Regulated Practice Behavior for Beginning and Intermediate Instrumental Music Students

2011 ◽  
Vol 59 (4) ◽  
pp. 321-338 ◽  
Author(s):  
Peter Miksza
Keyword(s):  
Construct Validity ◽  
Predictive Validity ◽  
Time Management ◽  
Instrumental Music ◽  
Social Influences ◽  
Self Regulation ◽  
Self Report ◽  
Validity And Reliability ◽  
Confirmatory Factor ◽  
Practice Habits

The purpose of this study was to develop and test the construct validity and reliability of a self-report measure of self-regulated practice behaviors for beginning and intermediate instrumentalists. A questionnaire was designed to assess the motive, method, behavior, time management, and social influences dimensions of the theoretical model of self-regulation proposed by McPherson and Zimmerman. The questionnaire’s construct validity was tested using confirmatory factor analysis, and a preliminary assessment of predictive validity was estimated by correlating the measure with self-reported practice habits. The measure’s reliability in regard to internal consistency and consistency over time was assessed as well. The sample consisted of middle school band students in grades 6 to 8 ( N = 302). Confirmatory factor analyses indicated that a model including factors representing the dimensions self-efficacy, method/behavior combined, time management, and social influences was the best fit. Cronbach’s alpha and test–retest reliability results indicated good to excellent consistency across all self-regulation subscales, with coefficients ranging from .76 to .90. Significant correlations ( p < .001) between the self-regulation subscales and self-reported practice habits (i.e., time spent practicing, average daily practice efficiency, percentage of time spent on formal practice) provided preliminary evidence of predictive validity of the measure.

scholarly journals Parental Distress and Perception of Children’s Executive Functioning after the First COVID-19 Lockdown in Italy

2021 ◽  
Vol 10 (18) ◽  
pp. 4170
Author(s):  
Concetta Polizzi ◽  
Sofia Burgio ◽  
Gioacchino Lavanco ◽  
Marianna Alesi
Keyword(s):  
Executive Functioning ◽  
Online Survey ◽  
Pearson Correlation ◽  
Self Regulation ◽  
Self Report ◽  
Parental Distress ◽  
Parental Competence ◽  
Increased Risk ◽  
Novel Coronavirus ◽  
The Impact

The spread of the novel coronavirus (COVID-19), and the consequential first italian lockdown to minimize viral transmission, have resulted in many significant changes in the every-day lives of families, with an increased risk of parental burnout. This study explores the impact of the first COVID-19 lockdown in Italy on parental distress and parental perceptions of children’s executive functions (EFs). Participants were 308 Italian parents with children between 4 and 17 years of age; they were recruited through online advertisements on websites and social media, and they were given an online survey. The measures were: the balance between risks and resources (BR2) and the executive functioning self-report (EF). Findings of the study suggest that the most distressed parents perceived their children as less competent in EF, highlighting a cognitive fragility on attention, memory, and self-regulation (Pearson correlation coefficient, p < 0.05); significant differences were found between parents of children exhibiting typical and atypical patterns of development (ANOVA, p < 0.05). The study reinforces the need to provide families with psychological aid to support parental competence in restrictive lockdown conditions.

Sign in / Sign up

Export Citation Format

Share Document