Assessing Screenshots: An in-school evaluation of a middle-school digital citizenship curriculum (Preprint)
BACKGROUND Screenshots is an in-school curriculum that uses aspects of digital citizenship to develop the emerging digital social skills of middle-schoolers with the long term goal of improving their health and well-being. The program seeks to create a knowledge base on which young adolescents can build a set of beliefs and behaviors that foster respectful interactions, prosocial conflict resolutions, and safe and secure uses of communication technology. Intervening in this way can improve mental health by limiting their exposure to cyberbullying and other forms of negative online interactions. This study reports on an evaluation of Screenshots conducted with 7th graders in a public school system of a mid-size New England City. OBJECTIVE The goal of this study is to determine the effectiveness of the Screenshots program in increasing participants’ knowledge about key concepts of digital citizenship and in shifting beliefs and intended behaviors to align with pro-social, respectful, and safe online interactions. Additionally, the study examines the extent to which the program has differing effects for boys and girls in terms of their conflict and bullying resolution strategies. METHODS This quasi-experimental evaluation was conducted in four middle schools in which one group of 7th graders received the Screenshots curriculum and another did not. Before and after the curriculum, all students completed a questionnaire that measured their knowledge of and beliefs about digital citizenship and related online behavioral concepts, their attitudes regarding strategies for stopping online bullying, and their intended online conflict resolution behaviors. RESULTS The sample included 92 students who received the curriculum and 71 in the comparison group. Pre- to post-test retention rates ranged from 52.4% to 84% varying by school and condition. Results showed an increase in knowledge about key curricular concepts for some students (F1,32= 9.97, P = .003). In response to some individual items, student increased their beliefs supportive of online privacy (F1,42= 4.389, P=.04) and safety (F1,76= 2.79, P=.099) compared to the comparison group. Gender moderated the results related to conflict resolution with some boys reducing their endorsement of an aggressive option (F2,40= 5.77, P = .006), and some girls increasing their tendency to pursue a non-aggressive option (pre-test=3.83, post-test=3.58). Participants, on average, reported learning something new from the classes. CONCLUSIONS This study represents a rare evaluation of an in-school digital citizenship program and demonstrates the effectiveness of Screenshots. Students’ increased knowledge of key curricular concepts represents a foundation on which to develop future beliefs and healthy behaviors. Differences in how boys and girls experience and perpetrate online aggression likely explain the conflict resolution findings and emphasizes the need to examine gender differences in response to these programs. Students high rating of the relevance of Screenshot’s content reinforces the need for this type of intervention.