Tackling the Wicked Problem of Measuring What Matters: Framing the Questions

Purpose: Making policy makers, researcher, education leaders, and assessment developers aware that what matters in education assessment is a wicked problem that cannot be easily solved following traditional approaches. Design/Approach/Methods: Starting from the questions that what matters in education assessment, this article presented such questions as a wicked problem because there is no consensus, not right or wrong answer, and certain solutions may lead to side effects on students and society. Therefore, a new approach of ecology should be involved, and different education outcomes or intended qualities of learners are presented in complex relationships. Findings: Deciding what matters in education assessment is a wicked question. It is not a tame or technology problem and can be resolved by any conventional approaches. What is pivotal now is to decipher what matters in education and then what should be measured and ultimately how to measure. The ecology and collaborate approach deliberated in this article could expedite such a process. Originality/Value: This article advocates paradigm change in understanding and resolving one of the most urgent problems in education. It provides an ecology explanation of the relationships that exist among the different education outcomes and students’ qualities. By guiding through the dissecting of the problem step by step, this article has demonstrated a unique angle of understanding the wicked problem.

Download Full-text

Lessons learned from a search for common ground

Phi Delta Kappan ◽

10.1177/0031721721992562 ◽

2021 ◽

Vol 102 (5) ◽

pp. 29-32

Author(s):

Rick Hess ◽

Pedro Noguera

Keyword(s):

School Choice ◽

Common Ground ◽

Social And Emotional Learning ◽

Lessons Learned ◽

Policy Makers ◽

Teachers College ◽

Social And Emotional ◽

Education Leaders ◽

K 12 ◽

Common Understanding

In 2020, Rick Hess and Pedro Noguera engaged in a long-running correspondence that tackled many of the biggest questions in education — including topics like school choice, equity and diversity, testing, privatization, the achievement gap, social and emotional learning, and civics. They sought to unpack their disagreements, better understand one another’s perspectives, and seek places of agreement or points of common understanding. Their correspondence appears in their book, A Search for Common Ground: Conversations About the Toughest Questions in K-12 Education (Teachers College Press, 2021). In this article, they reflect on the exercise, what they learned from it, and what lessons it might offer to educators, education leaders, researchers, and policy makers.

Download Full-text

The Future of Transversal Competencies in Higher Education Assessment

10.4018/978-1-7998-8275-6.ch015 ◽

2022 ◽

pp. 253-268

Author(s):

Jean Cushen ◽

Lauren Durkin

Keyword(s):

Higher Education ◽

Teaching And Learning ◽

Early Stage ◽

University Teaching ◽

Assessment Practices ◽

Education Assessment ◽

Higher Education Assessment ◽

Education Leaders ◽

Higher Education Leaders ◽

Student Centred Learning

This chapter evaluates the rising significance of transversal competencies and the implications for higher education assessment practices. Transversal competencies are expected to play a definitive role in future of work scenarios. This chapter evaluates the decisions and impacts surrounding the integration of transversal competencies into higher education assessments. In particular, the chapter explores the commitments and adjustments that higher education leaders must make to build the competence assessment infrastructure and supports required. The guiding role ‘student-centred learning' pedagogies can play is discussed. Relatedly, early-stage competence frameworks are offered as insight into how student-centred learning can deliver novel, active, reflective assessments that embrace competence diversity and target meaningful development. Finally, a roadmap is offered for higher education leaders to guide them in this challenging but pertinent transformation of university teaching and learning.

Download Full-text

Recidivism Among Justice-Involved Youth: Findings From JJ-TRIALS

Criminal Justice and Behavior ◽

10.1177/0093854820922891 ◽

2020 ◽

Vol 47 (9) ◽

pp. 1059-1078

Author(s):

Angela A. Robertson ◽

Zhou Fang ◽

Doris Weiland ◽

George Joe ◽

Sheena Gardner ◽

...

Keyword(s):

Drug Abuse ◽

Juvenile Justice ◽

County Level ◽

Multilevel Analyses ◽

Recidivism Rates ◽

Policy Makers ◽

Individual Level ◽

Research Questions ◽

Complex Relationships ◽

Using Data

Recidivism, and the factors related to it, remains a highly significant concern among juvenile justice researchers, practitioners, and policy makers. Recent studies highlight the need to examine multiple measures of recidivism as well as conduct multilevel analyses of this phenomenon. Using data collected in a National Institute on Drug Abuse (NIDA)-funded Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS) cooperative agreement, we examined individual- and site-level factors related to 1-year recidivism among probation youth in 20 sites in five states to answer research questions related to how recidivism rates differ across sites and the relationships between individual-level variables and a county-level concentrated disadvantage measure and recidivism. Our findings of large site differences in recidivism rates, and complex relationships between individual and county-level predictors of recidivism, highlight the need for more nuanced, contextually informed, multilevel approaches in studying recidivism among juveniles.

Download Full-text

Teaching the right stuff

Phi Delta Kappan ◽

10.1177/0031721720970697 ◽

2020 ◽

Vol 102 (3) ◽

pp. 23-27

Author(s):

William J. Bennett

Keyword(s):

Content Knowledge ◽

Background Knowledge ◽

Policy Makers ◽

Education Leaders ◽

Working Together ◽

The Right

In this excerpt from How to educate an American: The conservative vision for tomorrow’s schools, William J. Bennett argues that instilling knowledge, not just skills, is an important element of good schooling. In fact, developing certain skills actually requires a considerable amount of background knowledge. He offers examples of states that have made content knowledge an important part of their curricula and urges education leaders and policy makers to make a long-term commitment to working together on such efforts in their own states.

Download Full-text

Addressing the Wicked Problem of American Gun Violence

Journal of Macromarketing ◽

10.1177/0276146717715744 ◽

2017 ◽

Vol 37 (4) ◽

pp. 393-408 ◽

Cited By ~ 7

Author(s):

Aimee Dinnin Huff ◽

Michelle Barnhart ◽

Brandon McAlexander ◽

Jim McAlexander

Keyword(s):

Interest Groups ◽

Violence Prevention ◽

Social Marketing ◽

Internal Marketing ◽

Gun Violence ◽

Wicked Problem ◽

Marketing Tactics ◽

Policy Makers ◽

Consumer Interest ◽

Marketing Tool

Building on work on social and macro-social marketing, we provide an empirical account of ways in which American gun violence prevention groups (GVPGs) act as macro-social marketers as they address the wicked problem of gun violence, which they define as deaths and injuries with firearms. We find that, as a collective, GVPGs attempt to change the culture related to guns by targeting up-, mid-, and downstream agents. We contribute to theory by (1) expanding the concept of macro-social marketing beyond government entities to include consumer interest groups and collectives; (2) introducing internal marketing as a macro-social marketing tool critical for macro-social marketers dependent largely on volunteers; (3) elucidating ways that macro-social marketers can accomplish upstream changes indirectly, by encouraging consumers and citizens to influence policy makers; and (4) revealing marketing tactics that can be leveraged across up-, mid-, downstream, and internal efforts.

Download Full-text

Building Back Better after COVID-19: a systematic scoping review of wicked problems affecting developed countries and implications for global governance

10.1101/2021.04.26.21256126 ◽

2021 ◽

Author(s):

Robin van Kessel ◽

Brian Li Han Wong

Keyword(s):

Public Health ◽

Demographic Change ◽

Scoping Review ◽

Wicked Problems ◽

Developed Countries ◽

Wicked Problem ◽

List Type ◽

Driving Mechanisms ◽

New Findings ◽

Complex Relationships

AbstractThe COVID-19 pandemic is a textbook example of a ‘wicked problem’, one which is complex, open-ended, unpredictable, or intractable and seems resistant to any solution. This presents a window of opportunity to explore other wicked problems and their implications after the pandemic. A systematic scoping review was conducted to investigate the COVID-19 aftermath and identify public health topics which may be of great significance in the years to come. Through the adoption of three megadrivers as fundamental drivers of change (globalisation, demographic change, and digitalisation), it narratively explored how different wicked problems – and the driving mechanisms which sustain them – persist. It further explored the implications of these public health topics on global (health) governance. While the wicked problems mapped in this article show a large variance in where their apparent roots lie, they share one factor in common: health. These wicked problems must be first and foremost addressed if we as a globalised world are to successfully and sustainably build back better from COVID-19.Summary BoxWhat is already known?-COVID-19 is a textbook example of a wicked problem; a problem that is complex, open-ended, unpredictable, or intractable and seems resistant to any solution;-Digitalisation, globalisation, and demographic change are seen as the three megadrivers of change and are theorised to create and sustain modern wicked problems;-The megadrivers respectively have complex relationships with health; this includes both positive and negative associations.What are the new findings?-A series of wicked problems exist in multiple domains of society that continue to obstruct the progress of the 2030 Agenda for Sustainable Development;-These wicked problems each have roots in the three megadrivers: digitalisation, globalisation, and demographic change;-There exists a complex interrelation between the megadrivers, wicked problems, and health.What do the new findings imply?-It currently remains unclear to what extent wicked problems and megadrivers respectively affect health outcomes; therefore, further research is indicated;-A ‘once in a generation’ opportunity has presented itself to build back better from the COVID-19 pandemic by addressing the existing wicked problems; therefore, governance mechanisms should follow and adapt accordingly;-Wicked problems have bidirectional implications for modern policy; this has created an environment for wicked problems to manifest and sustain themselves, which in turn produce further policies that sustain wicked problems.

Download Full-text

Competing With Charter Schools: Selection, Retention, and Achievement in Los Angeles Pilot Schools

Evaluation Review ◽

10.1177/0193841x20946221 ◽

2020 ◽

Vol 44 (2-3) ◽

pp. 111-144

Author(s):

Caitlin Kearns ◽

Douglas Lee Lauen ◽

Bruce Fuller

Keyword(s):

Public Schools ◽

Los Angeles ◽

Charter Schools ◽

Low Income ◽

English Language ◽

Public Funding ◽

Traditional Schools ◽

Policy Makers ◽

Education Leaders ◽

Lower Test

Charter schools place competitive pressure on school districts to retain students and public funding. Many districts also have moved to decentralize control of budgets and teacher hiring down to school principals, independent of competitive pressures. But almost no evaluation evidence gauges the effectiveness of charter-like schools, relative to traditional public schools. We find that autonomous pilot schools in Los Angeles enroll more low-income and Spanish-speaking students, compared with traditional schools. Pilot pupils are significantly less likely to exit the school district. But pilot pupils displayed lower test scores in mathematics and fell slightly below traditional students in English-language arts, taking into account prior performance and their propensity to enter pilot schools. We tracked 6,732 students entering pilot high schools between 2008 and 2012, statistically matched in multiple ways with traditional peers from identical sending middle schools. We discuss the advantages of our evaluation strategy and the implications of our findings for education leaders and policy makers.

Download Full-text

“Like a Sheriff in a Small Town”: Status, Roles, and Challenges of Ethics Committees in Academic Colleges of Education

Journal of Empirical Research on Human Research Ethics ◽

10.1177/15562646211005253 ◽

2021 ◽

pp. 155626462110052

Author(s):

Daniel Sperling

Keyword(s):

Research Ethics ◽

Ethics Committees ◽

Formal Training ◽

Colleges Of Education ◽

Research Ethics Committees ◽

Ministry Of Education ◽

Policy Makers ◽

Academic Management ◽

Complex Relationships ◽

Shed Light

In recent years, Research Ethics Committees in academic colleges of education have constituted to review research proposals in the field of education. Yet, little is known about their work, composition, challenges, and relationships with external partners. This study explores the views and attitudes of 13 members and chairpersons of Research Ethics Committees in colleges of education in Israel, and two policy makers at the Ministry of Education about their roles, responsibilities, challenges, and limitations. Findings revealed an instrumental attitude towards the ethics committee. Committees are perceived as supportive rather than enforcing. Interviews shed light on the complex relationships between committee members, college lecturers/researchers, ethics regulators, and academic management. Moreover, the findings emphasized the lack of formal training and broad discussion on ethics. The study calls for strengthening committees’ raison d'être and the internalization of ethics among committee members, researchers, and lecturers in the field of education.

Download Full-text

Developing a Preliminary Causal Loop Diagram for Understanding the Wicked Complexity of the COVID-19 Pandemic

Systems ◽

10.3390/systems8020020 ◽

2020 ◽

Vol 8 (2) ◽

pp. 20 ◽

Cited By ~ 5

Author(s):

Oz Sahin ◽

Hengky Salim ◽

Emiliya Suprun ◽

Russell Richards ◽

Stefen MacAskill ◽

...

Keyword(s):

Systems Approach ◽

Unintended Consequences ◽

Wicked Problem ◽

Causal Loop Diagram ◽

Fundamental Change ◽

Negative Consequences ◽

Policy Makers ◽

Leverage Points ◽

Loop Diagram ◽

Causal Loop

COVID-19 is a wicked problem for policy makers internationally as the complexity of the pandemic transcends health, environment, social and economic boundaries. Many countries are focusing on two key responses, namely virus containment and financial measures, but fail to recognise other aspects. The systems approach, however, enables policy makers to design the most effective strategies and reduce the unintended consequences. To achieve fundamental change, it is imperative to firstly identify the “right” interventions (leverage points) and implement additional measures to reduce negative consequences. To do so, a preliminary causal loop diagram of the COVID-19 pandemic was designed to explore its influence on socio-economic systems. In order to transcend the “wait and see” approach, and create an adaptive and resilient system, governments need to consider “deep” leverage points that can be realistically maintained over the long-term and cause a fundamental change, rather than focusing on “shallow” leverage points that are relatively easy to implement but do not result in significant systemic change.

Download Full-text

Continuing to enhance the quality of case study methodology in health services research

Healthcare Management Forum ◽

10.1177/08404704211028857 ◽

2021 ◽

pp. 084047042110288

Author(s):

Shannon L. Sibbald ◽

Stefan Paciocco ◽

Meghan Fournie ◽

Rachelle Van Asseldonk ◽

Tiffany Scurr

Keyword(s):

Health Services Research ◽

Health Services ◽

Research Process ◽

Services Research ◽

Policy Makers ◽

Case Study Methodology ◽

Study Methodology ◽

Complex Relationships

Case study methodology has grown in popularity within Health Services Research (HSR). However, its use and merit as a methodology are frequently criticized due to its flexible approach and inconsistent application. Nevertheless, case study methodology is well suited to HSR because it can track and examine complex relationships, contexts, and systems as they evolve. Applied appropriately, it can help generate information on how multiple forms of knowledge come together to inform decision-making within healthcare contexts. In this article, we aim to demystify case study methodology by outlining its philosophical underpinnings and three foundational approaches. We provide literature-based guidance to decision-makers, policy-makers, and health leaders on how to engage in and critically appraise case study design. We advocate that researchers work in collaboration with health leaders to detail their research process with an aim of strengthening the validity and integrity of case study for its continued and advanced use in HSR.

Download Full-text