Credibility contests: The contributions of experiential knowledge to radicalisation expertise

Radicalisation has become a highly influential idea in British policy making. It underpins and justifies Prevent, a core part of the UK's counter-terrorism strategy. Experts have theorised the radicalisation process, often beset by a weak evidence base and mired in fundamental contestation on definitions and explanatory factors. Experiential experts have been active contributors to these debates, presenting a challenge to the low-ranking role often given to experiential knowledge in evidence hierarchies and a contrast to policy areas in which it remains poorly valued. This paper draws on interviews with radicalisation experts to examine the dynamics of this pluralisation in practice. With a focus on credibility contests, it explains how experiential experts can claim authoritative knowledge and the challenges they face from those who prioritise theory-driven empirical data as the basis for contributions to knowledge. The paper draws out the implications for understandings of expertise of this newly conceptualised, evidence poor and highly applied topic area.

On the Meaning of Medical Evidence Hierarchies

Evidence hierarchies are investigative strategies ordered with regard to the claimed strength of evidence. They have been used for a couple of decades in EBM, particularly in assessing evidence for treatment recommendations, but remain controversial. An under-investigated question is what the order in the hierarchy means. Four interpretations are discussed here. The two most credible are “typically stronger” or “ideally stronger.” The well-known GRADE framework seems to assume some “typically stronger” reading. Even if the interpretation of an evidence hierarchy were established, hierarchies are rather unhelpful for the task of evidence aggregation. Specifying the intended order relation may help to sort out disagreements.

John, Peter (2018). How Far to Nudge? Assessing Behavioural Public Policy. Cheltenham (New Horizons in Public Policy), UK/Northampton, MA, USA: Edward Elgar. 173 Seiten. ISBN: 9781786430564

In the past decade, interventions informed by behavioural economics and psychology have spread across jurisdictions and policy areas. Worldwide, more than one hundred organizations and networks are developing and implementing nudges and other behavioural tools. After an initial phase of curiosity, attention is now shifting to the varieties of behavioural public policy, its institutional and cultural embeddedness, its impact and limitations. In his most recent book, Peter John explores some of the crucial questions related to this next phase of nudge. He discusses the role of nudge units, the limitations of behavioural approaches and the ethics of nudge. Most importantly, John proposes a deliberative and reflective version of nudging, nudge plus. Readers might miss an in-depth discussion of pressing problems such as the globalizing influence of behavioural expertise, the imperialism of evidence hierarchies and the political repercussions of nudging. Despite these deficits, the book will inspire both further research and critical debates.

Realist Synthesis for Public Health: Building an Ontologically Deep Understanding of How Programs Work, For Whom, and In Which Contexts

Realist synthesis is a literature review methodology for understanding how, for whom, and under what circumstances complex interventions function in complex environments. Using a heuristic called the context-mechanism-outcome (CMO) configuration, realist synthesis produces evidence-informed theories about the interactions between intervention mechanisms and their implementation contexts. Public health interventions and their effects unfold over time and develop differently in different contexts. Much of what causes programs to function remains in a realm beyond empirical measurement. By scrutinizing the theories relevant to the interventions of interest, and by enhancing the conceptualization of complex outcomes, the realist approach deprioritizes evidence hierarchies and harnesses insights from diverse data sources to generate causal understanding. The dynamic learning process that can arise in conducting a realist synthesis may generate new ideas for program development and innovation apart from what can be achieved in reviews providing a summation and aggregation of quantified evidence.

Plural Methods, One Causation

No single method is perfect for identifying causation. One response is to adopt causal pluralism: the view that causation is many things. But this is problematic. A simpler response is to take such pluralism as epistemic or methodological only. If we cannot rely on one method alone, we instead have the option of approaching causation via a number of different methods. A method is useful if it attaches to one or more of the symptoms of causation, where those symptoms are also plural. Such symptoms are the more or less reliable indicators of the presence of causation. Evidence hierarchies could be based on how reliable each such method is but hierarchies can themselves be understood in a dispositional rather than strict way.

What is mechanistic evidence, and why do we need it for evidence-based policy?

It has recently been argued that successful evidence-based policy should rely on two kinds of evidence: statistical and mechanistic. The former is held to be evidence that a policy brings about the desired outcome, and the latter concerns how it does so. Although agreeing with the spirit of this proposal, we argue that the underlying conception of mechanistic evidence as evidence that is different in kind from correlational, difference-making or statistical evidence, does not correctly capture the role that information about mechanisms should play in evidence-based policy. We offer an alternative account of mechanistic evidence as information concerning the causal pathway connecting the policy intervention to its outcome. Not only can this be analyzed as evidence of difference-making, it is also to be found at any level and is obtainable by a broad range of methods, both experimental and observational. Using behavioral policy as an illustration, we draw the implications of this revised understanding of mechanistic evidence for debates concerning policy extrapolation, evidence hierarchies, and evidence integration.

Down with the Hierarchies

An evidence hierarchy is a rank ordering of kinds of research methods according to the potential for those methods to suffer from systematic bias. Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. Chapter 5 gives several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. Medical science should not employ evidence hierarchies, including even the latest and most sophisticated of such hierarchies. There is an advantage to evidence hierarchies, namely, their apparent rigidity, which could constrain researcher degree of freedom when performing a review of available evidence. But this constraint comes at the cost of an overly simple conception of evidence, and does not in fact constrain researcher degree of freedom in a substantive way.

A Commentary on the Role of Randomized Controlled Trials in Massage Therapy

Some massage therapists (MTs) view research as a way to demonstrate to other healthcare professionals (OHPs) that massage therapy is safe and effective and should be an integral part of patients’ health care. This desire for credibility through research, however, requires studies that are acceptable to medical professionals. Therefore, researchers have begun to study massage therapy, primarily using randomized controlled trials (RCTs). Many of the RCTs of massage therapy, rather than proving efficacy, have been met with criticism, including their lack of reproducibility and lack of a suitable control. The belief that RCTs will save the profession of MT, or any health care practice, by proving treatments work, is unfounded. Evidence hierarchies suggest that practitioners should accept the results of RCTs, or the systematic review of RCTs, as the gold standard for efficacy research. Privileging one methodology over another does not use the benefits of the multiple approaches to research available. Researchers should consider whether there are other methodologies that allow for rigorous investigation of massage therapy in a way that would be useful for stakeholders of this research. It is only through research that is rigorously and authentically conducted that the credibility of massage therapy will be established.

Why, in medicine, evidence is still more important than narratives or values? Or “The importance of being evident.”

The importance of being evident is what counts for medicine. The diagnosis must be evident and the treatment must be based on evidence. If that evidence is and always must be based on statistics, as we have seen, it becomes questionable. Evidence is good when it is robust and when it fits the individual patient. Only then does evidence-based medicine (EBM), make sense and only then the patient can be sure to be treated in the best possible way. EBM, the movement of medicine that is strictly based on evidence which is judged in a hierarchical order, is under scrutiny and heavily criticised, chiefly because it has lost the patient out of its focus. Numbers are more important than the individual diagnosis and treatment is administered according to population-based statistics and not ‘made to order’. Although this criticism is very valid, the solution cannot be to simply replace EBM with something else, but the solution must be to still base medicine and medical treatment on the best available evidence we have, while putting the patient back into focus. In order to do so, it is important to topple evidence hierarchies, to divide EBM into research and practice and to acknowledge that sometimes the statistical best evidence is not the best evidential treatment for the actual patient.