When evaluating a complex intervention, a good theoretical understanding of the intervention is needed. Theory-based evaluation is an approach which describes the hypothetical pathways through which an intervention is thought to achieve its effects (‘causal mechanisms’) and then evaluates whether these causal pathways occurred in practice. A theory-based approach considers how change occurs through the interplay between mechanisms and context.
Programme Theory
Fundamental to the implementation of a theory-based evaluation approach is the specification of the Programme Theory. The Programme theory describes ‘how an intervention is expected to lead to its effects and under what conditions’. The theory outlines the key components of the intervention and the mechanisms of the intervention (how the components interact) along with features of the context that are expected to influence mechanisms, and how mechanisms might influence context.
Developing a programme theory leads to a shared understanding of the intervention and can help to identify key uncertainties and research questions. The programme theory should be developed prior to the implementation of an intervention and be developed in collaboration with the relevant stakeholders, drawing on different sources of information, including academic theories and evidence from a range of disciplines, programme experience, and implementer and beneficiary insights.
During implementation, programme theory can be used for management and monitoring. When the intervention is being evaluated, programme theory guides the identification of appropriate populations for data collection and the selection of outcomes. Once developed, the programme theory should be a living document to be refined over time. It is an important resource for evaluation and future transferability across settings.
Using illustrations for Programme Theory
A common approach to displaying the programme theory is to illustrate the intervention components and the hypothesised causal pathways to the outcomes of interest in a diagram/visual aid.
The diagram depicts all the activities that need to happen while carrying out the intervention and how different changes occur, by mapping out hypothesised causal pathways. It sets out both the process, the intended outcomes of activities, and the overall impact of the intervention. Visual aids also describe the context in which these processes occur, the contextual factors that are likely to interact with the intervention, as well as the assumptions underlying whether they will indeed play out as expected. Potential unintended or harmful consequences should also be considered.
Two of the most commonly used visual aids are the logic model and theory of change. Other examples include realist matrix, systems map, road map, conceptual map, pathways map, blueprint for change, framework for action, programme framework, programme hypothesis. These may all use slightly different approaches, and some might be more appropriately suited to understanding or presenting different programmes or initiatives, but they all describe essentially how change is meant to happen.
The visual representation, the ‘picture’, needs to be accompanied by text, which will provide a richer description of your programme theory. The details of the activities, such as the “dose”, intensity, timing, duration, and the characteristics (target group & providers) should all be specified in the programme theory description. If the programme theory draws on existing theories, e.g. of behaviour, these should also be outlined. The diagram should be easy to understand by a variety of stakeholders that might be involved in different stages of the programme. Harms should be theorised as well as benefits.
In the paper below, Skivington and colleagues present an update to the UK Medical Research Council’s widely used guidance for developing and evaluating complex interventions and recommend an increased emphasis on improving theories and on understanding how interventions contribute to change.