Close

When to apply specific designs and methods in evidence synthesis

Evidence synthesis should be fit for purpose. Designing and carrying out any synthesis effort that meets its objectives depends on several factors, including the type of review question, the nature of evidence available, and the audience or disciplinary conventions that are accepted as best practice for synthesizing the state of the available evidence.

Thematic questions

Useful questions that can be answered through evidence synthesis range from the very broad to very narrow. From the widest perspective, this may include questions about the nature, scope and extent of research on one or more themes, such as disability and climate change.  

For this type of enquiry, scoping, overview and rapid realist review methodologies might be appropriate.  In these types of questions, there may not be quantified effects to compare between studies, and almost certainly effects across studies would not be homogenous enough to combine.  These types of questions might also entail reviewing a vast amount of relevant literature and may require a tailored approach given typical resource limitations. 

Policy questions

Policy questions are increasingly popular, given the understanding that change is most likely to occur when systems themselves are shaped and transformed with better outcomes in mind. 

For these types of questions, the nature of the evidence is usually different, i.e. informed by grey literature, policy documents, government reports, and tracking of indicators and statistics through centralized mechanisms. Therefore, the methods applied to answer these types of questions must be able to identify, prioritise (as appropriate) and synthesise these types of information.  

As with Big Picture reviews, appropriate methodologies are unlikely to follow the PICO or exposure/outcome frameworks.  Rather, the important findings from a policy review question might include information about context, actors, enabling environments, underlying evidence based on other study designs than RCTs, or intersections with other policies and priorities.

Public health and epidemiological questions

Traditional public health and epidemiological questions are typically answered through a narrow scope, with well-defined research questions about exposures and outcomes. These types of questions are meant to aggregate research findings from many studies (usually experimental and observational) to produce a credible, overall view of the state of evidence. 

This, however, is not the common approach to benchmarking current knowledge outside the field of public and global health.  For instance, in STEM sciences or economics, other literature review methods prevail. Thus, when working in an interdisciplinary space, the approach to evidence synthesis should be carefully considered. 

Lastly, many questions fit for evidence synthesis review are about ‘how’, ‘why’, when and for whom, e.g. realist reviews, which focus on contextual and mechanistic factors. These questions may beg a qualitative, mixed methods, and/or theory-based approach whereby, for instance, qualitative research informs a quantitative inquiry or vice versa, or methods applied in implementation science.