Evidence synthesis is the transparent process of bringing together results from multiple relevant studies and other pre-defined information sources to make a statement about a body of evidence. It connects a clearly formulated question with systematic search, selection, data preparation, evaluation and interpretation, so that the conclusion remains comprehensible. A single study is not replaced by this, but placed in its context.
Question and systematic process
At the beginning is a precise question, for example regarding population, intervention, comparison and endpoint. The protocol of the synthesis determines which sources are searched, which inclusion and exclusion criteria apply, which data are collected and how to deal with multiple publications, deviating definitions or missing information. Pre-defined steps reduce the risk of selection or analysis being influenced after the results are known.
After the search, the studies are selected based on the criteria and their characteristics, results and limitations are recorded in a structured manner. For the interpretation, not only effect measures count, but also study design, comparator treatment, time of measurement, target population and risk of systematic bias. Tables and comprehensible justifications show which evidence was actually used for which statement. If data are similar, results can be combined; if there are relevant differences, these differences must remain visible.
Assessment of consistency and uncertainty
An evidence synthesis assesses whether the findings agree across studies in direction and magnitude of the effect and are transferrable to the clinical question. Differences can arise from population, dosing, concomitant therapy, endpoint definition or follow-up time. Heterogeneity is therefore not a purely statistical problem, but influences whether a common conclusion is appropriate.
Incompletely reported results, study discontinuations and missing values can also bias the conclusion. Sensitivity analyses and alternative, pre-justified assumptions make it apparent how robust a conclusion is. For confirmatory studies, ICH E9 requires a planning of design and essential statistical analyses defined before the start of the study. ICH E9(R1) adds that the clinical question and the treatment effect to be estimated should be precisely described as an estimand. This information helps to correctly contextualize study results within a synthesis.
The conclusion of a synthesis should therefore always state the scope of the underlying evidence. A result from narrowly selected populations is not readily transferrable to routine care. Likewise, the number of included studies must not be confused with certainty: Multiple studies with similar methodological weaknesses can replicate the same uncertainty.
Differentiation from meta-analysis
Evidence synthesis is the umbrella term for the planned bringing together of a body of evidence. A meta-analysis, on the other hand, is a single quantitative procedure within this process: It combines suitable numerical results from multiple studies into a common estimate. It is only meaningful if comparison, effect measure and clinical context are sufficiently compatible and the chosen modelling can be justified.
If studies are too different or data are not available in a comparable way, a structured tabular or narrative synthesis can be the more appropriate method. Even without meta-analysis, selection, evaluation and conclusion must remain transparent. Conversely, a calculated common effect size does not automatically make a synthesis robust if the underlying studies are unsuitable, selectively chosen or methodologically too heterogeneous.
Relevance for clinical trials
In clinical development, evidence synthesis supports the choice of clinically relevant endpoints, the justification of the comparator treatment and the integration of new data into existing knowledge. For study protocol, statistical analysis plan and clinical study report, assumptions, external evidence and remaining uncertainties must be presented consistently. Especially with complex therapy landscapes or small populations, it must be comprehensibly separated what was directly observed and what is derived from multiple sources.
Full-service CROs such as Mediconomics support the formulation of the question, the documentation of search and selection processes, data extraction as well as the methodological evaluation and presentation of results. This includes biostatistics for appropriate quantitative syntheses, medical writing for consistent presentation of evidence and quality checks of the traceability between protocol, analysis plan and report.
Frequently Asked Questions (FAQ)
Is evidence synthesis the same as a systematic review?
A systematic review is a frequent form of evidence synthesis. The term evidence synthesis emphasizes the entire process of structured bringing together; the specific scope depends on the question and the pre-defined methods.
When is a meta-analysis not appropriate?
It is not appropriate if relevant differences between studies make a common quantitative estimate contextually misleading or if the reported data are not comparable. Then a structured synthesis without statistical pooling estimate can be more suitable.
What role do missing data play?
Missing data can cause biases and must be considered in the evaluation of each included study. The synthesis should disclose which assumptions or sensitivity analyses the studies used regarding missing values.
Regulatory references
- ICH E9 “Statistical Principles for Clinical Trials” – basis for pre-planned statistical principles in clinical trials.
- ICH E9(R1) “Addendum on Estimands and Sensitivity Analysis” – assigns treatment effects and sensitivity analyses to the clinical question.
- EMA/CPMP/EWP/1776/99 Rev. 1 “Guideline on Missing Data in Confirmatory Clinical Trials” – describes the regulatory assessment of missing data.
- Regulation (EU) No 536/2014 – regulates clinical trials on medicinal products for human use in the Union.