Why Risk of Bias Matters
Risk of bias assessment evaluates the internal validity of studies included in a systematic review. Studies with high risk of bias may overestimate or underestimate the true effect, leading to misleading conclusions. Cochrane and most high-impact journals require formal risk of bias assessment.
RoB 2.0: For Randomized Controlled Trials
The revised Cochrane Risk of Bias tool (RoB 2.0) assesses RCTs across five domains:
- Domain 1 — Randomization process: Was the allocation sequence random? Was it concealed? Were there baseline imbalances?
- Domain 2 — Deviations from intended interventions: Were participants and personnel aware of assignments? Were there deviations that could affect outcomes?
- Domain 3 — Missing outcome data: Was outcome data available for all or nearly all participants? Could missingness depend on the true value?
- Domain 4 — Measurement of the outcome: Was the outcome assessor blinded? Could the method of measurement have differed between groups?
- Domain 5 — Selection of the reported result: Were multiple outcomes or analyses pre-specified? Is there evidence of selective reporting?
Each domain is judged as "Low risk," "Some concerns," or "High risk." The overall judgment follows the worst-performing domain.
ROBINS-I: For Non-Randomized Studies
ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) assesses seven domains:
- Confounding: Were important confounders identified and controlled?
- Selection of participants: Was selection into the study related to both intervention and outcome?
- Classification of interventions: Was intervention status well-defined and recorded?
- Deviations from intended interventions: Were there co-interventions or switches?
- Missing data: Was outcome data reasonably complete?
- Measurement of outcomes: Could measurement have been influenced by knowledge of intervention?
- Selection of reported results: Were reported results selected from multiple analyses?
Practical Steps for Assessment
- Read the full text of each study carefully, focusing on methods and supplementary materials
- Use the signaling questions provided in RoB 2.0 or ROBINS-I worksheets
- Have two independent reviewers assess each study
- Resolve disagreements through discussion or a third reviewer
- Document your judgments with supporting quotes from the study
- Present results in a traffic-light plot and summary bar chart
Visualizing Results
Use the robvis R package or the online robvis tool (mcguinlu.shinyapps.io/robvis) to create publication-quality traffic-light plots and summary bar charts. These visualizations are expected by Cochrane and most peer-reviewed journals.
