How to Read: Bias in meta-analysis detected by a simple, graphical test [Reading Guide]

This guide will help you effectively read "Bias in meta-analysis detected by a simple, graphical test" by Egger et al., published in the British Medical Journal in 1997, using the three-pass approach.

First pass (5–10 min)

What to skim: Begin by quickly reviewing the core components to grasp the paper's essence.

• Title: "Bias in meta-analysis detected by a simple, graphical test" immediately tells you the subject (bias in meta-analysis) and the proposed solution (a simple, graphical test).
• Abstract (page 1): Read the Objective, Design, Main outcome measure, Results, and Conclusions sections. This will provide a high-level overview of the problem, the method, and the key findings.
• Introduction (page 1): Skim the first and last paragraphs to understand the context of meta-analysis, the problem of discrepancies with large trials, and the paper's aim.
• Conclusions (page 6): Read this section to confirm the main takeaways and the authors' final recommendations.
• Key messages (page 6): Review the bullet points for a concise summary of the paper's most important contributions.

What to notice:

• Contributions: The paper proposes a simple, graphical, and statistical method (funnel plot asymmetry) to detect bias in meta-analyses. It demonstrates its utility in predicting discordance between meta-analysis results and large trials, and assesses the prevalence of such bias.
• Problem: Meta-analyses, while valuable, can be misleading due to biases (e.g., publication bias), leading to results that contradict large, definitive trials. Existing methods for assessing funnel plot asymmetry are often informal.
• Setting: The research is grounded in medical literature, specifically systematic reviews and meta-analyses of clinical trials. The examples cover various medical fields like cardiovascular, diabetic, and geriatric medicine.
• Key terms: Note "funnel plots," "meta-analysis," "publication bias," "asymmetry," "odds ratio," and "precision."

Second pass (up to 1 hour)

Using SpatialRead in the Second Pass

• Use the Screenshot tool to capture figures, tables, diagrams, and formulae from the PDF and use AI for simple explanations.
• Ask: "Explain Figure X at a high level", "Compare Table 2 vs 3", or "What does the y-axis represent here?"
• Download SpatialRead →

What sections/figures to focus on: This pass involves a more detailed reading to understand the methodology and key results.

• Introduction (page 1): Read thoroughly to understand the background of meta-analysis, the concept of funnel plots, and the rationale for a systematic assessment of asymmetry.
• Methods (pages 1-2):
  • "Measures of funnel plot asymmetry" (page 1): Focus on how asymmetry is quantified using linear regression of the standard normal deviate against precision. Understand the regression equation (SND = a + b × precision) and that the intercept 'a' measures asymmetry.
  • "Identification of meta-analyses and matching large randomised trials" (page 2): Understand how studies were selected and matched.
  • "Concordance and discordance of results" (page 2): Pay attention to the criteria used to define concordance (effects in the same direction, meta-analytic estimate within 30% of the single trial estimate).
• Results (pages 2-4):
  • Table 1 (page 2): "Characteristics of nine pairs of meta-analyses and corresponding large trials." Examine the types of interventions and the concordant/discordant classifications.
  • Figure 1 (page 3): "Results from four concordant and four discordant pairs..." Visually compare the odds ratios and confidence intervals for meta-analyses versus large trials.
  • Table 2 (page 3): "Analysis of funnel plot asymmetry." Focus on the 'Intercept (90% CI)' and 'P value' columns, particularly noting the significant P values associated with discordant pairs.
  • Figure 2 (page 3): "Funnel plots and single large trials." Observe the visual representation of asymmetry in the discordant pairs (e.g., Magnesium, Nitrates) compared to the more symmetrical plots in concordant pairs.
  • "Frequency of asymmetry in funnel plots" (page 3): Note the percentages of journal and Cochrane meta-analyses showing significant asymmetry.
• Discussion (pages 4-5): Skim the "Sources of funnel plot asymmetry" section (page 5) and the boxed summary (page 5) to get a comprehensive list of potential causes for asymmetry.

Key terms/questions to note:

• How is "precision" defined in the context of funnel plots?
• What is the significance of the intercept 'a' in the regression model?
• What specific types of bias can funnel plots help detect?
• How well did funnel plot asymmetry predict discordance in the study's pairs of meta-analyses and large trials?

Third pass (1–5 hours)

Using SpatialRead in the Third Pass

• Highlight complex keywords, dense paragraphs, equations, or proofs and ask for a plain-English explanation that preserves notation.
• Use AI to explain the concepts in a way that is easy to understand.
• Ask for alternative phrasings or analogies to confirm understanding without changing technical meaning.
• Connect concepts with each other using AI to build a comprehensive understanding of the paper.
• Download SpatialRead →

Deep reading guidance: This pass involves a critical and thorough examination of the paper's details, assumptions, and implications.

• Methods (pages 1-2):
  • "Measures of funnel plot asymmetry": Delve into the technical details of the linear regression model. Understand the theoretical basis for why small trials cluster near the origin and how asymmetry manifests as a deviation from a symmetrical inverted funnel. Consider the choice of odds ratio and its logarithmic scale.
  • Study Identification: Critically evaluate the search strategy and inclusion criteria for both meta-analyses and large trials. Are the criteria for "large trials" sufficiently stringent?
  • Concordance Definition: Reflect on the 30% difference threshold for concordance. Is this a reasonable and robust definition?
• Results (pages 2-4):
  • Detailed Analysis: Go through Table 1, Figure 1, and Table 2 in conjunction. For each pair, understand the specific intervention, the observed concordance/discordance, and the corresponding funnel plot asymmetry statistics (intercept, P-value).
  • Visual vs. Statistical Asymmetry: Compare your visual interpretation of Figure 2 with the statistical results in Table 2. Do they align?
  • Figure 3 (page 4): Examine how updating a meta-analysis with additional trials can reveal or change funnel plot asymmetry, highlighting the dynamic nature of bias detection.
  • Figure 4 (page 4): Analyze the distribution of intercepts for Cochrane and journal meta-analyses. What does the observed shift towards negative values imply about the direction of bias in published literature?
• Discussion (pages 4-5):
  • "Sources of funnel plot asymmetry": Read this section thoroughly. Understand the nuances of each potential source of asymmetry (e.g., publication bias, language bias, citation bias, true heterogeneity, methodological quality differences, artefactual issues, chance). Consider how these factors might interact.
  • Limitations: Pay attention to any limitations discussed by the authors, particularly regarding the capacity to detect bias when meta-analyses are based on a limited number of small trials.
  • Comparison to other studies: Understand how the authors' findings on the frequency of bias compare to other research.
• Conclusions (page 6): Re-read to fully grasp the practical implications of the findings and the authors' recommendations for routine critical examination of systematic reviews.

Replication pointers:

• The paper provides a clear statistical method for quantifying funnel plot asymmetry (linear regression of standard normal deviate against precision). This method is described in sufficient detail to be implemented.
• The criteria for identifying and matching meta-analyses with large trials, as well as the definition of concordance, are explicitly stated, allowing for similar comparative studies.
• The specific examples provided in Table 1 and Figures 1, 2, and 3 could serve as test cases for replicating the analysis or applying the method to new data.