Causal Inference in Statistics: A Primer — Chapter-by-Chapter Outline

Author: Judea Pearl, Madelyn Glymour, Nicholas P. Jewell First published: 2016 Edition covered: First Edition (John Wiley & Sons, 2016, ISBN 9781119186847). Single edition as of the research date; no revised or second edition exists. The companion website at bayes.cs.ucla.edu/PRIMER offers errata, additional homework sets, and solution manuals.

Central thesis

Statistics has long treated causation as a philosophical luxury — a concept that adds flavor to data analysis but cannot be formalized within the standard probability calculus. This primer argues the opposite: causation is not a philosophical add-on but a distinct mathematical layer above probability, one that can be made rigorous, expressed in formulas, and applied systematically to observational data. Without it, statistical inference is fundamentally incomplete.

The book's central move is the introduction of the Structural Causal Model (SCM) — a framework that encodes cause-and-effect relationships in equations and directed graphs — together with the do-operator, which mathematically distinguishes the act of intervening on a variable from merely observing it. These tools transform questions that could not even be stated in conventional statistics ("What is the effect of this drug on recovery?", "Would this patient have survived if treated differently?") into well-posed mathematical problems with calculable answers.

The argument is developed across three ascending rungs of what Pearl calls the Ladder of Causation: association (seeing), intervention (doing), and counterfactuals (imagining). Standard statistics occupies only the first rung. The primer's mission is to give readers a working grip on all three, illustrated throughout with concrete examples drawn from medicine, economics, and social policy.

Can we, from purely observational (non-experimental) data, determine the causal effect of one variable on another — and if so, how?

Chapter 1 — Preliminaries: Statistical and Causal Models

Central question

Why is causation a separate topic from statistics, and what mathematical tools — probability, graphs, and structural equations — do we need before we can reason about it?

Main argument

Why statistics alone is not enough — Simpson's Paradox

The chapter opens with a sustained treatment of Simpson's Paradox, a deceptively simple numerical phenomenon that reveals everything wrong with purely associational reasoning. The paradox: a drug appears to help men and women separately, yet harm the overall population — or vice versa — depending on which table of numbers one consults. In Pearl's running example, 93% of men who took the drug recovered vs. 87% who didn't; 73% of women who took it recovered vs. 69% who didn't; yet in the combined population, only 78% of drug-takers recovered vs. 83% of non-takers. The paradox is not a mathematical trick but a symptom of confounding. The correct answer — whether to prescribe the drug — depends on whether gender is a cause of treatment choice and recovery (condition on it) or whether the measured variable is caused by treatment (do not condition on it). No statistical formula can tell you which it is; only a causal model can.

This is contrasted with a parallel numerical example involving post-treatment blood pressure (Table 1.2). The numbers are identical to the drug/gender table, yet the correct policy is the opposite. The only difference is the causal story: gender is a common cause of both treatment and outcome, while blood pressure is an effect of treatment. Same numbers, different causal structure, different answer. This dramatically establishes the chapter's thesis: data alone are silent on causation.

A brief probability toolkit

Sections 1.3.1–1.3.11 provide the probability background needed for the rest of the book. Covered topics include: random variables and events; conditional probability and its definition P(A|B) = P(A ∩ B) / P(B); the independence condition P(A|B) = P(A); probability distributions; the Law of Total Probability Σz P(Y=y|Z=z)P(Z=z) = P(Y=y); Bayes' Rule P(A|B) = P(B|A)P(A) / P(B); expected values E(X) = Σx x·P(X=x); variance Var(X) = E[(X − E(X))²]; covariance Cov(X,Y) = E[(X − E(X))(Y − E(Y))]; regression coefficients b = Cov(X,Y) / Var(X) with the regression line E(Y|X=x) = a + bx; and multiple regression E(Y|X₁=x₁, X₂=x₂) = a + b₁x₁ + b₂x₂. These are the statistical tools that will be enriched — but also shown to be insufficient — in later chapters.

Graphs as the language of causation

Section 1.4 introduces directed acyclic graphs (DAGs). A node represents a variable; a directed edge X → Y means X is a direct cause of Y in the model. Key graph vocabulary: parent, child, ancestor, descendant, path, and the restriction to acyclic graphs (no variable can be its own cause). Graphs encode qualitative causal assumptions independently of any specific numerical values.

Structural Causal Models — the formal framework

Section 1.5 is the theoretical heart of Chapter 1. A Structural Causal Model (SCM) consists of:

• A set of endogenous variables V (the quantities of interest, determined by the model)
• A set of exogenous variables U (background variables, determined outside the model)
• A set of structural equations F = {f₁, f₂, …, fₙ} where each fᵢ specifies how the variable Xᵢ is determined by its parents and its exogenous noise term Uᵢ

The SCM in Example 1.5.1 models schooling (X), employment (Y), and salary (Z) as:

• X = Uₓ
• Y = X/3 + U_Y
• Z = Y/4 + U_Z

This is a deterministic skeleton with random noise injected via the U terms. The graph encodes which variables appear in which equations; the equations encode the functional form of each causal relationship.

Product decomposition and Bayesian networks

The product decomposition rule (Section 1.5.2) follows from the Markov condition. Because each variable in an SCM depends only on its parents and its own exogenous noise (which is independent of all others), the joint distribution factors as:

P(x₁, x₂, …, xₙ) = ∏ᵢ P(xᵢ | pa(xᵢ))

where pa(xᵢ) denotes the parents of xᵢ. This factorization has testable implications — conditional independencies — that can be read from the graph without knowing any specific functional forms. It also connects SCMs to the earlier tradition of Bayesian Networks, while being more powerful because it encodes causal, not merely probabilistic, relationships.

Key ideas

• Association does not imply causation, and no amount of additional conditioning on observables can resolve Simpson's Paradox without causal knowledge.
• The same numerical data set can demand opposite actions depending on the causal story that generated it.
• An SCM has three ingredients: variables, structural equations, and independent background noise terms.
• Directed acyclic graphs encode qualitative causal assumptions as a prior to any numerical analysis.
• The product decomposition rule — P(V) = ∏ P(xᵢ | pa(xᵢ)) — converts graph structure into a factorization of the joint distribution.
• Statistical tools like regression measure association; they do not, by themselves, identify causes.
• Simpson's Paradox is not resolved by collecting more data; it is resolved by causal reasoning about which variables to condition on.

Key takeaway

Causation requires its own mathematical language — structural equations and directed graphs — because the same statistical distribution can arise from multiple, action-dictating causal stories that standard probability theory cannot distinguish.

Chapter 2 — Graphical Models and Their Applications

Central question

How do the structural assumptions encoded in a causal graph translate into patterns of statistical dependence and independence in data, and how can those patterns be used to test causal models and search for new ones?

Main argument

From causal structure to statistical independence — Chains and Forks

The chapter opens by showing how a graphical structure implies a definite set of dependence and independence relations, independently of the specific functions in the SCM. Three SCMs with entirely different functional forms (one linear continuous, one logical/Boolean, one nonlinear continuous) all share the same graph X → Y → Z. Because they share the graph, all three must share the same independence pattern: X ⊥⊥ Z | Y (X and Z become independent once we condition on the intermediate node Y).

This introduces Rule 1 (Chains): in a chain X → Y → Z, X and Z are conditionally independent given Y. The intuition: information flows along the chain only through Y; blocking Y at Y = y freezes that flow.

The fork structure — where a common cause X emits arrows to both Y and Z (Y ← X → Z) — gives Rule 2 (Forks): Y and Z are conditionally independent given X. The temperature → ice cream sales fork with temperature → crime is the running illustration: high ice cream sales and high crime are correlated only because temperature drives both; controlling for temperature severs the association. Crucially, this fork example explains the pre-condition for confounding: Y ← X → Z is the structure of a confounded study when Z is the treatment, Y the outcome, and X an unobserved common cause.

Colliders — the most counterintuitive structure

Section 2.3 introduces the collider node: a variable Z that receives arrows from two parents, X → Z ← Y. Colliders behave opposite to chains and forks. In a collider:

• X and Y are marginally independent (no path between them flows freely).
• X and Y become dependent once we condition on Z (or on any descendant of Z).

The coin-and-bell example is the clearest illustration. Two fair coins (X and Y) are flipped independently; a bell (Z) rings whenever at least one lands heads. Marginally, knowing the outcome of coin 1 tells you nothing about coin 2. But once you know the bell rang (condition on Z=1), learning that coin 1 landed tails forces coin 2 to have landed heads. The two independent coins become dependent by conditioning on their common effect.

This Rule 3 (Colliders) has profound practical consequences: conditioning on a collider — or any of its descendants — creates spurious dependence between the collider's parents. Researchers who routinely "control for" variables they consider potentially confounding may be inadvertently opening collider paths and inducing bias that did not exist in the raw data.

The Monty Hall Problem is reframed as a collider: your initial door choice and the location of the car are the two independent parent nodes; Monty's door choice is the collider. Conditioning on which door Monty opens creates a dependence between your initial choice and where the car is, which explains why switching is advantageous.

d-separation — a general criterion

Section 2.4 generalizes Rules 1–3 into a single algorithm: d-separation. Given any two nodes X and Y and a conditioning set Z in a DAG, X and Y are d-separated by Z (and thus independent in any distribution consistent with the graph) if and only if every path between X and Y is "blocked" by Z. A path is blocked if it contains either:

1. A chain or fork whose middle node is in Z, or
2. A collider whose node — and all of its descendants — are not in Z.

Conversely, X and Y are d-connected given Z (and thus likely dependent) if at least one path is unblocked. d-separation is both a powerful theoretical result and a practical tool: it tells you which conditional independence tests are implied by a causal model before you look at any data.

Model testing and causal search

Section 2.5 shows that d-separation is testable: every d-separation statement is an implied conditional independence that any large data set generated by that model must satisfy (approximately). This allows researchers to:

1. Test whether a proposed causal graph is consistent with data (by checking that all implied independencies hold).
2. Search for graphs consistent with data (causal discovery algorithms like PC and FCI use d-separation-based independence tests to eliminate graphs that contradict observed independencies).

The section also introduces the idea of Markov equivalence — multiple distinct DAGs can imply identical d-separation statements and thus identical probability distributions, making them empirically indistinguishable from observational data alone. Distinguishing them requires interventional data or additional domain knowledge.

Key ideas

• Three simple path structures — chains, forks, and colliders — determine all independence patterns in any DAG.
• In chains (X → Y → Z) and forks (Y ← X → Z), conditioning on the middle node blocks the association between the outer nodes.
• In colliders (X → Z ← Y), conditioning on the middle node opens a previously closed path and creates dependence.
• d-separation is a sound and complete criterion for reading off all conditional independencies implied by a DAG.
• Every d-separation statement is an empirically testable constraint; data that violate it falsify the graph.
• Causal discovery is possible from observational data, but only up to Markov equivalence classes.
• "Controlling for" a variable is not always safe; it can introduce bias if the variable is a collider or a descendant of a collider.

Key takeaway

A causal graph is an empirically committal object: it implies specific, testable patterns of statistical independence, and a researcher who knows these patterns can both validate a proposed causal story against data and, in principle, infer causal structure from data alone.

Chapter 3 — The Effects of Interventions

Central question

How can we predict the effect of a deliberate intervention — a policy, treatment, or manipulation — on an outcome, using data from observational studies where no such intervention occurred?

Main argument

Interventions vs. observations — the do-operator

The chapter introduces the do-operator, the book's central formal contribution. The notation P(Y=y | X=x) means "the probability that Y=y given that we observe X=x." The notation P(Y=y | do(X=x)) means "the probability that Y=y given that we externally set X=x to the value x." These two quantities are generally different, and confusing them is the root cause of most errors in causal inference.

The ice cream / crime example crystallizes the distinction. If we observe high ice cream sales (X=x), we learn it's hot, which raises crime rates. But if we intervene to ban ice cream shops (do(X=low)), temperature is unchanged and crime is unaffected. Observational conditioning transmits correlational information through all paths; the do-operator severs all arrows into X, creating an intervention graph from which confounding paths are removed.

Formally, the do(X=x) operation is graph surgery: remove all edges into X and set X=x. The resulting distribution is:

P(V \ {X} | do(X=x)) = ∏ᵢ P(xᵢ | pa(xᵢ)) for all i where Xᵢ ≠ X

The Adjustment Formula — computing P(Y|do(X)) from observational data

When X has observed parents PA(X) = Z (variables that confound the effect of X on Y), the causal effect is:

P(Y=y | do(X=x)) = Σ_z P(Y=y | X=x, Z=z) · P(Z=z)

This is the adjustment formula, Section 3.2. Applied to the Simpson's drug example with gender (Z) as the confounder: P(Y=1 | do(X=1)) = 0.832 and P(Y=1 | do(X=0)) = 0.782, giving an Average Causal Effect (ACE) of 0.050. The drug helps. The adjustment formula tells us to condition on gender, compute within-gender effects, then average using the marginal gender distribution in the population — not the gender distribution among drug-takers.

The blood-pressure counterexample shows why mechanical conditioning is wrong: if blood pressure (Z) is caused by the drug (X → Z → Y), then conditioning on Z blocks part of the causal pathway and produces a biased estimate. The adjustment formula, derived from the graph, correctly instructs not to condition on post-treatment variables.

Section 3.2.2 introduces the truncated product formula (also known as the g-formula): for multiple simultaneous interventions, the post-intervention joint distribution is the pre-intervention joint distribution with all factors corresponding to intervened variables deleted.

The Backdoor Criterion — adjusting for the right variables

Section 3.3 formalizes when and how to adjust. A set Z satisfies the Backdoor Criterion relative to (X, Y) if:

1. No node in Z is a descendant of X, and
2. Z blocks every path between X and Y that has an arrow into X (a "backdoor path").

When Z satisfies the backdoor criterion, P(Y=y | do(X=x)) = Σ_z P(Y=y | X=x, Z=z) · P(Z=z). The parents of X always satisfy the backdoor criterion, but so may other sets — including sets that contain colliders, provided the collider itself is not unblocked. Section 3.3 works through the Figure 3.6 example: drug (X), recovery (Y), weight (W), and unmeasured socioeconomic status (Z). Although Z is unobserved, W blocks the backdoor path X ← Z → W → Y, so adjusting for W suffices. Lord's paradox (Study question 3.3.2 on student meal plans and weight) is another concrete application.

The Front-Door Criterion — identifying effects through a mediator

Section 3.4 shows that the backdoor criterion is not the only route to causal identification. The classic illustration is the smoking-lung-cancer-tar example (Figure 3.10b). Suppose an unobserved genetic confounder U affects both smoking (X) and lung cancer (Y), making the backdoor path unblockable. However, if tar deposits (Z) mediate the full effect of smoking on lung cancer (X → Z → Y with U not affecting Z), then the Front-Door Criterion applies:

P(Y=y | do(X=x)) = Σz [Σ{x'} P(Y=y | X=x', Z=z) P(X=x')] · P(Z=z | X=x)

This two-step formula first estimates the effect of smoking on tar (which is unconfounded because tar has no back-door paths) and then estimates the effect of tar on cancer (adjusting for smoking as a backdoor blocker). Remarkably, even though the direct confounding path cannot be blocked, the causal effect is identified by routing through the mediator.

Conditional interventions and covariate-specific effects

Section 3.5 extends the framework to conditional interventions — policies that set X based on observed covariates (e.g., "treat patients whose blood pressure is high") — and covariate-specific causal effects P(Y=y | do(X=x), W=w). These require adjusting for additional variables to block spurious paths opened by conditioning on W, especially if W is a collider or a descendant of a collider.

Inverse Probability Weighting (IPW)

Section 3.6 introduces IPW as an alternative to direct adjustment. Instead of partitioning the sample by Z and computing conditional expectations, we reweight each observation by 1 / P(X=x | Z=z), the inverse of the propensity score. The reweighted sample behaves as if treatment were randomized: IPW eliminates the association between Z and X while preserving the association between X and Y. This provides a second route to computing P(Y | do(X)) and is especially useful when Z is high-dimensional or continuous.

Mediation — decomposing total effects into direct and indirect

Section 3.7 introduces mediation analysis: decomposing the total causal effect of X on Y into a direct effect (the path not through any mediator) and an indirect effect (the path through a mediator M). In the gender / qualifications / hiring example (Figure 3.11), the total effect of gender on hiring decomposes into a direct path (gender → hiring) and an indirect path (gender → qualifications → hiring). This decomposition matters for legal and policy questions: discrimination law often hinges on whether a disparity is mediated by a legitimate factor (qualifications) or reflects direct discrimination.

Causal inference in linear systems

Section 3.8 applies the general theory to linear models, the arena where most applied statistics is done. Two types of coefficients are distinguished:

• Regression coefficients (from ordinary least squares): describe associational, predictive relationships; they change value when new variables are added.
• Structural coefficients (path coefficients in the SCM): describe the direct causal effect of one variable on another; they do not change when variables outside the causal path are added.

The chapter shows how to identify structural coefficients from observed correlations. For a simple chain X → Y → Z, the path coefficient from X to Z equals the product of the coefficients along the path. For a confounded model with unobserved U, the structural coefficient may be identifiable via an instrumental variable — a variable that affects X but is independent of U and has no direct effect on Y. The IV estimator Cov(I,Y) / Cov(I,X) recovers the structural coefficient from observational data even in the presence of unmeasured confounding.

Section 3.8.4 handles mediation in linear systems: the total effect decomposes as total = direct + indirect, where indirect = (X→M coefficient) × (M→Y coefficient), and each is identified by the structural analysis of the relevant subgraph.

Key ideas

• P(Y | X=x) and P(Y | do(X=x)) are mathematically and practically different; conflating them is the error of confounding.
• The do-operator is graph surgery: it removes all arrows into X before computing probabilities.
• The adjustment formula correctly estimates P(Y | do(X)) by conditioning on a sufficient set of covariates that satisfies the backdoor criterion.
• The backdoor criterion identifies which variables to condition on without inadvertently opening collider paths or blocking causal paths.
• The front-door criterion can identify causal effects even when all confounders are unobserved, by exploiting a mediating variable.
• Inverse probability weighting provides a propensity-score-based alternative to covariate adjustment.
• Structural (path) coefficients are stable across models; regression coefficients are not.
• Instrumental variables identify structural coefficients when all parents of X cannot be observed.
• Mediation analysis decomposes total effects into direct and indirect components, enabling causal attribution.

Key takeaway

With the right causal graph and a set of covariates satisfying the backdoor (or front-door) criterion, causal effects can be calculated exactly from observational data — without any randomized experiment — using only conditional probabilities that the data provide directly.

Chapter 4 — Counterfactuals and Their Applications

Central question

How can we reason about what would have happened under conditions that did not occur — and how does that reasoning allow us to answer questions about individual causation, credit, blame, discrimination, and mediation that population-level intervention analysis cannot?

Main argument

From interventions to counterfactuals — the third rung

Chapter 4 ascends to the highest rung of Pearl's Ladder of Causation: counterfactuals. While the do-operator (Chapter 3) asks "What would happen to the population if we set X=x?", a counterfactual asks "What would have happened to this specific individual if, contrary to what actually occurred, X had been set to x?" These are distinct questions. Population-level intervention analysis is agnostic about individuals; counterfactual reasoning is about specific units, cases, and their particular circumstances.

Example 4.1: A teacher gives homework (X) and tests students (Y). For a specific student, Y depends on X through the model Y = X + U, where U = 2 is fixed for that student. The student actually received X=1 homework assignments and scored Y=3. The counterfactual question: what would that student have scored if they had received X=2? The answer, Y₂(u=2) = 2+2 = 4, is obtained by "surgery" — substituting X=2 into the structural equation while holding U=2 fixed at its actual value. This is fundamentally different from E[Y | do(X=2)], which averages over all students and discards the information that this particular student has U=2.

The Structural Interpretation of Counterfactuals

Section 4.2.1 defines the potential outcome Yₓ(u) as the value that Y would have taken in unit u had X been set to x, computed by solving the SCM with X replaced by x while all exogenous variables U stay fixed at their actual values. This gives counterfactuals a precise, model-based definition that avoids the vagueness of informal "what if" reasoning.

The Fundamental Law of Counterfactuals

Section 4.2.2 states: For any SCM M and any two variables X and Y in it, the counterfactual Yₓ(u) is the solution for Y in the modified model Mₓ (the model M with the equation for X replaced by the constant X=x), evaluated at U=u. Formally, Yₓ(u) = Y{Mx}(u). This is the link between the graph-based causal framework and the potential outcomes (Neyman–Rubin) framework widely used in econometrics and epidemiology.

The Three Steps for Computing Counterfactuals

Section 4.2.4 establishes the standard three-step procedure:

1. Abduction: Update the distribution of U given the observed evidence (e.g., the actual values of variables for this individual), using Bayes' rule.
2. Action: Modify the model by replacing the equation for X with X = x (the counterfactual intervention).
3. Prediction: Compute the distribution of the outcome Y in the modified model under the updated distribution of U.

This three-step procedure is the counterfactual analogue of the do-calculus. It generalizes to probabilistic models (where U may not be fully identified) through Section 4.3.

Nondeterministic Counterfactuals and Probabilities of Causation

Section 4.3 extends to settings where U is not observed (as it is in most real studies). Here, the exogenous variables U cannot be pinned to individual values; instead, abduction updates a distribution over U. The key quantity is now the probability of the counterfactual, Yₓ = y. Section 4.3.1 introduces three fundamental probabilities of causation:

• PN (Probability of Necessity): P(Y₀ = 0 | X=1, Y=1) — the probability that the outcome would not have occurred had the treatment not been applied, given that it did occur and the outcome did occur.
• PS (Probability of Sufficiency): P(Y₁ = 1 | X=0, Y=0) — the probability that applying the treatment would produce the outcome, given that the treatment was not applied and the outcome did not occur.
• PNS (Probability of Necessary and Sufficient causation): P(Y₁=1, Y₀=0) — the probability that the treatment was both necessary and sufficient for the outcome.

These quantities formalize legal and attributional concepts like "but-for causation" (necessity) and "proximate cause" (sufficiency). They cannot in general be computed from observational data alone, but they can be bounded using observational data together with any experimental data that may be available. Section 4.5.1 provides the full attribution toolkit with identification conditions and formulas.

Graphical representation of counterfactuals

Section 4.3.2 shows how counterfactuals can be read from a modified DAG. The node Yₓ is represented as a new node in the graph derived from the original model Mₓ. This graphical representation clarifies independence relations among counterfactuals and enables the application of d-separation to counterfactual reasoning.

Counterfactuals in experimental settings and linear models

Section 4.3.3 derives the consistency rule: if X actually equals x (i.e., X(u) = x), then Yₓ(u) = Y(u). The observed outcome is the counterfactual outcome under the actual treatment. This connects the potential-outcomes framework to structural models and underlies the key formula: E[Y₁ - Y₀ | X=1] = E[Y|X=1] - E[Y₀|X=1], where the first term is observed and the second requires adjustment.

Section 4.3.4 specializes to linear models, where counterfactuals take particularly clean forms. For a linear SCM Y = αX + βZ + U with no hidden confounders, the counterfactual E[Yₓ | X=x', Z=z] = α(x-x') + E[Y|X=x', Z=z], showing that counterfactual effects in linear models reduce to differences in structural coefficients — a result that underlies most regression-based causal analysis in economics and the social sciences.

Practical Applications — recruitment, additive interventions, and personal decision-making

Section 4.4 applies the counterfactual machinery to a series of realistic policy problems.

Recruitment to a Program (Section 4.4.1): Should individuals be recruited to a beneficial program based on their predicted counterfactual gain, or based on their observed outcome? The Effect of Treatment on the Treated (ETT) E[Y₁ - Y₀ | X=1] answers this. The section shows that ETT is identifiable from observational data alone when Z satisfies the backdoor criterion, and provides the estimating formula. This directly addresses the ethical and policy question of targeting interventions toward those who benefit most.

Additive Interventions (Section 4.4.2): Real-world interventions often add a dose ε to an existing value of X (give 5mg more insulin, add one year of schooling) rather than setting X to an absolute value. The do(X=x+ε) formalism captures additive interventions, and counterfactual reasoning allows the same structural models to handle them without modification.

Personal Decision Making (Section 4.4.3): A specific student received encouragement X=0 and scored Y=3. Would she have scored differently with X=1? This requires abduction to update the distribution of U from the observed score, then prediction in the modified model. The section shows how a student can use the SCM to make personalized predictions that go beyond population averages.

Discrimination in Hiring

Section 4.4.4 uses the counterfactual framework to formalize legal questions about employment discrimination. The graph (Figure 3.11) represents gender (X) affecting hiring (Y) both directly and via qualifications (Z). The legal question of discrimination asks: "Would this applicant have been hired if they were of a different gender, holding qualifications fixed?" This is a counterfactual question: P(Y_{X=male} = 1 | X=female, Z=z, Y=0). The direct effect of gender on hiring, net of its effect through qualifications, is what courts assess as discrimination. The chapter shows how to compute the Controlled Direct Effect (CDE) — the effect of X on Y when the mediator M is held constant by intervention: CDE = P(Y | do(X=x), do(M=m)) — and distinguishes it from the Natural Direct Effect (NDE) and Natural Indirect Effect (NIE), which involve cross-world counterfactuals.

Mediation and Path-Disabling Interventions

Section 4.4.5 and the toolkit in Section 4.5.2 provide the full nonparametric mediation analysis framework. The total effect decomposes as:

TE = NDE - NIE (on the probability scale for binary outcomes)

where:

• NDE (Natural Direct Effect) = E[Y{x, M{x'}}] - E[Y_{x'}] : the effect of X on Y when M is set to the value it would naturally take under X=x' (the control condition).
• NIE (Natural Indirect Effect) = E[Y{x', Mx}] - E[Y{x'}] : the effect on Y of changing M from M{x'} to M_x while holding X at x'.

Unlike controlled direct effects (achievable by a single do-intervention), natural direct and indirect effects are cross-world counterfactuals — they involve a hypothetical individual who simultaneously receives X=x for one variable and X=x' for another. They cannot be identified from observational data alone in the presence of mediator-outcome confounding (the case in Figure 4.6b), but they can be identified under specific graph structures. Section 4.5.2 provides conditions and formulas.

Key ideas

• Counterfactuals ask what would have happened to a specific individual under a different treatment; this is distinct from population intervention analysis.
• The three-step counterfactual procedure (abduction → action → prediction) is the formal analog of Bayesian updating combined with do-calculus.
• The consistency rule links structural counterfactuals to the potential outcomes (Neyman–Rubin) framework: if X=x was actually observed, then Yₓ = Y.
• PN, PS, and PNS quantify degrees of causal attribution (necessity and sufficiency) and formalize legal concepts of "but-for" causation.
• ETT (Effect of Treatment on the Treated) captures the benefit of a policy for those it actually affects, enabling targeted interventions.
• Natural direct and indirect effects decompose total causal effects through mediators, but require cross-world counterfactuals not reducible to do-calculus alone.
• Discrimination analysis reduces to estimating the direct causal effect of a protected attribute on an outcome net of legitimate mediating factors.
• Linear models simplify counterfactual computation: counterfactual contrasts equal differences in structural coefficients.

Key takeaway

Counterfactual reasoning — enabled by structural causal models — extends causal inference from population-level intervention analysis to individual-level attribution, enabling answers to questions about credit, blame, discrimination, and mediation that cannot be formulated within either standard statistics or the do-calculus alone.

The book's overall argument

1. Chapter 1 (Preliminaries: Statistical and Causal Models) — establishes that standard statistics, powerful as it is, cannot distinguish between observing a value and setting it; Simpson's Paradox proves this limitation concretely, and Structural Causal Models — equations plus directed graphs plus independent noise terms — provide the missing mathematical layer.

2. Chapter 2 (Graphical Models and Their Applications) — shows that the causal graph is not merely a picture but an empirically committal object: it implies specific conditional independence relations (via d-separation) that can be tested in data, and from which causal structure can, within limits, be reverse-engineered.

3. Chapter 3 (The Effects of Interventions) — introduces the do-operator as the formal expression of "what if we intervened?", and develops the Backdoor Criterion, the Front-Door Criterion, inverse probability weighting, mediation analysis, and linear structural analysis as systematic tools for computing P(Y | do(X)) from observational data without randomized experiments.

4. Chapter 4 (Counterfactuals and Their Applications) — ascends to individual-level causal reasoning, showing how the same structural models support counterfactual computation via a three-step abduction–action–prediction procedure, and applying the result to attribution (PN, PS, PNS), discrimination law (natural direct and indirect effects), and policy targeting (ETT).

Common misunderstandings

Misunderstanding: "Adjusting for more variables is always safer."

Conditioning on a collider or on any descendant of a collider introduces bias that was not present in the raw association. The Backdoor Criterion tells you which variables to condition on; conditioning on others — especially post-treatment variables — can make estimates worse, not better.

Misunderstanding: "Regression coefficients measure causal effects."

Ordinary regression coefficients measure the slope of a conditional expectation — a purely associational quantity. They conflate direct and indirect effects, reverse causal directions, and change value when new variables are included. Structural (path) coefficients, identified via the SCM, measure causal effects. The two coincide only under specific conditions (no confounding, correct specification).

Misunderstanding: "Simpson's Paradox is resolved by using the most fine-grained available data."

The correct level of aggregation depends on the causal story, not on data granularity. The gender example requires the disaggregated data; the blood-pressure example requires the aggregated data. The numbers are identical in both cases; only the causal graph differs. More conditioning is not a cure; the right conditioning set is determined by the graph.

Misunderstanding: "Counterfactuals are too speculative to be scientifically useful."

Within a structural causal model, counterfactuals have precise, calculable definitions via the three-step abduction–action–prediction procedure. They are not more speculative than the model itself. Under specific conditions (particularly monotonicity), probabilities of causation (PN, PS) are point-identified from observational and experimental data.

Misunderstanding: "The potential outcomes (Neyman–Rubin) framework and Pearl's SCM framework are incompatible."

The Fundamental Law of Counterfactuals establishes a formal bridge: the potential outcome Yₓ(u) equals the solution for Y in the modified model Mₓ at U=u. Every potential-outcomes statement has an SCM translation and vice versa. The SCM framework is more expressive because it also encodes the causal graph (enabling d-separation and identification via front-door and backdoor criteria), which the potential-outcomes framework treats as external domain knowledge.

Misunderstanding: "The front-door criterion only works in the smoking example."

The front-door criterion is a general pattern: it applies whenever a treatment affects an outcome exclusively through a chain of mediating variables, each of which satisfies specific unconfoundedness conditions readable from the graph. It is not limited to any particular domain.

Central paradox / key insight

The deepest paradox in the book is the one that opens it: the same numbers can demand opposite actions. In Tables 1.1 and 1.2, the numerical entries are identical. In the gender case, the drug helps and should be prescribed. In the blood-pressure case, the aggregated data is the right guide. The difference is invisible in the data; it lives entirely in the causal graph.

This is Pearl's core insight, stated in the preface and proved in every chapter: data are causally silent. No statistical test, no p-value, no regression coefficient computed from a single data set can tell you whether to look at the row totals or the stratum-specific rates. That determination requires causal knowledge external to the data — encoded in the graph — and there is no way around it.

The corollary is equallyradical: once you have the graph, you need no experiment. The do-operator, the backdoor criterion, and the front-door criterion convert a causal question about interventions into a computation on observational probabilities. Randomization is the gold standard not because it is logically necessary but because it removes the need for a graph — it guarantees, by design, that the treatment has no back-door paths. When a graph is available and its assumptions defensible, observational data can answer causal questions just as precisely.

The same data set, interpreted through two different causal graphs, can justify two opposite treatment decisions — yet no statistical test can tell you which graph is right.

Important concepts

Structural Causal Model (SCM)

A triple (V, U, F) where V is a set of endogenous variables, U a set of exogenous (background) variables, and F a set of structural equations fᵢ : Xᵢ = fᵢ(pa(Xᵢ), Uᵢ) specifying how each variable is determined by its parents and independent noise. The SCM generates both the joint distribution of V and a directed acyclic graph encoding causal structure.

Directed Acyclic Graph (DAG)

A graph with nodes (variables) and directed edges (arrows) such that no node is its own ancestor. An arrow X → Y asserts that X is a direct cause of Y in the model. The graph encodes qualitative causal assumptions independently of specific functional forms.

do-operator

The notation do(X=x) represents an external intervention that sets X to the value x by severing all incoming edges to X in the graph. P(Y | do(X=x)) is the causal effect of X on Y — the probability of outcome Y if we made X equal to x. It is distinguished from the conditional probability P(Y | X=x), which is the probability of Y given that we observed X equal to x.

Simpson's Paradox

A data phenomenon in which an association that holds in the full population is reversed (or eliminated) in every subpopulation, or vice versa. It arises whenever a confounding variable affects both treatment selection and outcome. Resolution requires knowing the causal graph, not just the data.

d-separation

A graphical criterion for reading off conditional independence relations from a DAG. Two nodes X and Y are d-separated by a set Z if every path between them is blocked by Z: paths through chains or forks are blocked when the middle node is in Z; paths through colliders are blocked when neither the collider nor any of its descendants is in Z.

Backdoor Criterion

A set of variables Z satisfies the Backdoor Criterion relative to (X, Y) if (1) no member of Z is a descendant of X and (2) Z blocks every path from X to Y that has an arrow into X. When satisfied, the causal effect is P(Y=y | do(X=x)) = Σ_z P(Y=y | X=x, Z=z) · P(Z=z).

Front-Door Criterion

A pattern that allows identification of P(Y | do(X)) even when all confounders are unobserved, by exploiting a mediating variable M such that: (1) X → M has no unobserved confounders, (2) M → Y satisfies the backdoor criterion, and (3) X blocks all backdoor paths from M to Y. The formula routes through M in two steps.

Adjustment Formula

The formula P(Y=y | do(X=x)) = Σ_z P(Y=y | X=x, Z=z) · P(Z=z), which computes the causal effect of X on Y by stratifying on the confounders Z and averaging the within-stratum effects by the marginal distribution of Z. It is the formal expression of "controlling for" Z correctly.

Counterfactual (potential outcome) Yₓ(u)

The value that Y would have taken in unit u had X been set to x, computed by solving the SCM with X replaced by x while keeping the unit's exogenous variables U fixed at their actual value u. Formally, Yₓ(u) = Y_{Mₓ}(u), where Mₓ is the model with the X equation replaced by X=x.

Probability of Necessity (PN)

P(Y₀=0 | X=1, Y=1): the probability that the outcome would not have occurred in the absence of the treatment, given that the treatment was applied and the outcome did occur. It formalizes "but-for causation" in law — would this harm have happened but for the defendant's action?

Probability of Sufficiency (PS)

P(Y₁=1 | X=0, Y=0): the probability that the treatment would have produced the outcome in a unit that was not treated and did not experience the outcome. Together with PN it characterizes the degree to which a treatment is a reliable cause.

Natural Direct Effect (NDE)

The effect of X on Y when the mediator M is held at the value it would naturally take under the control condition (X=x'), not by intervention but counterfactually. NDE = E[Y{x, M{x'}}] - E[Y_{x'}]. Used in mediation analysis to isolate the direct causal pathway from X to Y that does not pass through M.

Natural Indirect Effect (NIE)

The effect on Y of changing the mediator M from the value it would naturally take under X=x' to the value it would take under X=x, while holding X fixed at x'. NIE = E[Y{x', Mx}] - E[Y_{x'}]. Together NDE + NIE = TE (total effect) in many settings.

Average Causal Effect (ACE)

ACE = P(Y=1 | do(X=1)) - P(Y=1 | do(X=0)): the difference in outcome probability between universal treatment and universal control, in the population. Distinguished from the Risk Difference RD = P(Y=1 | X=1) - P(Y=1 | X=0), which conflates ACE with selection bias.

Effect of Treatment on the Treated (ETT)

ETT = E[Y₁ - Y₀ | X=1]: the average causal effect of the treatment, restricted to the subpopulation that actually received it. Useful for evaluating whether those who chose or received a treatment actually benefited from it.

Instrumental Variable (IV)

A variable I that (1) affects X, (2) has no direct effect on Y other than through X, and (3) is independent of all unmeasured confounders. In a linear model, the structural coefficient of X on Y is estimated as β = Cov(I, Y) / Cov(I, X). IVs allow identification of causal effects even when X has unobserved confounders.

Product Decomposition (Markov factorization)

The factorization P(x₁, …, xₙ) = ∏ᵢ P(xᵢ | pa(xᵢ)) that holds for every distribution consistent with a DAG whose noise terms are mutually independent. It encodes conditional independence assumptions and underlies both Bayesian network inference and the truncated product formula for interventions.

References and Web Links

Primary book and edition information

• Pearl, J., Glymour, M., and Jewell, N. P. Causal Inference in Statistics: A Primer. John Wiley & Sons, 2016. First and only edition. 160 pp. ISBN 9781119186847.
  • Wiley publisher page
  • Google Books preview
  • UCLA companion website (errata, TOC PDF, homework, solution manual)

Background and overview

• Wikipedia: Causal inference
• Wikipedia: Judea Pearl
• Wikipedia: Structural causal model
• Wikipedia: Simpson's paradox

Foundational works the book builds on

• Pearl, J. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000 (2nd ed. 2009). The technical predecessor that proves many results summarized in the Primer.
  • Cambridge University Press page
• Pearl, J. "Causal diagrams for empirical research." Biometrika, 82(4):669–688, 1995. The foundational paper for DAG-based causal inference.
• Rubin, D. B. "Estimating causal effects of treatments in randomized and nonrandomized studies." Journal of Educational Psychology, 66(5):688–701, 1974. The potential-outcomes framework whose connection to SCMs is established in Chapter 4.

Key ideas — d-separation and graphical models

• Wikipedia: d-separation
• Wikipedia: Backdoor criterion

Key ideas — front-door criterion

• Arel-Bundock, V. "Frontdoor adjustment for causal inference: A primer with examples in R." Blog post with worked examples.
  • Frontdoor adjustment primer (arelbundock.com)

Reviews and reception

• Tsodikov, A. Review of Causal Inference in Statistics: A Primer. International Statistical Review, 2020.
  • Wiley Online Library review
• Hogan, J. W. Review in Biometrics, 2019.
  • Wiley Online Library review
• ResearchGate review entry

Additional study resources

These are secondary resources and should be used alongside, not instead of, the original book.

• Causal Inference: A basic taster (Behavioral Data Science Lab)
• A Guide on Data Analysis — Chapter 21: Causal Inference (bookdown.org)
• Stanford Encyclopedia of Philosophy: Simpson's Paradox