The last two articles on this blog leaned on a simple piece of algebra — divide the ITT effect by the exposure rate to recover the effect on exposed users. It's a convenient shortcut, which is exactly the kind of thing worth being suspicious of. This is a check of that shortcut against the causal inference and applied statistics literature: what holds up cleanly, and where it needs a caveat.
What Checks Out
1. LATE = ITT / r Is a Real, Established Result
The EGAP Methods Guide (van der Windt) defines the Local Average Treatment Effect as the average treatment effect specifically among compliers, and shows that under the standard instrumental-variables assumptions, LATE equals the ITT effect divided by the complier share. The guide's own illustrative example: an ITT estimate of 40 combined with a complier share of 0.8 implies a LATE of .
The formal proof of this relationship traces back to Bloom (1984), and the same result appears in Angrist & Pischke's Mastering Metrics (and its predecessor, Mostly Harmless Econometrics, section 4.4.3), as well as in Angrist's own instrumental-variables lecture materials (CEDLAS):
This is the same division used in the previous articles on this blog, just written in the notation the econometrics literature uses for it.
2. The t-Statistic and p-Value Carry Over Exactly (Under Simple Division)
A 2017 policy brief from the U.S. Department of Health and Human Services' Office of Adolescent Health states plainly that under the "Bloom adjustment" — the simple-division approach used here — the t-statistic and p-value of the resulting LATE estimate are algebraically identical to those of the original ITT estimate. Dividing both the point estimate and its standard error by the same constant cancels out in the ratio:
This equivalence isn't just an algebraic curiosity that happens to work under normal approximations — a 2016 arXiv paper (Laber et al., arXiv:1604.01055) shows the same result holds even under randomization-based (permutation) inference comparing an instrumental-variables estimator to the ITT estimator directly, because the two estimators share the same numerator — a covariance term that depends on the response data — while their denominators don't depend on the response data at all.
The Important Caveat: SE(LATE) = SE(ITT) / r
The same HHS brief is equally direct about the limitation: the Bloom-adjustment standard error doesn't account for uncertainty in the compliance rate itself, citing Schochet & Chiang (2009). Treating as a known, fixed constant — which the simple division does — means the resulting standard error (and any p-value or significance claim built on it) can understate the true uncertainty.
only holds exactly when is treated as a fixed constant. In practice, is itself estimated from the same experiment, with its own sampling variability — and that variability doesn't disappear just because the formula doesn't have anywhere to put it.
Power Analysis Has the Same Problem
Per the EGAP guide, fully characterizing the statistical power of a LATE estimator is more involved than doing the same for an ITT or ATE estimator. Even though LATE is mathematically just ITT divided by the compliance rate, the ITT estimate and the compliance-rate estimate are correlated with each other, and the direction and size of that correlation vary by setting. Power figures computed from the ITT side alone can diverge meaningfully — sometimes substantially — from the actual power of a properly specified LATE-based test.
Summary
| Claim | Verdict | Source |
|---|---|---|
| LATE = ITT / r | Correct | Bloom (1984); EGAP; Angrist |
| t-statistic identical to ITT's | Correct, under Bloom adjustment | HHS Brief (2017) |
| p-value identical to ITT's | Correct, under Bloom adjustment | HHS Brief (2017); arXiv:1604.01055 |
| SE(LATE) = SE(ITT) / r | Caveat: Bloom adjustment only; ignores uncertainty in r | HHS Brief (2017); Schochet & Chiang (2009) |
| ITT power ≈ LATE power | Caveat: can differ substantially in practice | EGAP |
Practical Recommendation
- When the exposure rate is close to a fixed design constant — set mechanically by the experiment's own logic rather than fluctuating much with user behavior — the simple Bloom-adjustment standard error and p-value are reasonable to use as-is.
- When the exposure rate carries meaningful uncertainty of its own — the more common case, including something like the share of users who happened to encounter a specific card during a session — a proper two-stage least squares (2SLS) estimation is needed to get an accurate standard error, rather than the simple division used for illustration.
None of this overturns the point estimates used in the earlier articles on this blog — the LATE = ITT / r arithmetic is sound, and well-supported in the literature. It does mean that any confidence interval or significance claim built directly on top of that division should be read as a reasonable approximation, not an exact result, whenever the exposure rate itself was measured rather than fixed by design.