The Math Behind Encouragement Design: A Sensitivity Table for Converting ITT to LATE

The encouragement-design article on this blog worked out a single number: the undiluted effect of GIF exposure, recovered from a diluted ITT estimate by dividing out the exposure rate. This is the reference version — where that division actually comes from, a full sensitivity table showing how much the answer moves as the exposure rate changes, and a pointer to where the same idea shows up in the wider instrumental-variables literature.

Restating the Problem

Overall, control (GIFs on) outperformed treatment (GIFs off) on the guardrail metric, but control mixes users who actually saw a GIF card with users who didn't happen to encounter one that session. What we actually want to know: how large is the effect specifically among users who were exposed?

Setting Up the Formula

Let $r$ be the share of control users who saw at least one GIF card, and let $X$ be the true effect among those exposed users. The overall (ITT) effect is that true effect, diluted by the exposure rate:

Users who were never exposed can't have been affected by something they never saw — their individual contribution to the effect is zero by construction. The entire observed ITT effect is generated by the exposed fraction alone, spread thin across everyone.

The Sensitivity Table

Holding the observed ITT effect fixed at $2.0\%$ and varying the assumed exposure rate $r$ shows how sensitive the recovered effect is to how diluted the sample actually was:

The actual measured exposure rate in this experiment came out to $r = 33\%$ — between the 30% and 40% rows above — giving a recovered effect of $2.0\% / 0.33 = 6.06\%$, consistent with the interpolation.

Why a Lower Exposure Rate Implies a Bigger Effect

The pattern in the table is monotonic and mechanical: the total effect is fixed by what was actually observed, and dividing a fixed quantity across a smaller exposed population necessarily means each exposed person is carrying a larger share of it. Lower exposure doesn't create a bigger effect out of nothing — it reveals that the same fixed, observed effect was being produced by fewer people all along.

This Is an Instrumental-Variables Estimator

Solving $\text{ITT} = r \times X$ for $X$ is exactly the Local Average Treatment Effect (LATE) from the instrumental-variables literature — no need to invoke the full IV machinery to use it, since dividing by the exposure (compliance) rate is the entire idea:

Where This Shows Up Elsewhere

This exact construction has a name in the applied experimentation literature: encouragement design. Benjamin Elbers, a data scientist at Spotify, covers this directly in "Encouragement Designs and Instrumental Variables for A/B Testing" (Spotify Engineering), framing the same ratio as:

where $E[D \mid Z=1]$ — the expected compliance rate given assignment to encouragement — plays exactly the role $r$ plays here. Different notation, same division.

A Caveat Worth Flagging

Everything above treats $r$ as a known, fixed number once measured. Strictly, $r$ is itself an estimate with its own sampling uncertainty, and a fully rigorous confidence interval on $X$ needs to account for that rather than simply dividing the original standard error by $r$. This blog has a dedicated, free article checking exactly this point against the published literature — what holds under the simple-division shortcut, and where it needs a proper two-stage least squares correction instead.