Instrumental Variables and Encouragement Design: The GIF Experiment, Reframed

An earlier article in the A/B Testing section worked through a GIF-thumbnail removal experiment as a practical encouragement-design problem — dilution, exposure rates, dividing one number by another, resolved by a ratio instead of a subgroup filter. This article reconstructs the same example from the other direction: as a textbook instrumental-variables (IV) problem, with the formal vocabulary that makes clear exactly what's being assumed and exactly what the resulting number does and doesn't tell you.

The Problem IV Exists to Solve

You want the causal effect of some variable $D$ on an outcome $Y$, but $D$ isn't something you can directly randomize or force. In the GIF experiment, $D$ is "the user actually saw a GIF-preview card" and $Y$ is orders. Nobody can make a user notice a GIF card — you can only make one available to be noticed. What you can randomize is $Z$, an instrument: here, assignment to control (GIFs on) versus treatment (GIFs off). Instrumental variables is the machinery for extracting a valid causal effect of $D$ on $Y$ out of something you can only randomize indirectly, through $Z$.

The Three Assumptions Doing the Work

• Relevance: $Z$ has to actually move $D$. Being assigned to control has to measurably increase the chance of seeing a GIF card compared to treatment. This is directly checkable — it's exactly the exposure rate measured in the earlier article.
• Exclusion restriction: $Z$ affects $Y$ only through $D$, never through any other channel. Being in control versus treatment can't be allowed to change orders for any reason except whether the user personally saw a GIF card. This is the assumption that's impossible to verify directly from the data and the one most worth being suspicious of.
• Independence: $Z$ itself has to be as good as randomly assigned. Guaranteed here by construction — arm assignment is the randomized variable the whole experiment is built on.

The Wald Estimator

The instrumental-variables estimate, in its simplest binary-instrument form, is a ratio of two differences — the effect of the instrument on the outcome, divided by the effect of the instrument on the endogenous variable:

The numerator is the familiar intent-to-treat effect on the outcome. The denominator — $\text{ITT}_D$ — is the "first stage": how much the instrument moved the endogenous variable itself.

Why the GIF Example Collapses to a Simple Division

Map the GIF experiment onto this directly: $Z=1$ is control (GIFs on), $Z=0$ is treatment (GIFs off), $D$ is "saw a GIF card," $Y$ is orders per active user. Because the treatment arm removes GIF cards from the category page entirely, nobody assigned to treatment can possibly satisfy $D=1$ on that surface — $E[D \mid Z=0] = 0$ by construction. That makes the first stage collapse to just the exposure rate in control:

Substituting back into the Wald estimator:

which is exactly the ITT-over-exposure-rate division used in the earlier articles. It wasn't a shortcut invented for convenience — it's the Wald IV estimator, specialized to the case where compliance is one-sided (only one arm can possibly produce exposure at all). One-sided noncompliance is what makes the general IV formula simplify down to plain division instead of requiring a full two-stage least squares regression.

What Population the Estimate Actually Describes

An IV estimate is never automatically the effect on everyone — it's the Local Average Treatment Effect (LATE), specifically the effect among compliers: units whose value of $D$ would actually change depending on their assignment to $Z$. In this one-sided design, the complier population has an unusually clean description. There are no "always-takers," since nobody in treatment can ever have $D=1$ — the surface makes it structurally impossible. And "no defiers" (nobody who'd see a GIF specifically because they were denied one) is a trivial assumption to grant here, not a leap of faith. That leaves compliers as exactly one group: users who would have seen a GIF card if placed in control. The LATE is the effect on precisely that group — not on users who'd never notice a GIF card regardless of arm, and not on the full population.

Where the Exclusion Restriction Actually Needs Scrutiny

Relevance and independence are either directly measurable or guaranteed by the experiment's own construction. Exclusion restriction is the one that requires actual judgment, and it's worth asking pointedly: does being assigned to treatment change orders through any channel besides a user personally not seeing a GIF card? If removing GIF cards also changed the page's layout density, load time, or scroll behavior in some way that affects users who never would have looked at a GIF card either way, the exclusion restriction is violated, and the resulting estimate no longer cleanly isolates "the effect of seeing a GIF card" — it partly reflects whatever else changed on the page. This is exactly the kind of assumption that's worth stating explicitly and checking against implementation details, rather than treating as a formality on the way to a clean-looking number.

Why Encouragement Design Is IV's Natural Home in Product Experiments

Most product experimentation can't force exposure to anything — you can't make a user read a notification, notice a redesigned card, or open a feature. What you can always randomize is who gets the opportunity: who's encouraged. Instrumental variables is exactly the tool built for turning a randomized opportunity into a valid causal estimate of the thing you actually can't randomize directly, as long as relevance, exclusion, and independence hold. The original walkthrough of this experiment shows the arithmetic in practice. The point of reframing it here is that the arithmetic was never a shortcut — it was instrumental variables all along, simplified by a particularly clean, one-sided compliance structure.