What We Learned Building A/B Testing for Encrypted Links

A/B testing a link sounds simple until you remember that we don’t know where a link points until the exact moment someone clicks it.

Most A/B testing infrastructure assumes the server can see the destination — that’s how it decides which variant to serve and logs which version won. Our server can’t see destinations. Every one of them is encrypted client-side before it ever reaches us. So the obvious implementation — “look at the destination URLs, pick one based on the split percentage, redirect” — was never available to us.

The actual design

Instead of the server choosing between plaintext destinations, each variant is stored as its own independently encrypted target under the same parent link. When a click comes in, we run a weighted random selection across the active variants using the weights you configured — variant IDs and weights only, never their contents. Only after a variant is picked does the browser fetch that specific ciphertext and decrypt it using the key from the URL fragment, the same as it would for any single-destination link.

The practical effect: we can tell you “Variant B got roughly 60% of clicks,” but at no point in that process did our infrastructure need to know, or ever learn, what Variant A or Variant B actually redirect to.

What we chose not to build, at least not yet

The honest version of this: our variant selection is per-click random, not per-visitor sticky. If the same person clicks your link twice, they could land on Variant A the first time and Variant B the second — there’s no cookie or session mechanism keeping them on one variant across visits. For a lot of A/B testing use cases that’s a perfectly fine tradeoff, since results converge over enough clicks regardless of whether any individual visitor is consistent across sessions. For use cases where per-visitor consistency actually matters — testing something a visitor might notice or return to — it’s a real limitation, and we’d rather say so than let the phrase “A/B testing” imply more than what’s actually built.

Building sticky assignment without weakening the zero-knowledge model is a genuinely harder problem than the random version, because it means persisting an assignment decision somewhere without that assignment leaking anything about which encrypted variant was shown. It’s on our list. It isn’t shipped.

Why we’re building this at all

We could have skipped A/B testing entirely, or built it the easy way by making an exception to client-side encryption just for variants. We didn’t want to be a zero-knowledge shortener with an asterisk that says “except for the features marketing actually cares about.” If a feature can’t be built without seeing plaintext destinations, our answer needs to be “figure out how,” or “not yet” — never a quiet exception.