Running A/B Tests at the Edge

You split traffic for an experiment, and the results are garbage: the same user sees variant A on one request and B on the next, conversion numbers swing, and your CDN serves a cached B page to a bucketed-A user. Every one of those symptoms is the same root failure — non-deterministic bucketing and a cache key that ignores the variant. Running A/B tests at the edge fixes both by assigning a stable bucket at the V8 isolate before the request reaches origin, then teaching the cache to vary by it.

This guide is part of Vercel Edge Runtime vs Cloudflare Workers. It builds a deterministic bucketing function, persists the assignment in a cookie, and varies the cache by bucket — running the same way on both platforms.

Root cause: random bucketing and a variant-blind cache key

Two independent bugs produce flaky experiments:

1. Random assignment per request. Calling Math.random() on every hit re-rolls the bucket, so a returning user flips variants and your sample is meaningless. Bucketing must be a pure function of a stable identifier (a sticky id from a cookie), so the same id always lands in the same bucket.
2. A cache key that ignores the variant. If a PoP caches the response without folding the bucket into the key, the first variant served becomes the variant everyone gets. The cache must Vary on the bucket, or you must rewrite the cache key to include it.

The fix is deterministic hashing plus an explicit cache-key strategy. Both platforms expose crypto.subtle.digest, so the hash is identical; only the cache mechanism differs — Vercel relies on Vary and rewrites, Cloudflare lets you set the cache key directly via the Cache API.

Step 1: Hash a stable id into a deterministic bucket

Bucketing must be reproducible. Hash the id with SHA-256, take the first bytes as an integer, and map it onto your weighted split. The same id always yields the same bucket.

// bucket.ts — runtime-agnostic deterministic bucketing
export type Bucket = 'A' | 'B';

export async function assignBucket(id: string, weightA = 0.5): Promise<Bucket> {
  const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(id));
  const view = new DataView(digest);
  // First 4 bytes → unsigned 32-bit int → fraction in [0, 1)
  const fraction = view.getUint32(0) / 0xffffffff;
  return fraction < weightA ? 'A' : 'B';
}

Salt the id per experiment (e.g. assignBucket(exp_checkout:${id})) so a user is not correlated across unrelated experiments. The split weight is a single tunable.

Step 2: Read or mint the sticky id and bucket on Vercel Edge

On the first visit there is no id, so mint one with crypto.randomUUID(), assign a bucket, and persist both in cookies. On later visits, reuse them.

// middleware.ts — Vercel Edge Middleware
import { NextRequest, NextResponse } from 'next/server';
import { assignBucket } from './bucket';

export async function middleware(req: NextRequest) {
  let id = req.cookies.get('ab_id')?.value;
  let bucket = req.cookies.get('ab_bucket')?.value as 'A' | 'B' | undefined;

  if (!id || !bucket) {
    id = id ?? crypto.randomUUID();
    bucket = await assignBucket(`exp_home:${id}`);
  }

  // Rewrite so the variant is part of the path the cache sees
  const url = req.nextUrl.clone();
  url.pathname = `/_variants/${bucket}${url.pathname}`;

  const res = NextResponse.rewrite(url);
  res.cookies.set('ab_id', id, { path: '/', maxAge: 60 * 60 * 24 * 90 });
  res.cookies.set('ab_bucket', bucket, { path: '/', maxAge: 60 * 60 * 24 * 90 });
  res.headers.set('vary', 'cookie');
  return res;
}

export const config = { matcher: ['/'] };

Rewriting to a variant-specific path makes the bucket part of the URL the cache keys on, which is the most reliable way to keep variants separate on Vercel.

Step 3: Do the same on Cloudflare Workers with an explicit cache key

Cloudflare lets you set the cache key directly through the Cache API, so you fold the bucket into a synthetic key instead of rewriting the path.

// src/worker.ts — Cloudflare Workers
import { assignBucket } from './bucket';

export default {
  async fetch(req: Request, env: unknown, ctx: ExecutionContext): Promise<Response> {
    const cookies = req.headers.get('cookie') ?? '';
    let id = /(?:^|;\s*)ab_id=([^;]+)/.exec(cookies)?.[1];
    let bucket = /(?:^|;\s*)ab_bucket=([^;]+)/.exec(cookies)?.[1] as 'A' | 'B' | undefined;

    if (!id || !bucket) {
      id = id ?? crypto.randomUUID();
      bucket = await assignBucket(`exp_home:${id}`);
    }

    // Synthetic cache key includes the bucket so variants cache separately
    const cacheUrl = new URL(req.url);
    cacheUrl.searchParams.set('__ab', bucket);
    const cacheKey = new Request(cacheUrl.toString(), req);
    const cache = caches.default;

    let res = await cache.match(cacheKey);
    if (!res) {
      res = await fetch(req); // origin renders the bucket via a forwarded header below
      res = new Response(res.body, res);
      res.headers.set('cache-control', 'public, max-age=300');
      ctx.waitUntil(cache.put(cacheKey, res.clone()));
    }

    res = new Response(res.body, res);
    res.headers.append('set-cookie', `ab_id=${id}; Path=/; Max-Age=7776000`);
    res.headers.append('set-cookie', `ab_bucket=${bucket}; Path=/; Max-Age=7776000`);
    return res;
  },
};

The synthetic __ab query parameter only affects the cache key — origin still receives the bucket via a forwarded header or the cookie, never via the URL the user sees.

Configuration

Vercel needs the matcher scoped to the tested routes. Cloudflare needs nothing beyond a current compatibility date.

// Vercel: only intercept tested routes to avoid bucketing static assets
export const config = { matcher: ['/', '/pricing'] };

// wrangler.jsonc — Cloudflare Workers
{
  "name": "ab-edge",
  "main": "src/worker.ts",
  "compatibility_date": "2026-01-01"
}

Local vs production divergence

The trap: an experiment that looks balanced locally serves one variant to everyone in production because the cache key did not include the bucket. Validate the cache-key strategy, not just the hash.

Validation with Vitest

Determinism is the property worth testing: the same id always buckets the same way, and the split is roughly correct over many ids.

// bucket.test.ts
import { describe, it, expect } from 'vitest';
import { assignBucket } from './bucket';

describe('assignBucket', () => {
  it('is deterministic for the same id', async () => {
    const a = await assignBucket('exp:user-123');
    const b = await assignBucket('exp:user-123');
    expect(a).toBe(b);
  });

  it('respects the split weight within tolerance', async () => {
    let countA = 0;
    const n = 2000;
    for (let i = 0; i < n; i++) {
      if ((await assignBucket(`exp:user-${i}`, 0.3)) === 'A') countA++;
    }
    const ratio = countA / n;
    expect(ratio).toBeGreaterThan(0.26);
    expect(ratio).toBeLessThan(0.34);
  });
});

Named pitfalls

1. Bucketing with Math.random(). Re-rolls every request, so users flip variants. Fix: hash a sticky id with crypto.subtle.digest.
2. Caching without folding the bucket into the key. The first variant served becomes everyone’s. Fix: rewrite to a variant path (Vercel) or set a synthetic cache key (Cloudflare).
3. Putting the variant in the user-visible URL. Pollutes analytics and lets users self-select. Fix: keep the variant in a rewrite target or synthetic cache key, not the address bar.
4. Forgetting Vary: cookie. A shared PoP cache serves one user’s variant to another. Fix: set Vary: cookie or include the bucket in the key explicitly.
5. Not salting per experiment. Users get correlated across experiments, biasing interactions. Fix: prefix the id with an experiment name before hashing.

Production deployment checklist

• Bucketing is a pure function of a sticky id, never Math.random() Bucketing is a pure function of a sticky id, never `Math.random()` per request
• Sticky id persisted in a long-lived HttpOnly Sticky id persisted in a long-lived `HttpOnly` cookie
• Bucket folded into the cache key (variant rewrite or synthetic key)
• Vary: cookie `Vary: cookie` set wherever a shared cache is involved
• Id salted per experiment to avoid cross-experiment correlation
• Variant never appears in the user-visible URL
• Cookie set Secure and SameSite=Lax Cookie set `Secure` and `SameSite=Lax` in production
• Determinism and split distribution verified in CI

Frequently Asked Questions

Related

• Vercel Edge Runtime vs Cloudflare Workers
• When to use edge vs serverless functions for API calls
• Vercel Edge vs Cloudflare Workers for authentication
• Streaming responses: Vercel Edge vs Cloudflare Workers