before

No capture of the install prompt sitting over a game's control dock survives from before the rig existed. The bug lives on as the gate's dock-reachability assertion, which fails if anything ever covers a control again.

during

after

Walkthrough video: coming soon.

Problem

The site ships an install prompt that stays dismissed for thirty days. On my warm, logged-in profile every game looked fine. A first-time visitor on a cold profile got the prompt layered over the in-game control dock, and the taps that should have built a tower hit the prompt instead. Nobody files a bug for that. They leave.

Constraint

CI runners have no GPU, and the flag that disables it gives you a black canvas. No human visits every game in both render modes from a clean profile on a phone every day. And an LLM's opinion can never be the thing that blocks a merge.

Decision

Two rungs, built with different tools because they answer different questions. A nightly smoke sweep boots headless Chromium at an iPhone-class viewport with software WebGL, screenshots every game in 2D and 3D from a fresh browser context per shot, writes a report of console errors and failed assets, then hands the shots to Claude Code, which judges them against a written invariant list and files one labeled issue per failing game. A deterministic Playwright gate encodes the same invariants as hard assertions on every pull request: dock unobstructed, board large enough, WebGL context alive, scripted interaction error-free.

• One LLM loop that both judges and blocks

  A non-deterministic gate trains people to re-run until green. The judge files evidence; the gate blocks by determinism and uploads traces, so a failure is replayable, not arguable.

• Test from my own logged-in profile

  That profile had dismissed the install prompt. Every shot uses a fresh context, so the cold-profile bug can never be masked by a tester who already dismissed it.

What shipped

The capture harness, the nightly workflow that hands the shots to Claude Code and files issues, the per-PR gate with a reference spec, a stale-game advisory mode, and an operator that drains the playtest backlog into reviewed pull requests.

What I'd do differently

The gate then broke for the runner's reasons twice: one game's Claim button never stabilized and blocked every pull request (#1415), and the runner ran out of disk, not memory (#2381). Each was its own fix, because a gate that is red for the wrong reason is a gate people learn to skip.

Verify it yourself

• Run the deterministic merge gatenpm run playtest:gate
• Run the nightly smoke sweep locallynpm run playtest:smoke
• The reference gate spec
• The nightly workflow that hands the shots to a judge
• The per-PR gate workflow

If you ask me about this

Why software WebGL?

CI runners have no GPU. Routing the renderer to SwiftShader draws the full 3D pipeline on CPU, and the harness inspects the live WebGL context afterward because flags drift across Chromium versions.

What can the nightly LLM run change?

Nothing. It is read-only: its whole output is an issue naming the game, the mode, the invariant, and the screenshot. The fix is always a separate reviewed pull request.

Evidence

1. Wire up the playtest games QA harness and gate

   #722 · 2026-06-07 · feature

2. Add stale-game advisory mode to the playtest harness

   #725 · 2026-06-07 · feature

3. playtest smoke: inspect WebGL after the game starts, not at the cover screen

   #742 · 2026-06-07 · feature

4. Agents Store: ship the game-builder operator — games.fix drains the playtest backlog

   #1002 · 2026-06-11 · bugfix

5. the-pull: fix the Claim button never stabilizing (unblocks the playtest gate)

   #1415 · 2026-06-18 · bugfix

6. Playtest gate: the runner runs out of disk, not memory

   #2381 · 2026-09-18 · feature