Method

Every result on this site comes from the same fixed prompt, the same stop rule and the same browser tests. Only the stack changes.

The challenge

Each agent starts in an empty folder with one prompt (challenge v1; later runs use revised versions of the brief, v2, v3, creative-v1, creative-v2, creative-v3, and every result is labelled with its prompt): build an original voxel sandbox game in three.js, with its own name and look. The game needs procedural terrain, first-person movement with collision and jumping, breaking and placing blocks, a hotbar, a day/night cycle, particles and ambient effects, saving to localStorage, and keyboard, mouse and touch controls. It also needs a polished landing page with a live 3D preview, a gallery the game renders itself, and a Play Now button. Plain static files, three.js from one pinned CDN URL, no build step. No third-party game names, textures, sounds or assets.

The prompt also specifies a small read-only window.__game debug object and a few URL parameters (?debug=1&seed=…&time=…&daylength=…) so the tests can check behaviour exactly, not guess from pixels.

A combo

A combo is one harness (the agent program: omp, Hermes, opencrabs, mini-swe-agent, pi, …) × one engine (Strata, llama.cpp, LM Studio/MLX) × one model × its settings, on one node. One model is loaded at a time, and nothing else uses the GPU during a run.

When a run ends

Unsteered and steered

Unsteered runs get the prompt and nothing else. Steered runs follow a fixed steering protocol: at most 5 short hints, only at checkpoints where the checklist shows a failing feature, each pointing at the failing spec item and its evidence. Every steer is published verbatim with its timestamp.

Scoring

The checklist is 19 automated tests in headless Chromium. Each scores 1, 0.5 (partial) or 0, with evidence. The score is the mean × 100. The tests check that the landing page loads cleanly, the hero animates, the gallery has three or more generated non-blank images, Play Now opens the game, and the page fits a phone. In the game they check that it loads without console errors, renders, holds a frame-rate floor, has terrain (4+ block types, hills, 48+ blocks, deterministic), and that movement works (W walks, arrow keys turn, standing on ground, jumping). They also check breaking and placing, particles, the hotbar (keys, wheel, aria-selected), day and night, save and reload, touch controls, and that the build is self-contained and original.

Design is scored separately, 0–5, by a person reviewing the landing page and game screenshots at desktop and phone widths against a fixed rubric. It is never mixed into the checklist score.

The suite was validated against a hand-written reference build (marked as a test fixture, never ranked) and against broken variants of it, each of which must fail the matching test.

Caveats