One prompt · every local stack · all playable

Local agents build a game.
We publish every one.

The same fixed prompt asks each local coding agent to build an original voxel sandbox game in three.js: terrain, movement, blocks, a hotbar, day and night, saving, touch, and a landing page. Every combination of harness, inference engine and model runs on our own hardware. Then a browser test suite scores what it built, and you can play it.

30combos run
170results
3nodes
19automated checks

See the resultsHow it works

Gallery

Gameplay from the top builds. Every image is a real screenshot taken by the test suite.

Swift 1.5 IQ3_XXS (Strata): game-daySwift 1.5 IQ3_XXS (Strata) · ompSwift 1.5 IQ3_XXS (Strata): game-nightSwift 1.5 IQ3_XXS (Strata) · ompSwift 1.5 IQ3_XXS (Strata): game-breakSwift 1.5 IQ3_XXS (Strata) · ompQwen3.8 Flash Next IQ3_XXS (Strata): game-dayQwen3.8 Flash Next IQ3_XXS (Strata) · mswea-strata-flashQwen3.8 Flash Next IQ3_XXS (Strata): game-nightQwen3.8 Flash Next IQ3_XXS (Strata) · mswea-strata-flashQwen3.8 Flash Next IQ3_XXS (Strata): game-breakQwen3.8 Flash Next IQ3_XXS (Strata) · mswea-strata-flashQwen3.8 Flash Next IQ3_XXS (Strata): game-dayQwen3.8 Flash Next IQ3_XXS (Strata) · ompQwen3.8 Flash Next IQ3_XXS (Strata): game-nightQwen3.8 Flash Next IQ3_XXS (Strata) · ompQwen3.8 Flash Next IQ3_XXS (Strata): game-breakQwen3.8 Flash Next IQ3_XXS (Strata) · ompQwen3.8 Flash Next IQ3_XXS (Strata): game-dayQwen3.8 Flash Next IQ3_XXS (Strata) · hermes-flashQwen3.8 Flash Next IQ3_XXS (Strata): game-nightQwen3.8 Flash Next IQ3_XXS (Strata) · hermes-flashQwen3.8 Flash Next IQ3_XXS (Strata): game-breakQwen3.8 Flash Next IQ3_XXS (Strata) · hermes-flash

Builds

By machine: RTX 3060 (12 GB) (149) · Mac mini M4 (24 GB) (17) · Orange Pi 5 Plus (4)

RTX 3060 (12 GB)

Mac mini M4 (24 GB)

Orange Pi 5 Plus

Reference

Comparison

Click a column to sort. Checklist is the automated score out of 100. Design is a 0–5 rubric score from a human review, shown separately. Steered runs had a few logged hints; unsteered runs had none.

ModelHarnessEngineNodeVariantPromptChecklistDesignModel timeTool timeTool callsErrorsDecode tok/sSteersStop
Swift 1.5 IQ3_XXS (Strata) ompstrataomarchyunsteered + 1 steerv1 100 – 9hr 24mins 3hr 29mins 520 35 21.3 1 done
Qwen3.8 Flash Next IQ3_XXS (Strata) mswea-strata-flashstrataomarchyunsteered + 2 steersv1 100 – 1hr 39mins 13mins 71 5 21.9 1 done
Qwen3.8 Flash Next IQ3_XXS (Strata) ompstrataomarchyunsteeredv1 100 – 4hr 30mins 33mins 293 21 17.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 1hr 35mins 19mins 60 5 12.8 1 done
Qwen3.8 Flash Next IQ3_XXS (Strata) hermes-flashstrataomarchyunsteeredv3 100 – 7hr 2mins 3hr 7mins 294 38 15.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 1hr 51mins 26mins 71 0 12.3 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 1hr 17mins 15mins 49 0 13.8 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 1hr 39mins 19mins 55 2 23.0 1 crash
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 1hr 8mins 16mins 51 5 23.9 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 1hr 16mins 10mins 54 2 18.5 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 100 – 3hr 35mins 0s 44 2 23.4 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 2hr 1min 12mins 70 5 22.0 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 8mins 16mins 48 0 22.5 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 10mins 8mins 78 8 22.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 48mins 21mins 64 7 22.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 18mins 10mins 82 9 22.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 51mins 16mins 64 7 22.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 55mins 22mins 68 6 21.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 36mins 25mins 93 1 21.7 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 5mins 32mins 104 8 21.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 2 steerscreative-v4 100 – 1hr 35mins 35mins 61 3 22.4 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 45mins 11mins 56 1 22.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 4mins 1hr 30mins 88 8 21.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 11mins 16mins 77 0 22.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 2mins 15mins 95 5 21.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 39mins 9mins 49 0 21.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 22mins 25mins 77 7 22.2 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 56mins 7mins 65 3 21.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 58mins 16mins 88 6 21.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 58mins 18mins 55 7 22.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 42mins 6mins 50 2 21.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 6mins 14mins 56 0 22.2 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 52mins 7mins 54 4 21.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 17mins 13mins 94 2 21.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 14mins 12mins 46 4 18.9 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 2mins 20mins 66 1 22.2 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 50mins 7mins 51 1 21.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 38mins 8mins 43 1 22.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 45mins 13mins 60 2 21.5 0 done
Swift 1.5 IQ3_XXS (Strata) ompstrataomarchycreativecreative-v4 100 – 2hr 45mins 35mins 319 27 21.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 41mins 11mins 74 5 15.0 0 done
Swift 1.5 IQ3_XXS (Strata) ompstrataomarchycreativecreative-v4 100 – 2hr 39mins 1hr 40mins 245 29 21.8 0 done
Swift 1.5 IQ3_XXS (Strata) hermes-swiftstrataomarchycreativecreative-v4 100 – 1hr 47mins 11mins 136 13 21.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 44mins 15mins 59 1 24.9 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 1min 25mins 104 6 28.6 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 30mins 28mins 44 1 29.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 7mins 23mins 92 4 26.2 0 context_full
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 33mins 21mins 55 0 29.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 53mins 42mins 90 9 24.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 57mins 33mins 83 2 28.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 39mins 9mins 63 2 29.3 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 56mins 11mins 85 3 29.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 57mins 21mins 78 3 23.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 8mins 19mins 67 0 19.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 57mins 7mins 60 1 17.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 41mins 17mins 71 2 15.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 100 – 1hr 43mins 21mins 69 2 17.5 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 58mins 20mins 64 2 17.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 10mins 13mins 63 4 16.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 17mins 23mins 62 1 16.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 59mins 14mins 52 3 16.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 45mins 21mins 102 9 14.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 58mins 26mins 76 4 25.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 44mins 23mins 74 4 26.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 38mins 13mins 54 3 27.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 36mins 5mins 52 1 27.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 30mins 7mins 44 1 27.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 44mins 14mins 63 6 26.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 55mins 17mins 70 1 25.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 31mins 3mins 50 3 27.2 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 33mins 10mins 49 4 27.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 48mins 25mins 59 4 26.2 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 40mins 6mins 57 4 27.2 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 36mins 14mins 35 2 26.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 38mins 22mins 58 4 27.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 48mins 13mins 72 6 24.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 52mins 42mins 78 3 25.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 43mins 21mins 63 3 25.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 46mins 19mins 67 3 26.2 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 29mins 5mins 42 4 27.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 39mins 8mins 63 1 26.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 37mins 10mins 59 1 27.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 47mins 5mins 59 2 23.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 1hr 26mins 90 2 23.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 44mins 12mins 64 1 26.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 100 – 58mins 32mins 72 3 25.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchyunsteered + 2 steersv1 97.5 – 2hr 10mins 29mins 58 8 22.4 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.5 – 40mins 37mins 62 5 26.3 0 done
Swift 1.5 IQ3_XXS (Strata) hermes-swiftstrataomarchycreativecreative-v4 97.4 – 2hr 42mins 43mins 210 36 21.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.4 – 49mins 11mins 70 6 26.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.4 – 53mins 13mins 83 1 24.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 97.2 – 1hr 5mins 25mins 52 5 18.6 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 97.2 – 3hr 23mins 40mins 86 6 22.1 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 3 steerscreative-v4 97.2 – 2hr 34mins 1hr 42mins 86 4 22.7 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 97.2 – 1hr 6mins 18mins 55 5 21.0 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 35mins 9mins 52 1 21.6 0 done
Swift 1.5 IQ3_XXS (Strata) ompstrataomarchycreativecreative-v4 97.2 – 1hr 52mins 17mins 275 47 20.0 0 done
Swift 1.5 IQ3_XXS (Strata) hermes-swiftstrataomarchycreativecreative-v4 97.2 – 1hr 39mins 12mins 132 8 21.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 97.2 – 1hr 33mins 37mins 88 1 18.2 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 1hr 17mins 24mins 71 2 16.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 57mins 7mins 53 2 16.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 59mins 22mins 58 5 16.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 97.2 – 1hr 25mins 12mins 98 7 16.3 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 97.2 – 1hr 3mins 51mins 37 4 18.7 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 32mins 9mins 48 1 27.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 34mins 7mins 45 3 26.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 52mins 23mins 79 8 24.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 46mins 10mins 70 0 26.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 41mins 7mins 67 4 25.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 40mins 11mins 59 7 27.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 50mins 14mins 67 3 26.2 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 1hr 13mins 84 5 26.7 0 context_full
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 31mins 4mins 52 6 27.0 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 1hr 2mins 26mins 86 2 25.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 97.2 – 1hr 9mins 15mins 61 0 24.4 0 context_full
Qwen3.8 Flash Next Coder IQ1_M (Strata) mswea-strata-coderstrataomarchyunsteered + 3 steersv1 95 – 2hr 5mins 30mins 30 3 18.9 1 done
Qwen3.8 Flash Next Coder IQ1_M (Strata) ompstrataomarchyunsteered + 1 steerv1 95 – 2hr 36mins 37mins 228 28 19.5 1 done
Swift 1.5 IQ3_XXS (Strata) hermes-swiftstrataomarchyunsteeredv3 95 – 3hr 19mins 39mins 150 8 13.0 0 done
Swift 1.5 IQ3_XXS (Strata) ompstrataomarchyunsteeredv3 95 – 2hr 26mins 11mins 244 19 23.5 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 95 – 45mins 12mins 66 5 24.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 95 – 45mins 25mins 81 7 26.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 45mins 26mins 65 4 21.8 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 43mins 19mins 59 2 22.3 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 29mins 12mins 44 4 29.6 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 94.7 – 36mins 7mins 43 1 27.0 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 1hr 4mins 23mins 46 2 16.4 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 1hr 5mins 31mins 72 15 25.9 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 1hr 3mins 47mins 86 3 25.1 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 94.7 – 49mins 12mins 49 1 27.3 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.7 – 1hr 2mins 24mins 80 8 24.7 0 done
Qwen3.8 Flash Next IQ3_XXS (Strata) mswea-strata-flashstrataomarchycreative + 1 steercreative-v3 94.4 – 1hr 44mins 23mins 78 10 17.0 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 94.4 – 1hr 31mins 11mins 66 2 13.5 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 94.4 – 1hr 34mins 26mins 103 1 23.3 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 94.4 – 1hr 34mins 29mins 45 4 21.9 1 crash
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v3 94.4 – 1hr 5mins 20mins 41 1 19.6 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 94.4 – 40mins 9mins 46 0 29.3 1 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreativecreative-v4 94.4 – 42mins 7mins 51 2 26.9 0 done
Qwen3.8 Flash Next Coder IQ1_M (Strata) mswea-strata-coderstrataomarchyunsteeredv3 92.5 – 2hr 27mins 1hr 23mins 150 15 12.7 0 done
Swift 1.5 IQ3_XXS (Strata) mswea-strata-swift15strataomarchycreative + 1 steercreative-v4 92.1 – 1hr 18mins 12mins 69 8 22.2 1 done
Qwen3.8 Flash Next Coder IQ1_M (Strata) mswea-strata-coderstrataomarchycreative + 1 steercreative-v3 91.7 – 1hr 16mins 31mins 66 1 19.2 1 done
Qwen3.8 Flash Next IQ3_XXS (Strata) hermes-flashstrataomarchyunsteeredv1 85 – 2hr 10mins 44mins – – 21.8 0
Qwen3.6-35B-A3B UD-Q3_K_M (LM Studio llama.cpp, Mac mini) mswea-mac-qwen36lmstudiomacunsteered + 1 steerv3 85 – 2hr 41mins (wall) – 60 3 – 1 done
Qwen3.8 Flash Next Coder IQ1_M (Strata) deepseekstrataomarchyunsteeredv3 82.5 – 4hr 26mins 6mins – – 11.3 0 done
Qwen3.8 Flash Next Coder IQ1_M (Strata) hermes-coderstrataomarchyunsteeredv3 76.3 – 2hr 24mins 50mins 141 19 14.0 0 done
Gemma 4 12B QAT (LM Studio, Mac mini) omplmstudiomacunsteeredv1 68.4 – 20mins 4s 5 1 – 0 done
Gemma 4 26B-A4B QAT MLX 4-bit (LM Studio, Mac mini) omplmstudiomacunsteeredv1 39.5 – 23mins (wall) – 19 15 – 0 crash
Gemma 4 12B QAT (LM Studio, Mac mini) mswea-mac-gemma12blmstudiomacunsteeredv1 36.8 – 53mins 12mins 13 1 – 0 done
Gemma 4 12B QAT (LM Studio, Mac mini) mswea-mac-gemma12blmstudiomacunsteeredv1 36.8 – 1hr 7mins 10mins 16 2 – 0 done
Gemma 4 12B QAT (LM Studio, Mac mini) omplmstudiomacunsteeredv1 34.2 – 37mins 1min 13 11 – 0 loop
Gemma 4 12B QAT (LM Studio, Mac mini) mswea-mac-gemma12blmstudiomacunsteeredv1 28.9 – 37mins 52mins 8 1 – 0 done
Underdog Saluki 27B IQ2-mix (llama.cpp) hermes-salukillamacppomarchyunsteeredv3 28.9 – 2hr 20mins 6mins 125 13 16.7 0 done
Gemma 4 12B QAT (LM Studio, Mac mini) mswea-mac-gemma12blmstudiomacunsteeredv1 26.3 – 1hr 25mins 30mins 22 0 – 0 done
Gemma 4 26B-A4B QAT MLX 4-bit (LM Studio, Mac mini) mswea-mac-gemma26blmstudiomacunsteeredv1 26.3 – 24mins (wall) – 15 6 – 0 done
LFM2.5-24B-A2B Q4_K_M (llama.cpp, expert offload) mswea-llamacpp-lfm25llamacppomarchycreative + 3 steerscreative-v3 25 – 1hr 11mins 1hr 19mins 55 4 47.4 1 killed
LFM2.5-24B-A2B Q4_K_M (llama.cpp, expert offload) hermes-lfm25llamacppomarchycreative + 3 steerscreative-v4 22.2 – 10hr 35mins 15mins 158 61 24.4 1 crash
Swift 1.5 IQ3_XXS (Strata) hermes-swiftstrataomarchyunsteeredv1 21.1 – 1hr 47mins 53mins 42 8 26.7 0 stall
Qwen3.6-35B-A3B UD-Q3_K_M (LM Studio llama.cpp, Mac mini) mswea-mac-qwen36lmstudiomacunsteeredv2 21.1 – 37mins 5mins 25 2 – 0 crash
Qwen3.8-9B Distill Q8_0 GGUF (empero-ai, LM Studio llama.cpp, Mac mini) omplmstudiomacunsteeredv3 16.7 2.0 2hr 20mins 1hr 8mins 166 24 – 0 done
Gemma 4 12B QAT (LM Studio, Mac mini) omplmstudiomacunsteeredv1 13.2 – 1hr 7mins 5mins 20 18 – 0 loop
Gemma 4 26B-A4B QAT MLX 4-bit (LM Studio, Mac mini) omplmstudiomacunsteeredv1 10.5 – 18mins (wall) – 12 11 – 0 done
Gemma 4 26B-A4B QAT MLX 4-bit (LM Studio, Mac mini) omplmstudiomacunsteeredv1 10.5 – 14mins (wall) – 6 4 – 0 crash
Gemma 4 12B QAT (LM Studio, Mac mini) omplmstudiomacunsteeredv1 – – 14mins 9s 9 8 – 0 loop
Gemma 4 12B QAT (LM Studio, Mac mini) omplmstudiomacunsteeredv1 – – 1hr 37mins 30mins 18 18 – 0 stall
KAT-Coder V2.5 Dev OptiQ 4-bit REAP-18B (optiq serve / mlx-lm, Mac mini) mswea-mac-katmlxmacunsteeredv2 – – 4mins (wall) – 0 0 – 0 crash
Underdog Saluki 27B IQ2-mix (llama.cpp) mswea-llamacpp-salukillamacppomarchyunsteeredv3 – – 31mins 13s 23 4 15.1 0 done
LFM2.5-24B-A2B Q4_K_M (llama.cpp, expert offload) hermes-lfm25llamacppomarchycreativecreative-v4 – – 20s 7s 0 0 47.9 0 killed
Qwen3-4B Q4_0 (Pi CPU, 2 A76 threads) pi-builderllama.cpppipi+steer-v2v1 – – 6hr 5mins 14hr 14mins 51 37 0.6 7 stuck
qwen3.5-4b-16k pi-builderllama.cpppipi-onlyv1 – – 10hr 2mins (wall) – 53 – – 1 stopped-for-completion-run
qwen3-4b-q4_k_m pi-builderllama.cpppipi-onlyv1 – – 5hr 56mins (wall) – 17 5 – 1 stuck
Qwen3.5-4B (Pi NPU, rkllm, 16k) pi-builderllama.cpppipi+steerv3 – – 25hr 48mins (wall) – 152 – – 9 cap-24h
Hand-written reference (test fixture) fixture fixturenoneomarchyunsteeredv1 100 – 3s (wall) – – – – 0 done

Engine speed by node

Decode and prefill speeds measured on each node, including the Orange Pi lab.

NodeEngineModelDecode tok/sPrefill tok/sContextNotesSource
piRKLLM runtime 1.3.1 via lab rkllama (jaylfc/rkllama feat/rknpu-1.3.1 7e9f494, :8022)Qwen3-1.7B w8a8, 16K ctx, rkllm-toolkit 1.3.1 (our conversion)7.4192.2–pp512ours (029)
piRKLLM runtime 1.3.1 via lab rkllama (jaylfc/rkllama feat/rknpu-1.3.1 7e9f494, :8022)Qwen3-1.7B w8a8, 16K ctx, rkllm-toolkit 1.3.1 (our conversion)6.4135.2–pp2048ours (029)
piRKLLM runtime 1.3.1 via lab rkllama (jaylfc/rkllama feat/rknpu-1.3.1 7e9f494, :8022)Qwen3-1.7B w8a8, 16K ctx, rkllm-toolkit 1.3.1 (our conversion)5.185.5–turn2-after-2kours (029)
piRKLLM runtime 1.3.1 via lab rkllama (jaylfc/rkllama feat/rknpu-1.3.1 7e9f494, :8022)Qwen3-1.7B w8a8, 16K ctx, rkllm-toolkit 1.3.1 (our conversion)2.146.2–pp8192ours (029)