Gameable Engine

Troubleshooting ​

Symptom first, then the fix. If you hit something that is not here, add it — this page is part of the docs corpus small models read.

Status: planned

Entries marked "anticipated" describe failure modes the design implies. They will be replaced by observed ones as the milestones land.

Setup ​

npm install fails with EBADENGINE ​

.npmrc sets engine-strict=true and the repository requires Node >= 24. Check node --version against .nvmrc. Do not work around it by disabling engine-strict; install the right Node.

npm run typecheck says a referenced project is out of date ​

tsc -b builds project references in order. Delete stale build info and retry:

sh
node -e "for (const f of require('node:fs').globSync('**/*.tsbuildinfo', { exclude: ['**/node_modules/**'] })) require('node:fs').rmSync(f)"
npm run typecheck

ESLint reports no-bare-three-import ​

You wrote import { Mesh } from 'three'. Use three/webgpu (renderer and core), three/tsl (node materials) or three/addons/... (loaders and controls). The bare entry point pulls in the WebGL renderer and can create a second three singleton in the bundle, which breaks instanceof checks in confusing ways.

Rendering (anticipated) ​

Nothing renders and the console mentions requestAdapter ​

WebGPU is unavailable. Check chrome://gpu. Splat worlds fall back to the WebGL backend; splat characters do not — engine.caps.characters will be false and createCharacter rejects with CharacterUnsupportedError.

Splats look correct but transparent objects flicker through them ​

Splats draw after opaque geometry with depth test on and depth write off. A transparent mesh that intersects a splat volume has no correct ordering. Move the mesh out of the volume, or make it opaque.

renderer.backend.device is undefined ​

You read it before await renderer.init(). Bootstrap order is initWebGPUPatches() -> renderer.init() -> hand the device to onnxruntime-web -> create lift pipelines.

Wasm guest (anticipated) ​

The game behaves differently in npm run dev and npm run build ​

That is a parity bug between direct and wasm mode, and it is a real bug, not a configuration issue. Run the parity test; it hashes transform buffers from both modes over the same tape.

Every run produces identical "random" numbers ​

You seeded at module scope. Wizer snapshots the QuickJS heap at build time, so module-level state is frozen into the binary. Seed from env.seed() inside init().

TextDecoder is not defined (or a timer API is missing) ​

QuickJS does not ship the whole web platform. The SDK prelude polyfills what the engine needs; if you need something else, add it to the prelude rather than reaching for a browser global in guest code.

A command had no effect this frame ​

Commands are batched into frame-output and applied by the host afterwards. You cannot read back the result of a spawn in the same tick. The guest mints handles precisely so you do not need to.

Assets (anticipated) ​

Unknown asset id ​

The id is not in assets.json, or you used a path. Assets cross the boundary as string ids only. Add a manifest entry; npm run docs:lint validates the manifest against docs/schemas/assets.schema.json.

A binary file is a text stub after cloning ​

Git LFS is not installed and the file is LFS-tracked. Install LFS and git lfs pull. Note that fixtures/** and tests/fixtures/** are deliberately not LFS-tracked, so tests run on a clone without it.

Docs ​

npm run docs:lint fails with "llms bundles are stale" ​

You edited a doc but did not regenerate. Run npm run docs:llms and commit the result.

npm run docs:lint fails a size budget ​

llms-full.txt is capped at 300 KB and llms.txt at 4 KB. Either trim the page you just added or take something out of the bundle order in tools/docs/gen-llms.mjs.

ignoreDeadLinks is false on purpose. Relative links must point at files that exist, including the .md extension.

LLM evaluation harness ​

The harness is tools/llm-eval/; what it measures is explained here.

refusing to make a paid API call ​

By design. Nothing in the harness may spend money unless AOS_LLM_EVAL_LIVE=1 is set, and the guard is in the client constructor rather than the CLI, so it catches a test that builds a client by accident as well as a mistyped command.

sh
AOS_LLM_EVAL_LIVE=1 node tools/llm-eval/src/run.ts --model claude-haiku-4-5 --prompts all

To try the plumbing without spending anything, use --dry-run, which is offline and needs no key.

The scoreboard says "this run is not valid" ​

The untouched scaffold did not typecheck, so nothing downstream of it means anything. A generated game links the engine packages with file:, which means it typechecks the engine's live source — a red packages/ is a red eval. Run npm run typecheck at the repository root, fix what it reports, and run the eval again. The harness exits 2 for this rather than reporting 0%.

The scoreboard says the untouched scaffold already fails unit or build ​

Same probe, milder verdict. The harness scores the pristine scaffold on typecheck, unit and build before asking a model anything; a stage that is already failing measures the repository rather than the model, so it is skipped for every prompt and the pass rate is computed without it. Usually it means a template's own test suite is mid-rewrite. Fix it and run the eval again for a complete score.

The eval is scoring an old copy of a template ​

It should not: the shared install records the template's newest mtime and re-scaffolds over itself when the template is newer, keeping node_modules. If you have edited something the mtime walk skips (node_modules, dist, build, or a dotfile), pass --fresh to rebuild the cache from nothing.

Every prompt fails typecheck with TS2688: Cannot find type definition file ​

The scaffolded game is missing packages the template does not declare. Inside this monorepo @types/node, @webgpu/types, @types/three and @aosengine/assets are hoisted root devDependencies, so templates/fps typechecks in CI and fails the moment it is scaffolded anywhere else.

The harness detects this, adds them to its shared install, and lists them at the top of the scoreboard — so a template gap is never read as a model failure. The real fix is to add the four to templates/fps/package.json. When that happens the repair list goes empty on its own.

A prompt is reported under "Not attempted" ​

Its template is not finished. The harness wants both package.json and src/game.ts before it will scaffold from a template directory, because a half-written template would fail every stage and blame the model. Prompts in that state are excluded from the denominator, not counted as failures.

npm install runs on every prompt, and a run takes forever ​

The shared install cache was not reused. It lives at %TEMP%/aosengine-llm-eval/<template> (override with AOS_LLM_EVAL_CACHE) and every game directory gets a directory junction to its node_modules — 0.16 s instead of 18.6 s. If junctions are refused by policy or by the filesystem, the harness falls back to copying and says linkMode: "copy" in the result file. On Windows a junction needs no elevation; a symlink does, which is why the harness asks for a junction specifically.

The e2e stage is always skipped ​

It is off unless AOS_LLM_EVAL_E2E=1, and it needs a browser:

sh
npx playwright install chromium
AOS_LLM_EVAL_E2E=1 AOS_LLM_EVAL_LIVE=1 node tools/llm-eval/src/run.ts --prompts all

Playwright is resolved from the engine checkout, not from the scaffolded game — a starter project deliberately does not depend on it.

The model answered, but nothing was applied ​

Look at results/transcripts/<run>/<prompt>/response.md. Either it wrote prose instead of fenced blocks, or it asked for a path outside src/, which the harness refuses and records in rejectedPaths. A truncated bucket means it ran out of output tokens mid-file; raise --max-tokens.

The Ollama client cannot connect ​

It talks to Ollama's native API at POST /api/chat, not the OpenAI-compatible shim, because the shim normalises away the errors the eval exists to collect. Check the daemon is up, and pass --ollama-url if it is not on http://localhost:11434.

The harness's own tests are not collected by npm test ​

The root vitest.config.ts includes packages/**, templates/**, examples/** and tests/**. Run them with their own config:

sh
npx vitest run -c tools/llm-eval/vitest.config.ts

See also ​