Gaussian splats

Worlds are gaussian splats rendered by three.js r186's native GaussianSplat on a WebGPURenderer. There are two paths and they exist for different reasons.

Static is for worlds. loadSplat decodes SPZ (recommended), PLY, SPLAT or KSPLAT — or a glTF carrying KHR_gaussian_splatting — and createSplatObject puts an unmodified GaussianSplat in the scene. Nothing is forked: a world never changes, so three's one-time repack and its camera-driven re-sort are exactly right.

Dynamic is for characters. createAnimatedSplat allocates a fixed capacity of gaussians with no CPU copy at all and hands out the four GPUBuffers behind them, so a compute shader writes centres, covariances and colours straight into the buffers the vertex stage reads. That needs a maintained fork of GaussianSplat.js; see ADR 0005 for why a subclass cannot do it.

Status

Implemented in milestone M1 (static) and M3 (dynamic). The numbers on this page were measured on the reference Windows GPU box; regenerate them with examples/splat-viewer/?bench=1.

What a splat is, on the GPU

GaussianSplat repacks its source geometry once into four storage buffers and never looks at the source attributes again except for raycasting, bounds and the WebGL CPU sort. Those four buffers are the real data model, and a producer must match them exactly:

Buffer	WGSL type	Contents	Bytes/splat
center	array<vec4<f32>>	xyz = centre in local space, w unused	16
covarianceA	array<vec4<f32>>	(c00, c01, c02, c11)	16
covarianceB	array<vec4<f32>>	(c12, c22, 0, 0)	16
color	array<u32>	pack4x8unorm(vec4(r, g, b, a))	4

The six covariance floats are the upper triangle of the symmetric 3×3 covariance Σ = (R·S)(R·S)ᵀ, in the order c00, c01, c02, c11, c12, c22 — the order three's own writeCovariance writes, split 4/2 across the two vec4 buffers. covarianceB.zw is padding and is never read.

Two consequences worth internalising:

• 52 bytes per gaussian. A 250 000-gaussian character is 13 MB of GPU memory, fixed at allocation and never reallocated.
• A colour word of zero is invisible. Alpha 0 falls out of pack4x8unorm for a zeroed word, so a slot nobody has written renders as nothing. That is what makes unallocated capacity free, and what clearSlots relies on when a branch is retired.

All of this lives behind packages/splat/src/backendBuffers.ts, which is the only file in the repository allowed to touch renderer.backend (AGENTS.md hard rule 8). If a three upgrade moves the private surface, exactly one file fails, and it fails with a message naming the version it was written against.

Sorting

Splats are transparent and must be drawn back to front, so every frame's draw order is a depth sort of every gaussian. On WebGPU that is CountingSort: 4096 depth bins, four compute passes, and — this is the surprising part — a cost that is almost flat in the splat count.

Three decides whether to sort by comparing the view direction against the last one it sorted for:

needsSort = dot(sortDirection, lastSortDirection) < 0.9995   // about 1.81 degrees

So a camera that barely moves does not re-sort, and a static splat is usually free. A camera panning at 2°/frame re-sorts every frame, which is the worst case and the one budgeted below.

That threshold is also a trap for dynamic splats: if a producer moves the gaussians while the camera holds still, the order is stale and three has no way to know. markGaussiansChanged() is the fix — it clears the one internal flag that forces a dispatch — and a producer calls it once per frame after its compute pass. Rendering without it looks almost right, which is why the end-to-end suite asserts sortsPerFrame === 1 rather than trusting review.

Measured cost

Reference box, headless Chromium with WebGPU, 1280×720, camera turning 2°/frame so every frame re-sorts. sort is GPU time for the four CountingSort passes; render is GPU time for the draw; writeBuffer/frame counts every CPU→GPU upload the whole page makes.

Mode	Splats	Backend	Sort ms (p50)	Render ms (p50)	Producer ms (p50)	writeBuffer/frame
static	150 k	WebGPU	0.31	0.13	—	2 (160 B)
dynamic	100 k	WebGPU	0.31	0.09	0.004	2 (160 B)
dynamic	250 k	WebGPU	0.33	0.21	0.008	2 (160 B)
dynamic	500 k	WebGPU	0.37	0.42	0.013	2 (160 B)

Read that column again: the sort is 0.31 ms at 100 k and 0.37 ms at 500 k. The bin count dominates, not the splat count, so sorting is a fixed ~0.35 ms tax rather than a per-splat cost. The spike measured 0.42 ms for a 1 M-gaussian SPZ on the same box. What does scale is the draw, roughly linearly in splats.

The two writeBuffer calls per frame are three's own camera uniforms; they are present in the static case too. No gaussian data crosses the bus after allocation — that is the whole point of the dynamic path, and it is asserted, not assumed.

Dynamic splats: slots

One character is one AnimatedSplat, and every branch of it — head, body, eyes, hair — is a slot range inside that one buffer. That is not an optimisation: splats can only be sorted against others in the same object, so an avatar split across several GaussianSplats would have its own parts drawn in the wrong order against each other.

const sink = await createAnimatedSplat(renderer, {
  capacity: 250_000,
  boundingSphere: { center: [0, 1, 0], radius: 1.4 },
});
scene.add(sink.object3D);

const head = sink.allocate(120_000); // { offset: 0, count: 120000 }
const body = sink.allocate(90_000); //  { offset: 120000, count: 90000 }

Allocation is first fit, and frees coalesce with both neighbours, so a load/unload cycle does not fragment the buffer. Capacity is fixed for the object's lifetime: there is no growth path, because growing would mean reallocating four GPU buffers and rebuilding every bind group that points at them.

The bounding sphere is not cosmetic. The sort's depth range is derived from it, and a sphere that does not contain the gaussians crushes them into too few of the 4096 bins. Frustum culling is off in dynamic mode for the same reason it has to be declared rather than measured: a producer can move gaussians outside it between two frames and the CPU would never know. Keep it up to date with setBoundingSphere.

Draw order

Splats draw after opaque geometry, with depth test on and depth write off. The material is transparent, so three already puts it in the transparent pass; @aosengine/splat additionally sets renderOrder = 1000 (SPLAT_RENDER_ORDER) so splats come after ordinary transparent meshes.

This is a real constraint on level design, not a detail:

• A transparent mesh must not intersect a splat volume. Three sorts transparent objects by their object centre, so an intersecting pair has exactly one draw order for the whole overlap and one of them will be wrong somewhere.
• Two splat objects that overlap in space will interleave incorrectly, for the same reason. Each sorts its own gaussians perfectly; nothing sorts across objects. If two things must blend into each other, they belong in one AnimatedSplat, in different slot ranges.
• Opaque geometry is fine at any depth: it wrote the depth buffer first and the splats test against it.

The WebGL fallback

WebGPURenderer falls back to a WebGL backend when the browser has no WebGPU, and static splats still work there — three sorts on the CPU instead. It is slower, and it gets worse with size, because unlike the GPU sort this one really is linear:

Splats	CPU sort ms (p50)
75 k	0.9
150 k	1.7
250 k	3.0
1 M	11.6

At 1 M gaussians the sort alone is two thirds of a 16.6 ms frame. Budget the fallback at a few hundred thousand splats, and remember the sort only runs when the camera turns far enough — a player standing still pays nothing.

Dynamic splats do not run on the fallback at all. The CPU sort reads the source position attribute, and dynamic mode does not keep one, so updateSort throws SplatUnsupportedError rather than render an unsorted mess. Check engine.caps.characters (which tracks the WebGPU backend) before creating a character, and design a game that degrades: capsules instead of avatars, not a black screen.

The fork

packages/splat/src/three-fork/AnimatedGaussianSplat.js is three's GaussianSplat.js verbatim plus six marked insertions: a capacity constructor that skips the O(N) repack, markGaussiansChanged, owner-supplied bounds, no-op CPU-mirror readers, and the WebGL guard. Every deviation lives between // AOSENGINE EDIT <n> BEGIN and END, and packages/splat/scripts/diff-upstream.mjs strips those blocks and asserts what is left is byte-identical to the pinned upstream file. It runs as the package's pretest and as a unit test, so a three upgrade cannot slip through quietly. src/three-fork/UPSTREAM.md explains each edit, and each of the three planned edits that turned out to be unnecessary.

See also

• Characters
• Assets and the manifest
• Load a splat environment
• ADR 0005 — fork GaussianSplat
• packages/splat/README.md — @aosengine/splat