@aosengine/character

Classes

CharacterUnsupportedError

Thrown by createCharacter when the renderer is not on the WebGPU backend.

Characters are a WebGPU-only feature by construction: the rig deform, the lift and the decoders all run as compute on renderer.backend.device, and the WebGL fallback backend has no device to borrow. There is no CPU path — the POC had one and it cost seconds per frame, which reads as a hung avatar rather than as a slow one. Check engine.caps.characters and substitute a placeholder.

Example

import { CharacterUnsupportedError, createCharacter } from '@aosengine/character';

try {
  await createCharacter(bundle, { renderer, scene, sink });
} catch (err) {
  if (err instanceof CharacterUnsupportedError) console.warn(err.reason);
}

Extends

• Error

Constructors

Constructor

new CharacterUnsupportedError(reason): CharacterUnsupportedError;

Parameters

Parameter	Type
reason	string

Returns

CharacterUnsupportedError

Overrides

Error.constructor

Properties

name

readonly name: "CharacterUnsupportedError" = 'CharacterUnsupportedError';

Overrides

Error.name

reason

readonly reason: string;

Why the renderer could not host a character.

---

DebugVertexLift

One isotropic gaussian per rig vertex, in a sink's slot range.

Example

import { DebugVertexLift } from '@aosengine/character';

const lift = new DebugVertexLift({
  device,
  sink,
  vertsBuffer: backend.vertsBuffer,
  vertexCount: backend.vertexCount,
});
const encoder = device.createCommandEncoder();
backend.encode(encoder);
lift.encode(encoder);
device.queue.submit([encoder.finish()]);
sink.markGaussiansChanged();

Constructors

Constructor

new DebugVertexLift(options): DebugVertexLift;

Build the pipeline and reserve the slots.

Parameters

Parameter	Type	Description
options	DebugVertexLiftOptions	See DebugVertexLiftOptions.

Returns

DebugVertexLift

Throws

When the supplied range is smaller than the vertex count, which would silently draw part of a head.

Properties

range

readonly range: SlotRange;

The slots this preview owns. Fixed for its lifetime.

vertexCount

readonly vertexCount: number;

Vertices it draws.

Accessors

params

Get Signature

get params(): Readonly<DebugLiftParams>;

The params as they will next be uploaded. Read-only; use the setters.

Returns

Readonly<DebugLiftParams>

The current parameters.

Methods

dispose()

dispose(): void;

Release the GPU buffers and, when it allocated the range, the slots. Idempotent.

Returns

void

encode()

encode(encoder): void;

Record the pass. The rig's own pass must already be in this encoder, or an earlier submission: one device means one queue, so ordering alone is the synchronisation.

Parameters

Parameter	Type	Description
encoder	GPUCommandEncoder	The frame's encoder.

Returns

void

setShading()

setShading(shading): void;

Change the colour source.

Parameters

Parameter	Type	Description
shading	DebugShading	tint, normal or flat.

Returns

void

setSigma()

setSigma(sigma): void;

Change the gaussian radius.

Parameters

Parameter	Type	Description
sigma	number	Radius in object space, metres.

Returns

void

setTransform()

setTransform(transform, centroid?): void;

Change the rig -> object placement.

Parameters

Parameter	Type	Description
transform	readonly number[]	Row-major 3x4 affine, 12 numbers.
centroid?	readonly [number, number, number]	Object-space centre for normal shading. Left alone when omitted.

Returns

void

---

GnmRigBackend

GNM as a RigBackend.

Implements

• RigBackend

Constructors

Constructor

new GnmRigBackend(options?): GnmRigBackend;

Parameters

Parameter	Type
options	GnmRigBackendOptions

Returns

GnmRigBackend

Properties

kind

readonly kind: "gnm";

Which implementation this is.

Implementation of

RigBackend.kind

Accessors

assets

Get Signature

get assets(): AosRigPack;

The parsed pack — the topology and UVs a caller may want for a preview mesh.

Returns

AosRigPack

The pack parsed by init.

controlNames

Get Signature

get controlNames(): readonly string[];

The control space this backend drives, in setControls order.

Returns

readonly string[]

The control space this backend drives, in setControls order.

Implementation of

RigBackend.controlNames

vertexCount

Get Signature

get vertexCount(): number;

Vertices produced per pose.

Returns

number

Vertices produced per pose.

Implementation of

RigBackend.vertexCount

vertsAABB

Get Signature

get vertsAABB(): VertsAABB;

Bounds of the neutral pose, for the sink's bounding sphere.

Returns

VertsAABB

Bounds of the neutral pose, for the sink's bounding sphere.

Implementation of

RigBackend.vertsAABB

vertsBuffer

Get Signature

get vertsBuffer(): GPUBuffer;

The posed vertices, in the rig's own frame and units. Valid after the encoded pass has executed; the lift's copyBufferToBuffer is what reads it.

Returns

GPUBuffer

The posed vertices, in the rig's own frame and units. Valid after the encoded pass has executed; the lift's copyBufferToBuffer is what reads it.

Implementation of

RigBackend.vertsBuffer

Methods

bytes()

bytes(): number;

Approximate GPU + wasm bytes this backend holds, for memoryReport().

Returns

number

Implementation of

RigBackend.bytes

dispose()

dispose(): void;

Release the wasm heap and every GPU buffer. Idempotent.

Returns

void

Implementation of

RigBackend.dispose

encode()

encode(encoder): void;

Record the per-vertex pass into encoder. No submit, no fence, no readback.

Parameters

Parameter	Type
encoder	GPUCommandEncoder

Returns

void

Implementation of

RigBackend.encode

init()

init(options): Promise<void>;

Build the wasm/GPU resources. Idempotent; throws loudly on a bad asset.

Parameters

Parameter	Type
options	RigBackendInit

Returns

Promise<void>

Implementation of

RigBackend.init

runCpu()

runCpu(controls): Promise<Float32Array<ArrayBufferLike>>;

The CPU reference, for calibration. Never per frame — see gnmReference.ts.

Parameters

Parameter	Type	Description
controls	Float32Array	A head_ext vector; a short one is zero-padded and a long one truncated, since this runs off the calibration path rather than the hot one.

Returns

Promise<Float32Array<ArrayBufferLike>>

The posed vertices as (V,3) in METRES, skinned against the joint worlds currently set.

Implementation of

RigBackend.runCpu

setControls()

setControls(controls): void;

Set the whole head_ext vector.

A shorter vector is zero-padded, which is exactly what the reduced ML view (68 floats) means — block.expand pads per region, so a caller holding the reduced view must widen it first rather than passing it here.

Parameters

Parameter	Type	Description
controls	Float32Array	Up to head_ext.dim floats: the expression coefficients, then the four gaze angles in radians. A longer vector throws.

Returns

void

Implementation of

RigBackend.setControls

setJointOverrides()

setJointOverrides(overrides): void;

Per-joint rotation overrides, by name — the procedural head-aim path.

Applied as a parent-relative rotation on top of the joint's rest, then propagated down the pack's own parent chain, so aiming head carries the eyes with it exactly as skinning would.

Parameters

Parameter	Type	Description
overrides	readonly JointOverride[]	Joint name plus a (w, x, y, z) rotation. A name this pack does not carry is skipped silently — the body rig has joints the head does not. Each call rebuilds from the rest pose, so overrides do not accumulate.

Returns

void

Implementation of

RigBackend.setJointOverrides

setJointWorlds()

setJointWorlds(worlds): void;

Where the body rig's joints are this frame, J*16 row-major world matrices in the pack's own compact joint order.

Defaults to the pack's rest, which makes the skinning the identity — a head posed at bind. The animation layer supplies the real matrices; the head does not solve them, because neck and head rotation are the BODY's joints.

Parameters

Parameter	Type	Description
worlds	Float32Array	J*16 row-major world matrices, in METRES, in the pack's compact joint order. The skin rows are recomputed here, not per frame.

Returns

void

---

OrlRigBackend

OpenRigLogic as a RigBackend, in skinning or rigid-shell mode.

Implements

• RigBackend

Constructors

Constructor

new OrlRigBackend(options): OrlRigBackend;

Parameters

Parameter	Type
options	OrlRigBackendOptions

Returns

OrlRigBackend

Properties

kind

readonly kind: "orl";

Which implementation this is.

Implementation of

RigBackend.kind

Accessors

controlNames

Get Signature

get controlNames(): readonly string[];

The control space this backend drives, in setControls order.

Returns

readonly string[]

The control space this backend drives, in setControls order.

Implementation of

RigBackend.controlNames

isDriving

Get Signature

get isDriving(): boolean;

True once a shell-mode backend has adopted a frame and bound a shell.

Returns

boolean

True in skin mode always, and in shell mode once adoptRigSpace has matched at least one shell to a joint.

vertexCount

Get Signature

get vertexCount(): number;

Vertices produced per pose.

Returns

number

Vertices produced per pose.

Implementation of

RigBackend.vertexCount

vertsAABB

Get Signature

get vertsAABB(): VertsAABB;

Bounds of the neutral pose, for the sink's bounding sphere.

Returns

VertsAABB

Bounds of the neutral pose, for the sink's bounding sphere.

Implementation of

RigBackend.vertsAABB

vertsBuffer

Get Signature

get vertsBuffer(): GPUBuffer;

The posed vertices, in the rig's own frame and units. Valid after the encoded pass has executed; the lift's copyBufferToBuffer is what reads it.

Returns

GPUBuffer

The posed vertices, in the rig's own frame and units. Valid after the encoded pass has executed; the lift's copyBufferToBuffer is what reads it.

Implementation of

RigBackend.vertsBuffer

Methods

adoptRigSpace()

adoptRigSpace(lin, offset): boolean;

Place this branch's shells in RIG space, using the frame a sibling branch already fitted (the head's calibration).

This branch cannot fit that frame itself: its mesh is not in the DNA, so there is no correspondence to fit against. The head's frame is the right one to borrow because both branches are baked in a single bundle space — measured, the borrowed frame lands these shells 0.11-0.35 cm from the DNA's own eye and teeth meshes.

MUST run before calibration. Returns false when the branch stays neutral.

Parameters

Parameter	Type	Description
lin	Float32Array	The sibling's fitted rig -> bundle linear part, nine floats in the row-vector convention v·lin + offset. Inverted here to map this branch's neutral back into rig space.
offset	Float32Array	The matching translation, three floats.

Returns

boolean

True once at least one shell is bound to a joint. False — and the branch stays at its neutral pose — in skin mode, before init, when the pack carries no joint names, when the sibling's transform is singular, or when no shell centroid landed near an eye/teeth joint.

bytes()

bytes(): number;

Approximate GPU + wasm bytes this backend holds, for memoryReport().

Returns

number

Implementation of

RigBackend.bytes

dispose()

dispose(): void;

Release the wasm heap and every GPU buffer. Idempotent.

Returns

void

Implementation of

RigBackend.dispose

encode()

encode(encoder): void;

Record the per-vertex pass into encoder. No submit, no fence, no readback.

Parameters

Parameter	Type
encoder	GPUCommandEncoder

Returns

void

Implementation of

RigBackend.encode

init()

init(options): Promise<void>;

Build the wasm/GPU resources. Idempotent; throws loudly on a bad asset.

Parameters

Parameter	Type
options	RigBackendInit

Returns

Promise<void>

Implementation of

RigBackend.init

runCpu()

runCpu(controls): Promise<Float32Array<ArrayBufferLike>>;

Calibration path. Deliberately uses the CPU deform even when the GPU one is live: corr = neutral - runCpu(baseRig) is added to every GPU-produced frame, so the two must agree.

Parameters

Parameter	Type	Description
controls	Float32Array	One value per name in controlNames.

Returns

Promise<Float32Array<ArrayBufferLike>>

A fresh array of posed vertices as xyz triples, in CENTIMETRES — the rig's own mesh in skin mode, this branch's rigid-shell pose in shell mode. Never the reused scratch, since the caller keeps it.

Implementation of

RigBackend.runCpu

setControls()

setControls(controls): void;

Solve at controls. CPU-only and synchronous — there is no inference call to await — so a caller may call it several times a frame and only encode once.

Parameters

Parameter	Type
controls	Float32Array

Returns

void

Implementation of

RigBackend.setControls

setJointOverrides()

setJointOverrides(overrides): void;

No addressable joint namespace yet — head aim rides the body rig, not the DNA.

Parameters

Parameter	Type	Description
overrides	readonly JointOverride[]	Ignored. A non-empty list logs once, so a caller wondering why its aim does nothing is told rather than left guessing.

Returns

void

Implementation of

RigBackend.setJointOverrides

---

SlotAllocator

A deterministic first-fit allocator over [0, capacity).

Example

import { SlotAllocator } from '@aosengine/character';

const slots = new SlotAllocator(262144);
const head = slots.allocate(65536); // { offset: 0, count: 65536 }
slots.free(head);

Constructors

Constructor

new SlotAllocator(capacity): SlotAllocator;

Parameters

Parameter	Type
capacity	number

Returns

SlotAllocator

Properties

capacity

readonly capacity: number;

Accessors

available

Get Signature

get available(): number;

Slots not currently owned by any range.

Returns

number

The total free slot count, which may be spread over several blocks and so is an upper bound on the largest allocation that can still succeed.

Methods

allocate()

allocate(count): SlotRange;

Reserve count contiguous slots.

Throws rather than returning null: a branch that cannot be placed renders nothing at all, and a silent null would surface as a character missing one region with no error anywhere.

Parameters

Parameter	Type	Description
count	number	How many splat slots the branch needs; a positive integer.

Returns

SlotRange

The reserved range: offset is the branch's slot offset into the sink's storage buffers and count echoes the request.

blocks()

blocks(): SlotRange[];

The free list, for tests and memoryReport(). Copied, never the live array.

Returns

SlotRange[]

The free blocks in ascending offset order, each a fresh object, so a caller cannot corrupt the allocator by mutating what it reads.

free()

free(range): void;

Release a range and coalesce with its neighbours.

Overlap with an already-free range throws: it means two owners believe they hold the same slots, which renders one branch's gaussians at another's pose.

Parameters

Parameter	Type	Description
range	SlotRange	The range to give back, exactly as SlotAllocator.allocate returned it; it must lie inside the capacity and must not already be free.

Returns

void

Interfaces

AosRigHeader

The header JSON, parsed.

Properties

bindTransform

bindTransform: number[];

Row-major 4x4: head-local -> the body's bind space. See the header comment.

buffers

buffers: AosRigBuffer[];

coeffCount

coeffCount: number;

eyes

eyes: object;

names

names: string[];

headExt

headExt: HeadExtLayout;

joints

joints: AosRigJoint[];

maxInfluence

maxInfluence: number;

model

model: "gnm";

source?

optional source?: Record<string, unknown>;

Free-form provenance the packer writes; never read by the runtime.

units

units: "m" | "cm";

version

version: number;

vertexCount

vertexCount: number;

---

AosRigPack

A parsed pack: the header plus typed views over the blobs.

Properties

basis

basis: Uint32Array;

Packed fp16 pairs, vertex-major (V,E,3). Feed to the shader verbatim.

basisScale

basisScale: Float32Array;

(E,) f32 per-coefficient scale.

bindTransform

bindTransform: Float32Array;

Row-major 4x4 head-local -> body bind space. Identity when the bake had none.

byteLength

byteLength: number;

Total bytes of the pack, for memoryReport().

coeffCount

coeffCount: number;

eyePositions

eyePositions: Float32Array;

(2,3) f32.

eyeWeights

eyeWeights: Float32Array;

(2,V) f32.

faces

faces: Uint32Array;

header

header: AosRigHeader;

jointParents

jointParents: Int32Array;

(J,) i32 parents, -1 for a root.

maxInfluence

maxInfluence: number;

neutral

neutral: Float32Array;

(V,3) f32, metres.

quads?

optional quads?: Uint32Array<ArrayBufferLike>;

restWorld

restWorld: Float32Array;

(J,16) f32 row-major rest world matrices.

skinIndex

skinIndex: Uint16Array;

(V,4) u16 into header.joints.

skinWeight

skinWeight: Float32Array;

(V,4), expanded from the packed f16 lanes at load.

stitchLocal?

optional stitchLocal?: Float32Array<ArrayBufferLike>;

(V,3) f32 neck-seam displacement, head-local. Absent when the bake had no seam.

uv?

optional uv?: Float32Array<ArrayBufferLike>;

vertexCount

vertexCount: number;

---

ArkitGnmTerm

One coefficient an ARKit channel drives.

Properties

gain

gain: number;

Coefficient units per unit of ARKit weight. Signed.

index

index: number;

Coefficient index WITHIN that region.

region

region: string;

head_ext region: left_eye, right_eye, lower_face, tongue or pupils.

---

ArkitToGnmOptions

Options for createArkitToGnmMap.

Properties

clamp?

optional clamp?: number;

Clamp on every produced coefficient. Default 4.

gaze?

optional gaze?: boolean;

Also fill the four gaze angles from the EyeLook* channels. Default true.

The gaze half of head_ext is angles in radians, not coefficients, so it is not in the table: EyeLookUp/Down/In/Out are converted here at GAZE_FULL_SCALE radians per unit weight.

table?

optional table?: Readonly<Record<string, readonly ArkitGnmTerm[]>>;

Replace the default table.

---

BodyPose

A body pose the animation layer hands over.

Properties

bones?

optional bones?: readonly JointOverride[];

Per-joint parent-relative rotations, by the rig's own joint names.

rootPos?

optional rootPos?: readonly [number, number, number];

Root translation in scene metres. Moves the character's object3D.

---

Character

A live splat character.

Properties

branches

readonly branches: readonly string[];

The branches that came alive, in manifest order.

expressionSpace

readonly expressionSpace: ExpressionSpace;

What the animation layer is supposed to send.

object3D

readonly object3D: Object3D;

The scene node the character hangs under.

rigKind

readonly rigKind: string;

Which rig posed the face.

Methods

dispose()

dispose(): void;

Release every GPU buffer, ORT session, wasm heap and slot range. Idempotent.

Returns

void

memoryReport()

memoryReport(): MemoryReport;

Where this character's memory went.

Returns

MemoryReport

setBodyPose()

setBodyPose(pose): void;

Pose the body: per-joint rotations for the rig, plus a root translation.

Parameters

Parameter	Type
pose	BodyPose | null

Returns

void

setExpression()

setExpression(weights): void;

Drive the face in the bundle's declared expression_space.

ARKit-52 for an arkit52 bundle, head_ext (387) or the reduced view (68) for a GNM one. An ARKit vector reaching a GNM bundle with no arkit_map is REFUSED, loudly: the two spaces are unrelated, and a silent reinterpretation is a face that moves wrongly with nothing to see.

Parameters

Parameter	Type
weights	Float32Array

Returns

void

setLookAt()

setLookAt(target): void;

Look at a world point, or null to return to rest.

Parameters

Parameter	Type
target	readonly [number, number, number] | null

Returns

void

setRig()

setRig(controls): void;

Drive the rig directly, in the bundle's OWN control space.

This is the raw vector the decoders were trained on — rig_names.json order for an ORL bundle, head_ext for a GNM one. Prefer setExpression unless you are authoring against a specific character.

Parameters

Parameter	Type
controls	Float32Array

Returns

void

settled()

settled(): Promise<void>;

Resolves when every in-flight pass has drained. For tests and screenshots.

Returns

Promise<void>

update()

update(dt, camera): void;

Advance one frame.

Cheap and allocation-free when nothing changed: it compares the camera against the last decoded view and re-runs the appearance pass only when it has genuinely moved. The epsilon is 2 mm plus a turn/fov term, because a pass is an appr decode plus a lift.

Parameters

Parameter	Type
dt	number
camera	PerspectiveCamera

Returns

void

---

CharacterBundle

A loaded, ready-to-instantiate character.

Properties

byteLength

byteLength: number;

Sum of every resident file's bytes.

bytes

bytes: BundleBytes;

Every resident file, by bundle-relative name.

manifest

manifest: CharacterManifest;

Manifest + the engine's two added blocks.

preferFp16

preferFp16: boolean;

True when at least one decoder ships an fp16 sibling.

resolver

resolver: AssetResolver;

How a lazily-needed file is fetched.

scene

scene: MultiRegionScene;

Per-branch mesh topology, in manifest order.

---

CharacterManifest

The engine-side manifest: the exporter's scene plus the two blocks above.

Properties

expressionSpace

expressionSpace: ExpressionSpace;

rig

rig: RigManifest;

rigNames

rigNames: string[] | null;

rig_names.json, when the bundle ships one.

scene

scene: SceneManifest;

---

CharacterOptions

Tuning a game may pass at creation.

Properties

log?

optional log?: (message) => void;

Log sink; defaults to console.log.

Parameters

Parameter	Type
message	string

Returns

void

poseAnchors?

optional poseAnchors?: boolean;

Apply the bundle's baked per-pose anchors when it ships them. DEFAULT OFF.

They correct where the head IS — the training export fitted a rigid transform per FRAME, and a face control set has no rigid head controls, so head placement is not a function of the rig at all. The correction is real (the single row-0 anchor is out by a p50 of 9.1 mm at the other trained poses) but the blend moves the head whenever the nearest trained poses change, and during speech that happens on a BLINK: measured, a blink steps the head 5.10 mm in ONE frame. A ~4.6 mm mean correction does not pay for a head pop synchronised with blinking, so this is opt-in until the neighbour search can ignore the lids.

preferFp16?

optional preferFp16?: boolean;

Prefer the bundle's fp16-internal decoders. Default true.

---

CreateCharacterOptions

What createCharacter takes.

Properties

options?

optional options?: CharacterOptions;

renderer

renderer: RendererLike;

scene

scene: Object3D;

The scene the character's root is added to.

sink

sink: SplatSink;

The splat object this character writes its gaussians into.

---

CreateRigPreviewOptions

What createRigPreview takes.

Properties

backend

backend: RigBackend;

The backend to preview. Constructed, not necessarily initialised.

controlNames?

optional controlNames?: string[];

Control names for backend.init. Defaults to none, which GNM does not need.

device?

optional device?: GPUDevice;

The device, when the caller already has it from prepareLiftDevice.

fetchBytes?

optional fetchBytes?: (name) => Promise<Uint8Array<ArrayBufferLike>>;

Lazily fetch a file the eager pass skipped, handed to backend.init.

Parameters

Parameter	Type
name	string

Returns

Promise<Uint8Array<ArrayBufferLike>>

getBytes?

optional getBytes?: (name) => Uint8Array<ArrayBufferLike> | undefined;

Bundle-relative file bytes. Supply them and the preview calls backend.init itself; omit them and the backend is assumed to be initialised already.

Parameters

Parameter	Type
name	string

Returns

Uint8Array<ArrayBufferLike> | undefined

height?

optional height?: number;

Object-space height the rig's bounds are scaled to. Defaults to 0.35 m.

offset?

optional offset?: readonly [number, number, number];

Object-space position of the fitted centre. Defaults to the origin.

range?

optional range?: SlotRange;

Slots to draw into. Defaults to a fresh allocation of vertexCount.

renderer?

optional renderer?: RendererLike;

The engine's renderer. Either this or device; the renderer owns the device.

setBounds?

optional setBounds?: boolean;

Set the sink's bounding sphere from the fitted bounds. Defaults to true.

shading?

optional shading?: DebugShading;

Colour source. Defaults to tint when a tint is supplied, normal otherwise.

sigma?

optional sigma?: number;

Gaussian radius in object space, metres. Defaults to 2 mm.

sink

sink: SplatSink;

Where the gaussians go — an AnimatedSplat from @aosengine/splat.

tint?

optional tint?: Uint32Array<ArrayBufferLike>;

One packed RGBA per vertex; see vertexTint.ts.

transform?

optional transform?: readonly number[];

An explicit rig -> object affine (row-major 3x4), instead of the fit.

---

DebugLiftParams

Everything the params uniform carries.

Properties

centroid

centroid: readonly [number, number, number];

Object-space centre the normal shading points away from.

opacity

opacity: number;

Alpha written into every gaussian, [0, 1].

shading

shading: DebugShading;

Colour source.

sigma

sigma: number;

Gaussian radius in object space, metres.

slotCount

slotCount: number;

Slots the range owns.

slotOffset

slotOffset: number;

First slot of the range this preview owns.

transform

transform: readonly number[];

Rig -> sink-object affine, row-major 3x4 (12 numbers).

vertexCount

vertexCount: number;

Vertices in the rig buffer. Slots past this are cleared, not skipped.

---

DebugVertexLiftOptions

What DebugVertexLift needs at construction.

Properties

centroid?

optional centroid?: readonly [number, number, number];

Object-space centre for normal shading. Defaults to the origin.

device

device: GPUDevice;

The renderer's own device — the one the rig backend built its buffers on.

label?

optional label?: string;

Label prefix for the GPU objects, so a capture is readable.

opacity?

optional opacity?: number;

Alpha of every gaussian. Defaults to 1.

range?

optional range?: SlotRange;

A range to write into. Omit and the lift allocates vertexCount slots from the sink and frees them on dispose.

shading?

optional shading?: DebugShading;

Colour source. Defaults to tint.

sigma?

optional sigma?: number;

Gaussian radius in object space, metres. Defaults to 2 mm.

sink

sink: SplatSink;

Where the gaussians go.

tint?

optional tint?: Uint32Array<ArrayBufferLike>;

One packed RGBA per vertex, from vertexTint.ts. Defaults to a flat neutral.

transform?

optional transform?: readonly number[];

Rig -> object affine, row-major 3x4. Defaults to the identity.

vertexCount

vertexCount: number;

How many vertices that buffer holds.

vertsBuffer

vertsBuffer: GPUBuffer;

The rig's posed vertices, V * 3 f32 in the rig's own units.

---

ExpressionSpace

The expression_space block.

Properties

arkitMap?

optional arkitMap?: number[];

For a gnm space driven from ARKit: slot index per ARKit channel, or -1. Absent until the ARKit -> GNM map is trained; a caller sending ARKit to a GNM bundle without it gets a loud refusal rather than a face that moves wrongly.

dim

dim: number;

Vector width the animation layer sends.

kind

kind: ExpressionKind;

names

names: string[];

One name per slot. Empty when the space is implied by kind alone.

segments

segments: ExpressionSegment[];

Named runs — ARKit has none; GNM has left_eye / right_eye / lower_face / tongue / pupils / gaze.

---

HeadExtLayout

The head_ext parameter block.

Properties

dim

dim: number;

387 = expression + gaze.

exprDim

exprDim: number;

gazeDim

gazeDim: number;

4: [pitch_L, yaw_L, pitch_R, yaw_R] radians.

reduced

reduced: Record<string, number>;

The reduced ML view's per-region counts; they sum to 64, plus gaze = 68.

regions

regions: [string, number][];

[["left_eye",100], ["right_eye",100], ["lower_face",150], ["tongue",32], ["pupils",1]].

---

JointOverride

A per-joint override the animation layer applies on top of the solved rig.

Properties

joint

joint: string;

Joint name in the backend's own namespace.

rotation

rotation: readonly [number, number, number, number];

Parent-relative rotation as a quaternion (w, x, y, z).

---

LoadCharacterBundleOptions

Options for loadCharacterBundle.

Extends

• BundleFetchOptions

Properties

fetch?

optional fetch?: {
  (input, init?): Promise<Response>;
  (input, init?): Promise<Response>;
};

Overrides the global fetch; ignored when resolver is given.

Call Signature

(input, init?): Promise<Response>;

MDN Reference

Parameters

Parameter	Type
input	URL | RequestInfo
init?	RequestInit

Returns

Promise<Response>

Call Signature

(input, init?): Promise<Response>;

MDN Reference

Parameters

Parameter	Type
input	string | URL | Request
init?	RequestInit

Returns

Promise<Response>

Inherited from

BundleFetchOptions.fetch

preferFp16?

optional preferFp16?: boolean;

Skip the fp16 decoders even where the bundle ships them.

resolver?

optional resolver?: AssetResolver;

Resolve a bundle-relative name to bytes yourself (CDN, asset manager, tests).

Inherited from

BundleFetchOptions.resolver

signal?

optional signal?: AbortSignal;

Inherited from

BundleFetchOptions.signal

---

MemoryReport

Where a character's memory went.

Properties

branches

branches: object[];

Per branch: name, slots, GPU bytes.

gpuBytes

gpuBytes: number;

name

name: string;

slots

slots: number;

bundleBytes

bundleBytes: number;

Resident bundle bytes (decoders, meshes, rig pack).

gpuBytes

gpuBytes: number;

Approximate GPU bytes across every branch's lift and rig.

slots

slots: number;

Splat slots this character owns.

---

OrtDeviceAttachment

What attachOrtDevice managed to do.

Properties

gpuBufferIo

readonly gpuBufferIo: boolean;

When false, every decoder session must be built with downloadOutputs so its outputs come back as CPU arrays. A cross-device GPUBuffer is a validation error, not a slow path.

reason

readonly reason: string | null;

Why not, when shared is false. Null on success.

shared

readonly shared: boolean;

True when ORT accepted the renderer's device.

---

RendererLike

The slice of WebGPURenderer this package reads.

Properties

backend?

optional backend?: object;

---

RigBackend

One rig -> vertices implementation.

Lifecycle: init -> (setControls -> encode)* -> dispose. setControls is pure CPU work (the solve); encode records the per-vertex half into a caller-supplied encoder so a whole character is one submission.

Properties

controlNames

readonly controlNames: readonly string[];

The control space this backend drives, in setControls order.

kind

readonly kind: RigKind;

Which implementation this is.

vertexCount

readonly vertexCount: number;

Vertices produced per pose.

vertsAABB

readonly vertsAABB: VertsAABB;

Bounds of the neutral pose, for the sink's bounding sphere.

vertsBuffer

readonly vertsBuffer: GPUBuffer;

The posed vertices, in the rig's own frame and units. Valid after the encoded pass has executed; the lift's copyBufferToBuffer is what reads it.

Methods

bytes()

bytes(): number;

Approximate GPU + wasm bytes this backend holds, for memoryReport().

Returns

number

dispose()

dispose(): void;

Release the wasm heap and every GPU buffer. Idempotent.

Returns

void

encode()

encode(encoder): void;

Record the per-vertex pass into encoder. No submit, no fence, no readback.

Parameters

Parameter	Type
encoder	GPUCommandEncoder

Returns

void

init()

init(options): Promise<void>;

Build the wasm/GPU resources. Idempotent; throws loudly on a bad asset.

Parameters

Parameter	Type
options	RigBackendInit

Returns

Promise<void>

runCpu()

runCpu(controls): Promise<Float32Array<ArrayBufferLike>>;

Solve and deform ON THE CPU, for calibration only.

The rig->bundle similarity and the per-vertex corr are fitted from this, and corr is then added to every GPU-produced frame — so the two implementations must agree. Never call it per frame.

Parameters

Parameter	Type
controls	Float32Array

Returns

Promise<Float32Array<ArrayBufferLike>>

setControls()

setControls(controls): void;

Solve at controls. CPU-only and synchronous — there is no inference call to await — so a caller may call it several times a frame and only encode once.

Parameters

Parameter	Type
controls	Float32Array

Returns

void

setJointOverrides()?

optional setJointOverrides(overrides): void;

Apply per-joint rotation overrides on top of the solved pose (procedural head aim, gaze). Optional: a backend with no addressable joints simply omits it.

Parameters

Parameter	Type
overrides	readonly JointOverride[]

Returns

void

---

RigBackendInit

Everything a backend needs before it can pose anything.

Properties

controlNames

controlNames: string[];

The bundle's control-name list, in the order setControls takes values.

device

device: GPUDevice;

The device the lift owns. A backend must build every buffer on THIS device.

expectVerts?

optional expectVerts?: number;

Expected vertex count; a mismatch throws rather than posing garbage.

fetchBytes?

optional fetchBytes?: (name) => Promise<Uint8Array<ArrayBufferLike>>;

Lazily fetch a bundle file the eager pass skipped.

Parameters

Parameter	Type
name	string

Returns

Promise<Uint8Array<ArrayBufferLike>>

getBytes

getBytes: (name) => Uint8Array<ArrayBufferLike> | undefined;

Bundle-relative file bytes, already resident.

Parameters

Parameter	Type
name	string

Returns

Uint8Array<ArrayBufferLike> | undefined

---

RigManifest

The rig block.

Properties

backend

backend: RigBackendKind;

orl (OpenRigLogic from the character's own DNA) or gnm (a baked .aosrig).

controlNames

controlNames: string[];

The control-name list, in the order setRig takes values.

pack

pack: string | null;

Bundle-relative pack filename: orl_pack.bin or <name>.aosrig.

vertexCount

vertexCount: number;

Expected vertex count of the rig's output; 0 when the bundle does not state one.

---

RigPreview

A live rig preview.

Properties

backend

readonly backend: RigBackend;

The backend, initialised.

lift

readonly lift: DebugVertexLift;

The debug lift, for setSigma / setShading / setTransform.

range

readonly range: SlotRange;

The slots the preview owns.

scale

readonly scale: number;

Uniform scale the placement fit chose, for an overlay to report.

Methods

dispose()

dispose(): void;

Release the lift's buffers, the backend's resources and the slot range.

Returns

void

encode()

encode(encoder): void;

Record the rig pass and the debug lift into one encoder, in that order.

Use this when the caller owns the submission — a frame that also wants timestamp queries around the two passes, for instance.

Parameters

Parameter	Type	Description
encoder	GPUCommandEncoder	The frame's command encoder.

Returns

void

render()

render(): void;

Encode, submit and ask the sink to re-sort. The one-call-per-frame path.

Returns

void

setControls()

setControls(controls): void;

Solve the rig at controls — the backend's own control space, so head_ext for GNM.

Parameters

Parameter	Type	Description
controls	Float32Array	The control vector.

Returns

void

---

SlotRange

A contiguous run of splat slots owned by one branch for its whole life.

Properties

count

readonly count: number;

Slot count. Constant: culled texels write opacity 0 rather than compacting.

offset

readonly offset: number;

First slot index.

---

SplatSink

The splat object a character writes its gaussians into.

Implemented by @aosengine/splat's AnimatedGaussianSplat. Slot ranges are allocated once per branch at load and never move: three's CountingSort keeps an index -> splat map across frames, so compacting on the GPU would tear the sort during camera motion.

Properties

buffers

readonly buffers: SplatSinkBuffers;

capacity

readonly capacity: number;

Total slots. A character must fit inside it or allocate throws.

object3D

readonly object3D: Object3D<Object3DEventMap>;

Scene node the splats hang under.

Methods

allocate()

allocate(count): SlotRange;

Reserve count contiguous slots. Throws when the sink is full.

Parameters

Parameter	Type
count	number

Returns

SlotRange

free()

free(range): void;

Return a range to the free list.

Parameters

Parameter	Type
range	SlotRange

Returns

void

markGaussiansChanged()

markGaussiansChanged(): void;

Force a re-sort on the next render, even when the camera has not moved.

Returns

void

setBoundingSphere()

setBoundingSphere(center, radius): void;

Owner-supplied bounds; the splat object never recomputes them from the GPU.

Parameters

Parameter	Type
center	[number, number, number]
radius	number

Returns

void

---

SplatSinkBuffers

The GPU buffers a lift writes into. All four are indexed by absolute slot.

Properties

center

readonly center: GPUBuffer;

array<vec4<f32>>: xyz = centre in scene metres, w unused.

color

readonly color: GPUBuffer;

array<u32>: pack4x8unorm(vec4f(r, g, b, opacity)).

covarianceA

readonly covarianceA: GPUBuffer;

array<vec4<f32>>: (c00, c01, c02, c11).

covarianceB

readonly covarianceB: GPUBuffer;

array<vec4<f32>>: (c12, c22, 0, 0).

---

VertsAABB

Axis-aligned bounds of the posed vertices, in the rig's own units.

Properties

max

max: readonly [number, number, number];

min

min: readonly [number, number, number];

Type Aliases

ArkitGnmTable

type ArkitGnmTable = Readonly<Record<string, readonly ArkitGnmTerm[]>>;

The table: ARKit-52 channel name -> the coefficients it drives.

---

DebugShading

type DebugShading = "tint" | "normal" | "flat";

How the shader colours a vertex.

---

ExpressionKind

type ExpressionKind = "arkit52" | "gnm" | "gnm68";

How a caller addresses this character's face.

---

RigBackendKind

type RigBackendKind = "orl" | "gnm" | "none";

Which rig backend drives this bundle's branches.

---

RigKind

type RigKind = "orl" | "gnm";

Which implementation is behind a backend, for logs and the manifest.

---

TintKind

type TintKind = "joint" | "uv" | "height" | "flat";

Which per-vertex colouring a preview asks for.

Variables

ARKIT_NAMES

const ARKIT_NAMES: string[];

ARKit blendshape names in the exact order the server sends them (indices 0-51). Source: AvatarosWhisper/Private/AnimationFrameLiveLinkSource.cpp

Lives in its own module so pure-logic consumers can read the canonical channel order without pulling in anything else.

Ported from aos-threejs-poc/src/lib/arkitNames.js @ cdd63b10

---

ARKIT_TO_GNM_DEFAULT

const ARKIT_TO_GNM_DEFAULT: ArkitGnmTable;

The default ARKit-52 -> GNM table.

Covers the channels a face clip actually moves in a shipped idle: blink, squint, wide, the brows, the jaw, the smile/frown pair, the lateral mouth shift and the tongue. Channels with no entry contribute nothing rather than being approximated by a neighbouring mode, because a wrong mode is worse than a still one.

Example

import { ARKIT_TO_GNM_DEFAULT } from '@aosengine/character';

console.log(ARKIT_TO_GNM_DEFAULT.JawOpen[0].region); // 'lower_face'

---

DEBUG_LIFT_PARAMS_BYTES

const DEBUG_LIFT_PARAMS_BYTES: 96 = 96;

Bytes of the params uniform: four u32, three vec4 rows, four f32, one vec4.

---

DEBUG_LIFT_WORKGROUP

const DEBUG_LIFT_WORKGROUP: 64 = 64;

Threads per workgroup in debug_vertex_lift.wgsl. Mirrors the shader's literal.

---

GAZE_FULL_SCALE

const GAZE_FULL_SCALE: number;

Radians of gaze per unit ARKit EyeLook* weight — 20 degrees at full deflection.

Example

import { GAZE_FULL_SCALE } from '@aosengine/character';

console.log((GAZE_FULL_SCALE * 180) / Math.PI); // 20

---

GNM_PACK_FILE

const GNM_PACK_FILE: "gnm_head.aosrig" = 'gnm_head.aosrig';

Default pack name when a manifest declares none.

---

IDENTITY_3X4

const IDENTITY_3X4: readonly number[];

The identity 3x4, row-major: a rig whose vertices are already in object space.

---

MH_LEN

const MH_LEN: 188 = 188;

The MetaHuman control-vector width arkitToMh writes into.

---

MH_RIG_NAMES

const MH_RIG_NAMES: string[];

The 188 MetaHuman board controls, in arkitToMh index order.

---

ORL_HEAD_VERTS

const ORL_HEAD_VERTS: 24049 = 24049;

MetaHuman head_lod0 vertex count — what an ORL pack bakes. A branch mesh of any other size is not a head_lod0 and cannot be skinned by the DNA.

---

PACKAGE

const PACKAGE: "@aosengine/character";

Package identity marker for @aosengine/character.

Example

import { PACKAGE } from '@aosengine/character';

console.log(PACKAGE); // '@aosengine/character'

---

REQUIRED_STORAGE_BUFFERS

const REQUIRED_STORAGE_BUFFERS: 8 = 8;

WebGPU's DEFAULT maxStorageBuffersPerShaderStage, which every adapter grants. The lift's heaviest pass (lift_pass1.wgsl) binds exactly 8, so no device is refused the lift over a limit. It bound 9 until valid was folded into triim (inference/liftTriim.ts), and the integrated GPUs that report exactly 8 were silently landing on a seconds-per-frame CPU path.

Functions

arkitToMh()

function arkitToMh(arkit, out): Float32Array;

Map 52 ARKit blendshape weights onto the 188-element MetaHuman control array.

Parameters

Parameter	Type	Description
arkit	ArrayLike<number>	Live ARKit weights, length >= 52.
out	Float32Array	Destination, length MH_LEN. ZEROED first — many indices are never written below and would otherwise carry over from the previous frame.

Returns

Float32Array

out, filled with the 188 MetaHuman control weights.

---

arkitToRig()

function arkitToRig(
   arkit52, 
   gather, 
   outN, 
   scratch188?, 
   rest?, 
   range?
): Float32Array;

Map a live 52-element ARKit frame to a character's N-element rig vector.

ABOUT rest. ARKit-52 can only express a SUBSET of a MetaHuman board, and arkitToMh ZEROES every control it does not write — so without a rest vector the output is "the few controls ARKit reached, and 0 everywhere else". That is only a neutral face if the control space is centred on 0. For one declaring rig_range [0,1] it is an extreme corner the decoders never saw: measured on eyeline V10-C, ARKit reaches 53 of 168 controls (115 are permanently 0) against a trained mean with 133 non-zero, and the resulting vector sits FURTHER from the training distribution (L2 3.32) than a real trained pose does (2.96). The decoders then extrapolate, which is what a washed-out, detail-free face is.

ARKit weights are deltas from neutral BY DEFINITION (0 = neutral, 1 = full expression), and arkitToMh maps them into an MH space with the same convention — arkitToMh(zeros) is all-zeros, verified. So ARKit's contribution is ADDED to rest rather than replacing it: an ARKit-neutral frame reproduces rest exactly, and the 115 controls ARKit cannot reach hold their rest value instead of collapsing to 0.

Passing no rest reproduces the previous behaviour exactly, which is correct for a bundle whose controls do rest at zero.

Parameters

Parameter	Type	Default value	Description
arkit52	ArrayLike<number>	undefined	Live ARKit weights (length >= 52).
gather	readonly number[]	undefined	N indices into the 188 MH controls (gatherFromRigNames).
outN	Float32Array	undefined	Destination, length N === gather.length (overwritten).
scratch188?	Float32Array<ArrayBufferLike>	undefined	Optional reusable 188-length buffer, to avoid a per-call allocation on the hot path.
rest?	ArrayLike<number> | null	null	The bundle's rest configuration, length N. Ignored when its length does not match the gather.
range?	RigRange | null	null	Either the bundle's declared rig_range as one [lo,hi] pair, or PER-CONTROL trained-domain limits, one [lo,hi] per gathered control. Prefer the latter — see below.

Returns

Float32Array

outN, holding the N gathered controls — rest plus the ARKit delta, clamped to range, when a matching rest was supplied.

Example

import { arkitToRig, ARKIT_NAMES, gatherFromRigNames } from '@aosengine/character';

const gather = gatherFromRigNames(bundle.manifest.rig.controlNames);
const controls = new Float32Array(gather.length);
const weights = new Float32Array(ARKIT_NAMES.length);

weights[ARKIT_NAMES.indexOf('jawOpen')] = 0.4;
arkitToRig(weights, gather, controls);

---

attachOrtDevice()

function attachOrtDevice(device): OrtDeviceAttachment;

Hand the renderer's GPUDevice to onnxruntime-web.

The property is ort.env.webgpu.device, and in onnxruntime-web 1.29 it is a genuine accessor pair: the SETTER takes a GPUDevice and the GETTER returns a Promise<GPUDevice> (it will create one on demand if nothing was set). Setting it only has effect BEFORE the first WebGPU inference session is created — which is why this must run immediately after prepareLiftDevice and before any decoder is built.

env.webgpu.adapter also exists and is settable, but it is deprecated in 1.29 and it is the wrong lever anyway: an adapter would let ORT build its own device off the same hardware, which is still a DIFFERENT device and still refuses a cross-device buffer. If a future ORT drops the device setter, the fallback is the one below — gpuBufferIo: false, every decoder output downloaded to a CPU Float32Array and re-uploaded by the lift. That costs one round trip per decode and is correct; it is the POC's own demote path (DecoderSession.setDownloadOutputs).

Never throws: a character that cannot share the device still renders, slower.

Parameters

Parameter	Type	Description
device	GPUDevice	The renderer's device, from prepareLiftDevice.

Returns

OrtDeviceAttachment

Whether ORT took the device, why it did not, and whether decoder outputs may therefore stay in GPUBuffers instead of being downloaded.

---

createArkitToGnmMap()

function createArkitToGnmMap(layout, options?): object;

Build an ExpressionSpace.map for a pack's layout.

The returned function is what @aosengine/animation's createAnimator calls every frame, so it ALLOCATES NOTHING: the table is resolved to absolute slots here, once, and the per-frame path is a loop over a flat array.

Terms that fall outside the layout — a coefficient a truncated pack dropped, a region that pack does not declare — are discarded at build time and reported, rather than being clamped into a neighbouring coefficient.

Parameters

Parameter	Type	Description
layout	HeadExtLayout	The pack's header.headExt.
options	ArkitToGnmOptions	See ArkitToGnmOptions.

Returns

object

The mapper plus what it resolved: dim to size the output with, the terms it kept, and the entries it dropped with the reason.

dim

dim: number;

dropped

dropped: string[];

map

map: (arkit, out) => void;

Parameters

Parameter	Type
arkit	Float32Array
out	Float32Array

Returns

void

terms

terms: number;

Example

import { createArkitToGnmMap, parseAosRig } from '@aosengine/character';

const pack = parseAosRig(bytes);
const { dim, map } = createArkitToGnmMap(pack.header.headExt);
const animator = createAnimator({ root, expressionSpace: { kind: 'gnm', dim, map } });

---

createCharacter()

function createCharacter(bundle, init): Promise<Character>;

Bring a loaded bundle to life against a splat sink.

Parameters

Parameter	Type	Description
bundle	CharacterBundle	The already-loaded bundle, from loadCharacterBundle.
init	CreateCharacterOptions	The renderer whose device everything borrows, the scene to parent the splat object under, the sink whose slots the branches are lifted into, and the optional tuning in init.options.

Returns

Promise<Character>

The live character: setRig / setExpression / setBodyPose / setLookAt to drive it, update once a frame, and dispose to give back every GPU buffer, ORT session and slot range.

Throws

(from ./errors.ts, via prepareLiftDevice) when the renderer is not on the WebGPU backend.

Example

const bundle = await loadCharacterBundle('/assets/characters/myra');
const character = await createCharacter(bundle, { renderer, scene, sink });
character.setExpression(arkitWeights);
character.update(dt, camera);

---

createRigPreview()

function createRigPreview(options): Promise<RigPreview>;

Put a rig backend on screen without any decoders.

Parameters

Parameter	Type	Description
options	CreateRigPreviewOptions	See CreateRigPreviewOptions.

Returns

Promise<RigPreview>

The live preview.

Throws

When the renderer is on the WebGL fallback.

Example

import { createRigPreview, GnmRigBackend, jointTint } from '@aosengine/character';

const preview = await createRigPreview({
  renderer,
  sink: splat,
  backend: new GnmRigBackend({ packFile: 'myra_head.aosrig' }),
  getBytes: (name) => files.get(name),
});
preview.setControls(headExt);
preview.render();

---

debugLiftDispatch()

function debugLiftDispatch(slotCount): number;

Dispatch size for a slot range.

Parameters

Parameter	Type	Description
slotCount	number	Slots the preview owns.

Returns

number

Workgroups to dispatch — every slot is visited, including the ones past the vertex count, because those have to be cleared rather than left stale.

Example

import { debugLiftDispatch } from '@aosengine/character';

console.log(debugLiftDispatch(17_821)); // 279

---

fitVertsTransform()

function fitVertsTransform(aabb, options?): object;

A rig -> object transform that centres the vertices and scales them to a target size.

The two rigs disagree about units (GNM metres, ORL centimetres) and neither puts a head anywhere near the origin, so a preview that did not normalise would render off screen at the wrong size and look like a failure. This is a VIEWING convenience and nothing downstream depends on it.

Parameters

Parameter	Type	Description
aabb	VertsAABB	The backend's neutral-pose bounds, in its own units.
options	{ height?: number; offset?: readonly [number, number, number]; scale?: number; }	Placement overrides.
options.height?	number	Object-space height the bounds are scaled to. Defaults to 0.35 m, a head at conversational distance.
options.offset?	readonly [number, number, number]	Where the fitted centre lands. Defaults to the origin.
options.scale?	number	A uniform scale, overriding height outright.

Returns

object

A row-major 3x4 affine, ready for DebugLiftParams.transform, plus the scale it chose and the object-space centroid the normal shading needs.

centroid

centroid: [number, number, number];

scale

scale: number;

transform

transform: number[];

---

flatTint()

function flatTint(vertexCount, colour?): Uint32Array;

One flat colour for every vertex.

Parameters

Parameter	Type	Description
vertexCount	number	Vertices to colour.
colour	readonly [number, number, number]	RGB in [0, 1]. Defaults to a light neutral.

Returns

Uint32Array

One packed RGBA per vertex, opaque.

---

gatherFromRigNames()

function gatherFromRigNames(rigNames): number[];

Parameters

Parameter	Type	Description
rigNames	string[]	The bundle's rig_names.json (N control names).

Returns

number[]

N indices into the 188-control arkitToMh output.

Throws

When rigNames is not a non-empty string array, or any name is missing from the canonical 188 ordering.

---

gazeToEyeRotations()

function gazeToEyeRotations(gaze): [EyeQuaternion, EyeQuaternion];

Gaze [pitch_L, yaw_L, pitch_R, yaw_R] (radians) -> a rotation per eye.

GNM composes R = Rx(pitch) · Ry(yaw) (block.gaze_to_eye_rotations), which as quaternions is q = qx(pitch) * qy(yaw) in that order. Pitch > 0 looks DOWN and yaw > 0 looks toward character-LEFT, both in the head-local canonical frame. Getting the composition order backwards is a gaze that is right on each axis alone and wrong on the diagonal, which is exactly the kind of thing that survives a casual look.

Parameters

Parameter	Type	Description
gaze	ArrayLike<number>	Four radians: [pitch_L, yaw_L, pitch_R, yaw_R].

Returns

[EyeQuaternion, EyeQuaternion]

The left and right eye rotations, each (w, x, y, z).

---

gnmForward()

function gnmForward(pack, headExt): GnmForwardResult;

head_ext -> head-local vertices, BEFORE the seam stitch and before skinning.

Reproduces BakedHead.forward: the linear expression model, then GNM's own eye rotation blended by each eye joint's skinning weight.

Parameters

Parameter	Type	Description
pack	AosRigPack	The parsed .aosrig, supplying the neutral, the basis and the eyes.
headExt	ArrayLike<number>	One frame's packed rig vector — expression coefficients plus the four gaze angles, split by unpackHeadExt.

Returns

GnmForwardResult

The head-local vertices as (V,3) in METRES, before the seam stitch and before skinning.

---

gnmPose()

function gnmPose(
   pack, 
   headExt, 
   jointWorld?
): Float32Array;

The full rig -> vertices path on the CPU: gnmForward, the baked neck seam, then linear-blend skinning against jointWorld.

jointWorld is J*16 row-major world matrices in the pack's compact joint order, defaulting to the pack's own rest (which makes the skinning the identity).

Parameters

Parameter	Type	Default value	Description
pack	AosRigPack	undefined	The parsed .aosrig.
headExt	ArrayLike<number>	undefined	One frame's packed rig vector, as gnmForward takes it.
jointWorld	Float32Array	pack.restWorld	J*16 row-major world matrices in the pack's compact joint order. Defaults to pack.restWorld.

Returns

Float32Array

The posed vertices as (V,3) in METRES, in the body's bind space — bindTransform is folded into the skin matrices rather than applied here.

---

headExtNames()

function headExtNames(layout): string[];

Every control name the layout declares, in order — the manifest's control_names.

Parameters

Parameter	Type	Description
layout	HeadExtLayout	The pack's header.headExt.

Returns

string[]

layout.dim names: <region>_000-style per-coefficient names, then the four gaze angles.

---

heightTint()

function heightTint(verts, vertexCount): Uint32Array;

A colour per vertex from its height inside the vertex bounds — a plain ramp.

The backend-agnostic fallback: it needs nothing but the vertices, so it works for a rig whose pack this package cannot read at all.

Parameters

Parameter	Type	Description
verts	Float32Array	(V,3) vertices in the rig's own units.
vertexCount	number	Vertices to colour.

Returns

Uint32Array

One packed RGBA per vertex, opaque. A zero-height bound yields a flat ramp value rather than a division by zero.

---

jointTint()

function jointTint(pack): Uint32Array;

A colour per vertex from the vertex's dominant skinning joint, with the eyes called out.

This is the colouring that makes a rig bug obvious: a vertex weighted to the wrong joint is a wrongly-coloured patch on an otherwise clean head, and a pack whose eye weights did not survive the bake has no cyan in it at all.

Parameters

Parameter	Type	Description
pack	AosRigPack	The parsed .aosrig, for skinIndex, skinWeight and eyeWeights.

Returns

Uint32Array

One packed RGBA per vertex, opaque.

---

loadCharacterBundle()

function loadCharacterBundle(url, options?): Promise<CharacterBundle>;

Fetch and parse a character bundle.

Parameters

Parameter	Type	Description
url	string	The bundle directory, or any string the supplied resolver understands.
options	LoadCharacterBundleOptions	fetch / signal / resolver and the fp16 preference.

Returns

Promise<CharacterBundle>

Everything createCharacter needs: the parsed manifest with the engine's rig and expression blocks, per-branch mesh topology in manifest order, every eagerly fetched file kept resident, a resolver for the lazy ones, whether the fp16 decoders are to be used, and the resident byte total.

Example

import { loadCharacterBundle } from '@aosengine/character';

const bundle = await loadCharacterBundle('/assets/characters/myra/');
console.log(bundle.manifest.rig.backend, bundle.manifest.expressionSpace.dim);

---

orlDriveMode()

function orlDriveMode(vertexCount): OrlDriveMode;

Decide the drive mode from the branch mesh alone.

Parameters

Parameter	Type	Description
vertexCount	number	The branch mesh's vertex count.

Returns

OrlDriveMode

skin at exactly ORL_HEAD_VERTS, shell at or below ORL_SHELL_MAX_VERTS, and none for anything bigger — a real body or hair branch, which renders at its neutral pose.

---

packAosRig()

function packAosRig(header, blobs): Uint8Array;

Build a pack.

Blobs are written in the given order and the header's key order is fixed, so packing the same bake twice is BYTE-IDENTICAL — which is what makes a re-bake diff meaningful.

Parameters

Parameter	Type	Description
header	Omit<AosRigHeader, "buffers">	Everything but buffers, which is derived from the laid-out blobs.
blobs	AosRigBlob[]	The blobs, in the order they are written. Each is copied verbatim; the declared shape and dtype are what parseAosRig checks its length against.

Returns

Uint8Array

The whole container: magic, header JSON, then the 8-byte-aligned blobs.

Throws

When the header's encoded length does not converge in four passes.

---

packRgba()

function packRgba(
   r, 
   g, 
   b, 
   a?
): number;

Pack a colour the way pack4x8unorm does.

Parameters

Parameter	Type	Default value	Description
r	number	undefined	Red in [0, 1]; out-of-range values are clamped.
g	number	undefined	Green in [0, 1].
b	number	undefined	Blue in [0, 1].
a	number	1	Alpha in [0, 1]. Defaults to 1.

Returns

number

The 32 bits a WGSL unpack4x8unorm reads back as that colour.

Example

import { packRgba } from '@aosengine/character';

console.log(packRgba(1, 0, 0, 1).toString(16)); // 'ff0000ff'

---

parseAosRig()

function parseAosRig(bytes): AosRigPack;

Parse a .aosrig pack.

Throws on anything it cannot trust. A GNM head that silently loads half its expression basis renders a face that moves a little and looks nearly right, which is the failure mode with no symptom.

Parameters

Parameter	Type	Description
bytes	ArrayBuffer | Uint8Array<ArrayBufferLike>	The downloaded pack, as bytes or a whole buffer.

Returns

AosRigPack

The parsed pack: the header plus typed views over the blobs. Only skinWeight is materialised (widened from its f16 lanes); everything else is a view into bytes. bindTransform falls back to the identity when the bake declared none.

Throws

On a bad magic, a length that disagrees with the header, a version or model this build does not read, a missing required blob, or any array whose length contradicts V, E, the joint list or maxInfluence.

Example

import { parseAosRig } from '@aosengine/character';

const pack = parseAosRig(await (await fetch('/assets/myra.aosrig')).arrayBuffer());
console.log(pack.vertexCount, pack.header.headExt.dim); // 17821 387

---

prepareLiftDevice()

function prepareLiftDevice(renderer): GPUDevice;

The GPUDevice a character's compute runs on: the renderer's own.

Throws CharacterUnsupportedError when the renderer is on the WebGL fallback backend or has not been initialised, because there is then no device to borrow and no CPU path to demote to.

Parameters

Parameter	Type	Description
renderer	RendererLike	The engine's WebGPURenderer, already init()ed; only backend.device and backend.isWebGPUBackend are read.

Returns

GPUDevice

The renderer's own device, confirmed to grant at least REQUIRED_STORAGE_BUFFERS storage buffers per shader stage.

Example

import { attachOrtDevice, prepareLiftDevice } from '@aosengine/character';

await renderer.init();
const device = prepareLiftDevice(renderer);
attachOrtDevice(device); // before the first ONNX session

---

regionSlices()

function regionSlices(layout): Record<string, {
  end: number;
  start: number;
}>;

Where each expression region sits inside head_ext.

Parameters

Parameter	Type	Description
layout	HeadExtLayout	The pack's header.headExt.

Returns

Record<string, { end: number; start: number; }>

A { start, end } half-open COEFFICIENT range per region name, tiled in declaration order from 0. Gaze sits past the last region and is not included.

---

unpackHeadExt()

function unpackHeadExt(headExt, layout): object;

head_ext -> the two halves the model consumes.

The split is a slice, not a parse, and it is the one place the 383/4 boundary is written down on this side. Gaze is [pitch_L, yaw_L, pitch_R, yaw_R] in radians, head-local: pitch > 0 looks DOWN (about +X), yaw > 0 looks toward character-LEFT (about +Y). Neck and head rotation are NOT here — they live in the body joints neck_01 / neck_02 / head.

Parameters

Parameter	Type	Description
headExt	ArrayLike<number>	One frame's rig vector, at least layout.dim long. A longer one is accepted and its tail ignored.
layout	HeadExtLayout	The pack's header.headExt, which owns the 383/4 boundary.

Returns

object

The exprDim expression coefficients and the gazeDim gaze angles, as fresh arrays.

expr

expr: Float32Array;

gaze

gaze: Float32Array;

Throws

When headExt is shorter than the layout declares.

---

uvTint()

function uvTint(uv, vertexCount): Uint32Array;

A colour per vertex from the mesh's UVs: red = u, green = v.

The UV view is how a texture-space problem shows itself — a seam in the wrong place, a flipped island, a bake that dropped uv entirely (every vertex then comes out the same colour, which is itself the finding).

Parameters

Parameter	Type	Description
uv	Float32Array	(V,2) texture coordinates.
vertexCount	number	Vertices to colour; uv must hold at least 2 * vertexCount.

Returns

Uint32Array

One packed RGBA per vertex, opaque.

---

verifyOrtDevice()

function verifyOrtDevice(device): Promise<boolean>;

Confirm ORT is really on device once the first session exists.

The getter resolves to whatever ORT actually built, so this is the only honest check — a successful set is an intention, not an outcome. A mismatch DEMOTES loudly rather than handing the lift a foreign buffer.

Parameters

Parameter	Type	Description
device	GPUDevice	The device ORT was asked to adopt, from prepareLiftDevice.

Returns

Promise<boolean>

True when ORT's device is that same object; false — with a warning — when ORT built its own or exposes no env.webgpu, meaning outputs must be downloaded.

---

writeDebugLiftParams()

function writeDebugLiftParams(params, into?): ArrayBuffer;

Pack the params uniform.

Separate from the class so the byte layout can be asserted in a node test — a mis-packed uniform is a head at the wrong scale or in the wrong place, with no error anywhere.

Parameters

Parameter	Type	Description
params	DebugLiftParams	The values to write.
into?	ArrayBuffer	An existing 96-byte buffer to fill. A fresh one is allocated when omitted, which the per-frame path never does.

Returns

ArrayBuffer

The buffer, filled.

Throws

When transform is not 12 numbers, or into is the wrong size.