Decision

Replace Jaeger all-in-one with VictoriaTraces in the single-machine observability stack.

• VictoriaTraces listens on 10428 (HTTP, UI + query + OTLP HTTP ingest).
• OTEL Collector exports traces via otlphttp/traces to http://localhost:10428/insert/opentelemetry.
• Data persists at /var/lib/victoriatraces on the shared Fly volume.
• Jaeger is removed entirely; no separate query UI process is needed.

Port map after this change:

Consequences

• All four services are now Apache 2.0; the three storage backends are from VictoriaMetrics.
• VictoriaTraces query UI is at http://...:10428/select/vmui.
• fly proxy 10428:10428 replaces the former fly proxy 16686:16686 for trace inspection.
• No explicit trace TTL is configured; add -retentionPeriod if disk pressure becomes a concern.

Decision

Run VictoriaMetrics (metrics), VictoriaLogs (logs), and Grafana Tempo (traces) under supervisord in a single Fly Machine, with an OpenTelemetry Collector as the ingest router.

• OTEL Collector listens on 4317 (gRPC) and 4318 (HTTP) on the private network.
• Collector routes: metrics -> VictoriaMetrics (8428 via Prometheus remote write), logs -> VictoriaLogs (9428 OTLP), traces -> Tempo (5317 OTLP internal).
• Data persists on a 10 GB Fly volume.

Consequences

• OTLP ports must not be exposed publicly — they accept unauthenticated writes.
• Other Fly apps send telemetry to multiplayer-fabric-observability.internal:4317 or :4318.
• Grafana Tempo v3 config changed significantly from v2: ingester and compactor top-level keys were removed; -target=all flag is required to run all components.
• See GHCR package ownership decision for why the zone binary image was renamed and moved to multiplayer-fabric-zone.

Decision

Replace Grafana Tempo with Jaeger all-in-one in the single-machine observability stack.

• Jaeger listens on 14317 (OTLP gRPC, internal) and exposes its UI on 16686.
• OTEL Collector routes traces to localhost:14317 instead of the former Tempo port (5317).
• Jaeger uses Badger embedded storage (/var/lib/jaeger/data and /var/lib/jaeger/keys) persisted on the same 10 GB Fly volume.
• tempo-config.yaml is deleted; Jaeger is configured entirely via CLI flags in supervisord.conf.

Port map after this change:

Consequences

• Jaeger UI replaces the Tempo query endpoint: fly proxy 16686:16686 instead of fly proxy 3200:3200.
• Badger is ephemeral-safe only with --badger.ephemeral=false; this flag must be set or traces are lost on restart.
• No explicit trace TTL is configured; traces accumulate until disk pressure. Add --badger.span-store-ttl if retention limits are needed.
• Jaeger’s Badger backend does not support distributed mode; acceptable for a single-machine stack.

Decision

Each repository builds and publishes the GHCR images it directly uses in its own deploy:

• multiplayer-fabric-baker builds and owns godot-editor-double.
• multiplayer-fabric-zone builds and owns multiplayer-fabric-zone-godot (renamed from godot-zone-double to reflect ownership).

workflow_run triggers only fire within the same repository, so the zone binary build workflow must live in the zone repo for the deploy to auto-trigger after a successful build.

Consequences

• Package names must reflect the owning repo to avoid confusion.
• Moving a package between repos requires deleting it (needs delete:packages API scope) and rebuilding, or renaming.
• Cross-repo GHCR access requires either a PAT with read:packages scope or making the package public.

Decision

Use WebTransport (HTTP/3 over QUIC) for all game traffic. Clients connect to the Elixir gateway on UDP port 443, which proxies into the Godot zone server on UDP port 7443.

Consequences

• UDP eliminates TCP head-of-line blocking.
• Fly.io passes UDP without DNAT, so the app must bind to the same port clients connect to.
• Port 443 requires running as root (or CAP_NET_BIND_SERVICE). The gateway container runs as root.
• Datagrams are used for game state messages. Streams are avoided for ping/pong to sidestep stream half-close deadlock (client must close write side before server response fires).
• Cloudflare’s proxy cannot forward QUIC/UDP. All game-traffic DNS records must be DNS-only (no proxy).

Decision

Build zone server binaries as target=template_release precision=double with no Mono. Build from the V-Sekai fork at V-Sekai-fire/multiplayer-fabric-build@b27142e94.

Consequences

• template_release has no import or export tools, making it smaller and faster to start.
• libstdc++-static must be included in the AlmaLinux 9 build environment because template_release links libstdc++ statically.
• The binary filename is godot.linuxbsd.template_release.double.x86_64.
• --headless flag is required at runtime.
• Double precision matches the physics and networking precision of the rest of the stack.

Decision

Deploy all services on Fly.io in the iad (Ashburn, Virginia) region.

• Machines handle deployment; Terraform (fly-apps/fly provider v0.0.21) manages resources (apps, IPs, volumes, secrets) but not machines, due to Machines API limitations in the provider.
• Private networking uses Fly’s 6PN (WireGuard mesh). All .internal DNS resolves to IPv6 only, requiring socket_options: [:inet6] in any Erlang/Elixir TCP stack.
• Dedicated IPv4 is required for UDP. Fly passes UDP without DNAT; the app must bind to the same port clients connect to.
• On-demand machines (Fly Machines API) are used for the baker — one machine per bake job, exits when done.

Consequences

• All services must be in the same region to communicate over the private network without egress charges.
• .flycast requires [[services]] on the target app; .internal works without it.
• Cloudflare proxy must be disabled for UDP/QUIC endpoints; DNS-only records point directly to the Fly IP.
• Terraform state is stored on a tfstate orphan branch in the infra repo to avoid requiring a remote backend.

Decision

Run a single-node CockroachDB on Fly.io with mTLS for all connections and three application roles:

• gateway_admin — DDL access for Ecto migrations
• gateway_writer — DML access for the running application
• gateway_reader — read-only access

Certs are generated by gen_crdb_certs.sh with 100-year validity and stored as Fly secrets. The node cert must include both serverAuth and clientAuth in extendedKeyUsage because CockroachDB uses the node cert for internal gRPC client authentication.

Consequences

• --advertise-addr must be localhost. A flycast address routes the internal gRPC loopback through Fly’s NAT, breaking the admin UI.
• prepare: :unnamed is required in Postgrex to avoid statement-cache OOM on single-node deployments.
• Port 26257 is never publicly exposed. Access is via Fly’s private network (6PN) using socket_options: [:inet6] in Ecto, because .internal DNS returns only AAAA records.
• The root cert is provisioned on the CRDB machine only. gateway_admin is the highest-privilege cert available to the application.

The Context

Describe the circumstances that led to this proposal. Give enough background that someone unfamiliar with the area can understand why the work matters. State what you were trying to accomplish, not what solution you reached for first.

The Problem Statement

State the specific problem, limitation, or gap being addressed. Be concrete. A reader should be able to confirm whether a proposed solution actually solves it.

Design

Detail the proposed solution precisely enough that it could be implemented without improvising the design. Include code, diagrams, or examples where they reduce ambiguity. Vague or underspecified proposals cannot be acted on.

CRIS Score

The Downsides

Describe the costs, risks, or trade-offs. Every proposal has them.

The Road Not Taken

Describe the alternatives considered and why this approach was chosen over them.

Status

Status: Draft

Decision Makers

• [Name or role]

Tags

• [relevant-tag-1], [relevant-tag-2], YYYYMMDD-this-filename-without-extension, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

The stack: multiplayer-fabric-gateway (Elixir, UDP 443 → zone UDP 7443, WebTransport/QUIC datagrams), multiplayer-fabric-zone (headless Godot template_release double), multiplayer-fabric-uro (Phoenix shard registry and asset API), single-node CockroachDB on Fly’s 6PN private network.

Zone authority is geometric — ZoneRange.contains(hilbert3D(pos)) determines ownership, gossip-learned, with uniqueness proven under DisjointRanges. Scores and persistent player state sit in a reserved logical partition of the 30-bit Hilbert space (the persona partition), governed by the same invariant as spatial zones. This is the first attempt to load all four layers together under cross-platform input.

The Problem Statement

The full scenario — cross-platform clients, authoritative rolling-core physics, and a causal cross-zone score commit — is too complex to build and validate in one pass. A system assembled from untested parts at full complexity has no debuggable baseline.

Design

Players start mid-breach on a moving Maglev Data-Courier train, fight drones, retrieve Quantum Data-Cores rolling across the banking floor, and slot them into the mainframe terminal within 3 minutes. The PCVR client sees the car as a waist-height diorama; the Steam Deck client sees the same instance as an isometric action-RPG.

Each layer is a separate cycle — a working system and a gate before the next begins.

Taskweft-confirmed ordering: 13 cycles (0–12) running in number order. Cycle 0 gates everything. Parallel tracks 2–5 all start after Cycle 1; Cycle 2 (observability) must complete before Cycle 6; Cycle 3 (baker) must complete before Cycle 9; Cycle 5 (zone-console) must complete before Cycle 10; bake jobs (1–3 h) always complete before the networking cycles (≥1 day each).

Cycle 7 combines 1 Hz routing and 10 Hz merge into one cycle; Cycle 8 adds full headset rate. If the team is confident in the merge after Cycle 7, Cycle 8 can be skipped.

multiplayer-fabric-predictive-bvh (interest management via predictive BVH) is exercised in Cycle 10: with 16 moving clients the BVH computes at least 2 distinct interest zones, verified in zone-console. Full cross-zone interest management (multiple zone servers) is left for a separate cycle series.

CRIS Score

The Downsides

Cycle 10 requires Cyberprep art assets and MToon shaders tuned for both targets before it can run. Physics replication will stress WebTransport bandwidth. If the persona zone goes down, that player’s attributes are offline until it recovers. The shared-world UX for a standing VR player and a seated thumbstick player needs a separate design pass that cannot begin until Cycle 10 is stable.

Three gaps against industry par remain out of scope for this smoke test. Network condition simulation (artificial latency, jitter, packet loss) is standard in Unity Netcode, Photon, and Mirror but is not injected in these cycles. Reconnect storm testing — all 16 clients disconnecting and reconnecting simultaneously — is absent; VRChat’s largest outage traced to this failure mode. Automated CI/CD execution is also absent; industry runs a 5-minute smoke gate on every deploy, whereas these cycles are human-run, week-scale exercises.

The Road Not Taken

A steampunk skyship setting was rejected — it conflicts with the anime/VRM art targets and drifts toward fantasy MMO conventions.

Building the full scenario in one pass was the original plan, dropped when the staging approach proved easier to attribute failures.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer
• Game Director

Tags

• smoke-test, cross-platform, tilt-shift, cyberprep, in-medias-res, persona-zone, shared-nothing, galls-law, 20260506-maglev-intercept-smoke-test, present-proposal-template

Further Reading

@misc{gall_1975,
  author = {Gall, John},
  title  = {Systemantics: How Systems Work and Especially How They Fail},
  year   = {1975}
}

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

The gateway merges two input types at full rate after Cycle 9. This cycle loads the Maglev train scene with stationary cores on a banking train, verifies replication across clients, and confirms VRM avatar loading from the chunk store. Banking is a scene parameter change; VRM loading is verified in the same session as physics baseline.

Baked assets from Cycle 3 must be available in the chunk store before this cycle begins. Banking and VRM loading are combined here to reduce the number of cycles before the first physics demo.

The Problem Statement

Physics replication to clients with different frame rates requires validation before dynamic forces are added. Adding banking motion and rolling cores at the same time as replication makes any failure unattributable. This cycle proves the replication layer works before the physics is made dynamic.

Design

Load the Maglev train scene (greyboxed for this cycle: a CSG floor + banking pivot is sufficient) on the zone server. The train banks on a fixed schedule. Quantum Data-Cores sit stationary on the floor under authoritative physics (held by a constraint). The zone server sends datagram state each tick; both clients display the core and avatar positions.

The PCVR client sees the car as a waist-height diorama; the Steam Deck client sees the same instance as an isometric action-RPG. Both load mire.vrm (baked in Cycle 3 from multiplayer-fabric-humanoid-project/humanoid/art/mire/mire.vrm) as the avatar.

Pass criteria:

• Zone server loads the Maglev train scene from the chunk store
• Core positions agree between both clients each tick for 60 seconds under continuous banking
• Zone server tick rate holds at 20 Hz with physics active
• mire.vrm loads from the chunk store on both clients without error; humanoid skeleton root present
• No entity desync over the run

Estimate

10 days (2026-06-02 → 2026-06-16). First cycle to load game assets and run physics. The zone server initial deploy took 2 days; adding a physics scene, banking animation, and VRM loading from the chunk store is the first substantial game dev work with no prior baseline. Projecting 2× the zone-console build time (6 working days) plus physics integration overhead.

CRIS Score

The Downsides

A passing Cycle 9 does not validate rolling physics or causal ordering; those require Cycle 10.

The Road Not Taken

Starting with rolling cores was rejected — replication and rolling-core physics are easier to separate when cores are stationary.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer
• Game Director

Tags

• maglev-cycle-9, physics-vrm, replication, galls-law, 20260506-maglev-cycle-9-physics-vrm, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

Cycle 7 confirms routing and merge at 10 Hz. Cycle 8 raises the PCVR client to operating rate — continuous 6-DOF IK at headset cadence — alongside full-rate gamepad input, and verifies the gateway merges both without head-of-line blocking.

The Problem Statement

6-DOF IK datagrams arrive an order of magnitude more frequently than gamepad events. The gateway’s merge path must handle that rate differential without blocking lower-frequency gamepad events. A head-of-line stall on the IK queue would delay or drop gamepad events silently.

Design

The PCVR client sends 6-DOF IK datagrams at full headset rate. The Steam Deck client sends gamepad datagrams at normal cadence. Both arrive on UDP 443. The gateway merges them per tick and forwards to the zone server on UDP 7443. The world is static.

Pass criteria:

• Zone server processes ticks containing both input types without stalls or skipped events
• No gamepad datagram is silently dropped over 60 seconds under full IK load
• Gateway CPU stays within budget under both streams at full rate

Estimate

5 days (2026-05-26 → 2026-06-02). The merge path is an extension of Cycle 7; the risk is head-of-line behaviour under load, which requires profiling time. A hardware re-run on physical Meta Quest hardware before Cycle 9 adds 1 day outside this window.

CRIS Score

The Downsides

The Meta XR Simulator on macOS covers the initial run, but Cycle 8 must be re-run on physical hardware before Cycle 9 begins — the simulator does not reproduce real headset IK datagram rates under motion.

The Road Not Taken

Skipping Cycles 7 and 5 together was rejected — a failure at full rate conflates routing bugs with rate bugs, leaving both unattributable.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-8, ik-merge, heterogeneous-input, galls-law, 20260506-maglev-cycle-8-ik-merge, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

Two same-type Godot clients share a zone after Cycle 6. Cycle 7 swaps one Steam Deck client for a PCVR client and confirms that a single 6-DOF IK datagram travels through the gateway and reaches the zone server. The rate and merge logic are not yet in scope.

The Problem Statement

A PCVR client’s IK datagram format differs structurally from a gamepad datagram. The gateway has never parsed one. Before testing merge behaviour at full IK rate, the routing path itself must be shown to work for at least one IK datagram.

Design

The PCVR client — running the solved multiplayer-fabric-interaction-system addon via xr_controller_interaction_helper.gd and xr_action_host.gd — starts at 1 Hz IK and ramps to 10 Hz. The Steam Deck client sends gamepad datagrams at normal cadence. The gateway routes and merges both streams to the zone server on UDP 7443. The world is static.

The 1 Hz phase confirms the IK datagram format parses correctly. The 10 Hz phase confirms the gateway’s merge logic is correct under concurrent input before full rate is introduced.

Pass criteria:

• Zone server receives and logs at least one IK datagram without parse error at 1 Hz
• Zone server receives a merged tick containing both input types at 10 Hz
• No gamepad datagram dropped over 60 seconds at 10 Hz
• No gateway crash or restart

Estimate

5 days (2026-05-19 → 2026-05-26). The IK datagram format is new but the gateway routing path is identical to gamepad. The interaction system (xr_controller_interaction_helper.gd) is solved; the work is serialization format definition and gateway route registration.

CRIS Score

The Downsides

10 Hz IK does not trigger head-of-line pressure; that stress test is Cycle 8 at full headset rate.

The Road Not Taken

Going straight to full headset rate was rejected — a routing bug and a head-of-line bug at the same time are harder to separate than routing at 1 Hz and merge at 10 Hz in one cycle.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-7, ik-routing-merge, heterogeneous-input, galls-law, 20260506-maglev-cycle-7-ik-routing-merge, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

With one Godot client verified end-to-end and the observability stack live (Cycle 2), the next question is whether the gateway can multiplex two same-type connections into one zone server tick while keeping both clients synchronized. Observability must be running before this cycle begins — a broadcast failure here is unattributable without gateway traces.

The Problem Statement

16 clients in one zone is the minimum to qualify as MMOG load. 14 are taskweft bots (~15 MB each, not full Godot processes); 1 is the PCVR human; 1 is the Steam Deck human. Connection-slot handling, datagram ordering, and state broadcast to 16 sockets have not been tested under concurrent load.

Design

16 clients connect to the gateway simultaneously. All receive datagram state from the same zone server instance each tick. One static entity in an empty world.

Pass criteria:

• All 16 clients receive consistent entity state each tick for 60 seconds
• No client observes a tick gap or duplicate
• Zone-console shows 16 entities at 20 Hz
• Gateway process memory stays flat under 16 connections
• All 14 bots disconnect and reconnect simultaneously; gateway recovers within 5 seconds and all clients resync without a restart
• Under 100 ms injected latency and 2% packet loss, no client desyncs over 30 seconds

Estimate

5 days (2026-05-13 → 2026-05-19). Taskweft bots are proven (93 PropCheck properties passing). The work is wiring 14 bots to the gateway and adding the reconnect storm and network condition harnesses. No prior multi-bot gateway test exists in the history; projecting from the gateway build pace.

CRIS Score

The Downsides

16 same-type bots leave the heterogeneous input path unvalidated.

The Road Not Taken

Starting immediately with heterogeneous IK and gamepad inputs at 16-client scale was rejected — a broadcast failure at full complexity would be ambiguous between gateway capacity and input-mux issues. Cycle 6 uses uniform bots to isolate the broadcast path.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-6, mmog-load, gateway, smoke-test, galls-law, 20260506-maglev-cycle-6-mmog-load, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

multiplayer-fabric-zone-console is a ratatui TUI that observes the zone over WebTransport. It is in the production stack and is the operator’s primary visibility into zone health during and after mission cycles. Cycle 5 depends only on Cycle 1 (zone server live) and runs in parallel with the game-logic track.

The Problem Statement

The zone-console has not connected to the zone server under the Fly deployment and confirmed it can display entity state and tick rate. Without a passing Cycle 5, operators have no real-time view into the zone during Cycles 9–10, where physics desync and causal ordering failures are the primary risks.

Design

Start zone-console targeting the Fly zone server. Confirm the TUI displays: connected status, current tick rate (targeting 20 Hz), and at least one entity visible in the entity list.

Pass criteria:

• zone-console connects without error
• Tick rate displayed at 15 Hz or above (full 20 Hz target confirmed in Cycle 10)
• At least one entity visible in the TUI entity list
• Clean disconnect on exit with no orphaned connection

Estimate

2 days (2026-05-09 → 2026-05-12, parallel). Zone-console was built from scratch in 6 working days (2026-04-21–28). The work here is connecting the existing TUI to the deployed Fly gateway rather than a local target — config and endpoint change only.

CRIS Score

The Downsides

A passing Cycle 5 does not validate zone-console under load; the full 20 Hz tick rate and entity count targets are confirmed during Cycle 10.

The Road Not Taken

Deferring to after Cycle 10 was rejected — the observer is needed during Cycle 10 to confirm tick rate holds at 20 Hz under dynamic physics load.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-5, zone-console, ratatui, observer, galls-law, 20260506-maglev-cycle-5-zone-console, present-proposal-template

Further Reading

@techreport{20260506_cycle1,
  title       = {Maglev Cycle 1: Godot Client Gateway Handshake},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260506-maglev-cycle-1-gateway-handshake.md}
}

@techreport{20260501_webtransport,
  title       = {Use WebTransport over QUIC for game traffic},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260501-webtransport-over-quic-for-game-traffic.md}
}

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

Cycle 10 confirms cross-zone causal ordering via VClock.le gating. Cycle 4 verifies the database write path independently: multiplayer-fabric-uro writes a single test record to CockroachDB over Fly’s 6PN without any game logic or causal path involved.

The Problem Statement

The Ecto connection from uro to CockroachDB over Fly’s 6PN using the gateway_writer role has not been exercised under the Fly network topology. Verifying it in isolation means any failure in Cycle 11 is attributable to the causal path, not the connection.

Design

Trigger a test write from multiplayer-fabric-uro directly — a single row to a test table — using the gateway_writer role. The connection uses mTLS (cert generated by gen_crdb_certs.sh), port 26257 over Fly’s 6PN private network (IPv6 only, socket_options: [:inet6]), and prepare: :unnamed in Postgrex to prevent statement-cache OOM on the single-node deployment.

Pass criteria:

• Row appears in CockroachDB within one second of the trigger
• mTLS handshake succeeds; no cert or auth error
• gateway_writer role has DML access; gateway_admin is not used at runtime
• Connection reuse works across multiple sequential writes (prepare: :unnamed confirmed)

Estimate

1 day (2026-05-09, parallel). The uro CockroachDB connection is already working on Fly (commit “Fix CockroachDB connection on Fly: inet6 socket + unnamed prepare”, 2026-05-05). This cycle validates the existing connection against the pass criteria checklist.

CRIS Score

The Downsides

A passing Cycle 4 does not validate the causal write path.

The Road Not Taken

Testing the DB connection as part of Cycle 11 was rejected — a connection failure inside the causal write path is harder to distinguish from a VClock gating failure.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-4, db-connection, cockroachdb, galls-law, 20260506-maglev-cycle-4-db-connection, present-proposal-template

Further Reading

@techreport{20260501_crdb,
  title       = {CockroachDB with mTLS and role-separated access},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260501-cockroachdb-with-mtls-role-separation.md}
}

@techreport{20260501_fly,
  title       = {Fly.io for deployment},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260501-fly-io-for-deployment.md}
}

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

The Maglev train scene and player VRM avatars must be baked before Cycle 9 (physics replication), which is the first cycle to load actual game assets into the zone server. The baker pipeline runs as an on-demand Fly Machine, depends only on the Fly infrastructure proven in Cycle 1, and can run in parallel with Cycles 6–9.

multiplayer-fabric-baker uses Godot in headless editor mode (godot.linuxbsd.editor.double.x86_64, built from multiplayer-fabric-baker via ghcr.io/v-sekai-fire/godot-editor-double:latest) to validate and export scenes. aria-storage chunks the output using casync/desync .caibx format with zstd compression, uploads chunks to the zone-backend chunk store, and posts the .caibx index to uro at /storage/:id/bake.

The Problem Statement

The Cyberprep train environment (MToon shaders, banking train geometry) and the test VRM avatar have not been validated and stored through the baker pipeline under the Fly deployment. Without baked assets in the chunk store, the zone server in Cycle 9 cannot load the scene.

Design

Trigger two bake jobs via the Fly Machines API:

1. Train environment bake: multiplayer-fabric-baker validates the Maglev train scene (greybox CSG geometry is acceptable for cycle validation), exports it, chunks it with aria-storage, and posts the index to uro.
2. Avatar bake: same pipeline for multiplayer-fabric-humanoid-project/humanoid/art/mire/mire.vrm (51 MB), used by both PCVR and Steam Deck clients in Cycle 9.

flyctl machine run registry.fly.io/multiplayer-fabric-baker:latest \
  --app multiplayer-fabric-baker \
  --env ASSET_ID=<uuid> \
  --env URO_URL=https://hub.chibifire.com \
  -- avatar scenes/<uuid>.tscn out/<uuid>.scn

Pass criteria:

• Both bake jobs exit 0
• .caibx index appears in uro at /storage/:id/bake for each asset
• Zone server can fetch and assemble the train scene from the chunk store before Cycle 9 begins

Estimate

3 days (2026-05-09 → 2026-05-13, parallel). The baker pipeline is in production on Fly (14 days of commits 2026-04-23 to 2026-05-06). The work is Maglev-specific bake config (MToon shader settings, VRM export params) and a Fly Machine invocation test. mire.vrm and a greyboxed train scene are the placeholder inputs, so no art track blocks this cycle.

CRIS Score

The Downsides

Building the Cyberprep environment with MToon shaders tuned for both Steam Deck and PCVR requires dedicated art work. Bake failures surface late if the pipeline is not tested early; running this cycle in parallel with Cycles 6–9 surfaces them while networking cycles are still running.

The Road Not Taken

Deferring the bake to just before Cycle 9 was rejected — a failed bake stalls the physics track after the networking cycles that preceded Cycle 9 have already completed.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer
• Game Director

Tags

• maglev-cycle-3, baker, aria-storage, vrm, galls-law, 20260506-maglev-cycle-3-baker, present-proposal-template

Further Reading

@techreport{20260501_fly,
  title       = {Fly.io for deployment},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260501-fly-io-for-deployment.md}
}

@techreport{20260506_ghcr,
  title       = {GHCR packages must be built by the repo that consumes them},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260506-ghcr-package-ownership-same-repo.md}
}

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

The observability stack — OTEL Collector (4317/4318), VictoriaMetrics (8428), VictoriaLogs (9428), VictoriaTraces (10428) — is infrastructure-level and depends only on the Fly deployment being live after Cycle 1. It can run in parallel with the game-logic track (Cycles 3–10) and the DB connection (Cycle 9).

The Problem Statement

No cycle verifies that traces from multiplayer-fabric-gateway, multiplayer-fabric-zone, and multiplayer-fabric-uro reach VictoriaTraces, or that metrics reach VictoriaMetrics. Without this, any failure in Cycles 3–10 has no trace evidence to debug against.

Design

Start the observability stack on the Fly deployment:

• VictoriaMetrics on port 8428
• VictoriaLogs on port 9428
• VictoriaTraces on port 10428 (replaces Jaeger; fly proxy 10428:10428 for local inspection)
• OTEL Collector on 4317 (gRPC) and 4318 (HTTP), routing to all three backends

Trigger a single gateway request and a single zone tick. The zone server binary includes the opentelemetry-godot module compiled into the engine (verify with --test --test-case="*OTel*" before the run). Verify each service emits OTLP spans to the Collector.

Pass criteria:

• VictoriaTraces UI at http://localhost:10428/select/vmui shows spans from at least one gateway request
• VictoriaMetrics at port 8428 returns a non-empty result for a zone-server metric query
• VictoriaLogs at port 9428 shows at least one log line from uro
• All four services are Apache 2.0 (no Tempo, no Jaeger)

Estimate

3 days (2026-05-09 → 2026-05-13, parallel). VictoriaMetrics/Logs/Traces share a config pattern; the only unknown is Fly volume persistence. Comparable to the zone-console Fly deploy (2 days of config commits on 2026-04-24–25).

CRIS Score

The Downsides

A passing Cycle 2 does not validate trace volume or retention under mission load; that stress only appears during Cycles 9–2.

The Road Not Taken

Deferring observability to after Cycle 11 was rejected — a failure in Cycle 8 (dynamic physics) or Cycle 10 (score write) with no trace evidence requires rerunning the entire cycle with instrumentation added, doubling the debugging time.

Status

Status: In Progress (producer side complete 2026-05-07; verification pending)

The observability stack itself is deployed and running on Fly (machine 781ed59b437968 started, volume observability_data 10GB iad attached) per verify_fly_state workflow output. The producer side has now been wired in all three apps:

• ✅ gateway PR #8 — :opentelemetry deps + gateway.dispatch span around Gateway.Router.handle/1
• ✅ uro PR #13 — :opentelemetry_phoenix + :opentelemetry_ecto automatic instrumentation
• ✅ zone PR #3 — bootstrapped project/main.gd initializing the engine’s module_open_telemetry, emits zone.boot span

All ship to multiplayer-fabric-observability.internal:4318 (OTLP/HTTP over Fly’s 6PN). End-to-end verification runs via .github/workflows/verify_observability.yml on multiplayer-fabric-infra — uses flyctl ssh console -C to query the Victoria* APIs from inside the observability machine (the OTLP ports are intentionally private). Pending: trigger the verify workflow after redeploys land and confirm the cycle 1 ping span is visible in VictoriaTraces.

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-2, observability, victoriatraces, victoriametrics, otel, galls-law, 20260506-maglev-cycle-2-observability, present-proposal-template

Further Reading

@techreport{20260506_observability_victoriatraces,
  title       = {Replace Jaeger with VictoriaTraces for trace storage},
  institution = {V-Sekai Fire},
  year        = {2026},
  type        = {Architecture Decision Record},
  note        = {decisions/20260506-observability-stack-victoriatraces.md}
}

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

Cycles 0–5 prove the networking, physics, and persistence layers. Cycle 9 is the first cycle to run a real physics scene with assets; by the time it passes, the team has actual frame budget data from zone-console, a working grab-ready physics scene, and the interaction system (xr_controller_interaction_helper.gd) half-integrated. That is the earliest point at which game cycle estimates are reliable.

The Problem Statement

Six game systems remain unbuilt for the vertical slice and have no prior baseline in this codebase. Estimating them before Cycle 9 passes produces numbers with no evidence. Cycle 12 produces scoped ADRs and estimates for each remaining system using the Cycle 9 scene as a measured baseline.

Design

Run Cycle 12 immediately after Cycle 9 passes. For each of the six systems below, produce one ADR following the present-proposal-template with a pass criteria checklist, CRIS score, and estimate derived from Cycle 9 frame budget and zone-console tick data.

Art asset production (train geometry, drone models, terminal, Quantum Data-Core models, VRM avatars with MToon shaders) is a parallel track owned by the art team. Cycle 12 produces the art brief and schedule alongside the engineering ADRs; it does not estimate or gate on art completion.

Pass criteria:

• One ADR per system above, with pass criteria checklist and CRIS score
• Each ADR includes a working-days estimate with a stated basis in Cycle 9 measurements
• Art brief delivered to art team with asset list, MToon shader targets, and poly/texture budgets for both PCVR diorama and Steam Deck isometric viewports
• Updated project timeline incorporating all six game cycles into the Maglev Intercept sequence

Estimate

3 days (2026-06-16 → 2026-06-19). Planning cycle only; no code. Requires Cycle 9 zone-console tick data and frame budget before it can begin.

CRIS Score

The Downsides

Cycle 12 produces plans, not working software. A week of slippage in Cycle 9 delays the planning cycle by the same amount.

The Road Not Taken

Planning the game cycles now (before Cycle 9) was considered and rejected — estimates made without a running physics scene and real frame budget data have no evidence base.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer
• Game Director

Tags

• maglev-cycle-12, game-planning, vertical-slice, galls-law, 20260506-maglev-cycle-12-game-planning, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}

The Context

With causal ordering confirmed in Cycle 10 and the DB connection verified in Cycle 4, Cycle 11 activates persistence: the persona zone commits the score and multiplayer-fabric-uro writes it to CockroachDB.

The Problem Statement

The cross-zone write path — persona zone commits, uro writes via Ecto over Fly’s 6PN using the gateway_writer role — requires end-to-end validation. With causal ordering confirmed in Cycle 10, the only remaining unknown is whether the database write path performs correctly under mission load.

Design

On each core-slot event the persona zone commits the score update after the VClock.le check passes. On mission end multiplayer-fabric-uro writes the committed score to CockroachDB via Ecto over Fly’s 6PN using the gateway_writer role. RelReplica entries stamped with sentAt are broadcast to the interest band.

Write path:

spatial zone  →  QueueOp(VClock)  →  persona zone commits
              →  uro → CockroachDB (gateway_writer, Fly 6PN)
              →  RelReplica(sentAt) broadcast to interest band

Pass criteria:

• Committed score appears in CockroachDB within one tick of mission end
• RelReplica.stale correctly reflects the write on all nodes in the interest band
• No write occurs from any zone other than the persona authority

Estimate

5 days (2026-06-30 → 2026-07-07). The Ecto write path to CockroachDB is proven in Cycle 4; the work is the persona zone commit flow, VClock gating on the write side, and RelReplica broadcast. No prior end-to-end causal write exists in the history.

CRIS Score

The Downsides

If the persona zone goes down, that player’s attributes are offline until it recovers. Replication factor ≥ 2 on the persona partition is the mitigation, deferred to a later cycle.

The Road Not Taken

Writing scores directly from the spatial zone to CockroachDB was considered. Rejected — it violates DisjointRanges and allows concurrent writes to the same player record.

Status

Status: Draft

Decision Makers

• Lead Architect / Fabric Maintainer

Tags

• maglev-cycle-11, db-write, persona-zone, cockroachdb, galls-law, 20260506-maglev-cycle-11-db-write, present-proposal-template

Further Reading

@misc{v_sekai_2026,
  title = {V-Sekai},
  year  = {2026},
  url   = {https://v-sekai.org/}
}