Skip to content
KN-86 Deckline

ADR-0048: Clip cutscene format v2 — the playback surface (128×75 grid + Lisp FFI)

cart-facing FFI ergonomics (clip-play / on-done / clip-playing?) are the part most in need of review before this is accepted.

Sibling of: ADR-0046 + ADR-0047 — the foreground-surface split. ADR-0046 took the data tier (modes), ADR-0047 the pixel-rendered tiers (tactical / experience). This ADR adds the playback surface — the cutscene — and brings the already-built clip system formally under that framework. Like ADR-0047, the siblinghood is in placement, not construction.

Related / composes: ADR-0036 (the single pixel surface, the system-tier blit, the 20 fps cap), ADR-0027 (the constrained cart cell-API + the privilege boundary), ADR-0040 (screen router — gains the typed cutscene kind), ADR-0043 (objectives — consume the play→pop outcome), ADR-0045 (the background hunt keeps ticking under a cutscene), ADR-0006 (cart format — clips ship as bundled assets), ADR-0005 (NoshAPI — gains the clip-* verbs; this ADR amends it), ADR-0001 (carts are Lisp). Scope: runtime (runtime/), cart FFI, sdk/, carts/, hosts/device/, and the kn86-attract Remotion pipeline (umbrella). Half-block fidelity is deferred to a v2.1 follow-on.

Context

Section titled “Context”

We want cartridges to play pre-rendered terminal cutscenes — short authored animations that stand in for gameplay beats too expensive or impossible to render live (a deep hack sequence, a sonar prosecution, a cinematic failure). On this device, “video in a cell” means terminal animation, not codec video: the ceiling is monochrome amber on the canonical 128×75 grid of integer-scaled 8×8 glyphs (plus the 128×146 half-block sub-pixel layer), no RGB, no raw framebuffer for carts (Canonical Hardware Specification; ADR-0036; ADR-0027; ADR-0005).

Most of the machinery already exists:

  • The clip system is built and tested — and predates the ADR-0046/0047 foreground-surface framework. runtime/src/clip.c + nosh_clip.c decode a .kn86clip binary: a tiny (4–8 KB), zero-malloc, region-aware frame animation (opcodes HOLD / DELTA / KEYFRAME / CLEAR / CUE / END) that paints into a rectangular region and composes over live chrome. Consumed today by attract mode (attract.c) and C reference carts; covered by tests/test_clip.c / test_attract.c.
  • The authoring pipeline already emits the format. The kn86-attract Remotion tooling (tools/export-demo.ts → simulator.ts → encoder.ts) simulates a scene to grid frames and encodes the same .kn86clip the runtime plays. Marketing video and on-device clips come off one pipeline.

Where a cutscene sits in the foreground-surface framework

Section titled “Where a cutscene sits in the foreground-surface framework”

ADR-0047 framed foreground surfaces on two axes — address (cell vs pixel) × interaction (event/command vs active-loop/control) — and named three kinds: mode (cell+event, ADR-0046), tactical (pixel+event), experience (pixel+active-loop). A cutscene fits none of them: it is non-interactive. It advances on a clock and the only input is cancel.

So a cutscene introduces a third value on the interaction axis — timeline/playback — orthogonal to the two ADR-0047 enumerated (it is not the “cell+active-loop” quadrant ADR-0047 called unused; it is a different interaction mode entirely). Crossed with the address axis:

  • cell-addressed + timeline = the cutscene — this ADR, v2.0.
  • pixel-addressed + timeline = a pixel cutscene — v2.1 half-block, riding ADR-0047’s system-tier render_bitmap 1-bit-mask blit path.

Three gaps block the in-gameplay use case:

  1. Grid mismatch. Clips are authored at the legacy 80×25 grid; the canonical surface is 128×75 (ADR-0036/0027). The player paints into the legacy text_buffer, not the 128×75 cell grid.
  2. No Lisp playback path. Clips can only be driven by firmware (attract) or C cart code. Launch carts are Lisp (ADR-0001), and NoshAPI exposes no clip primitive — a Lisp cart cannot play a cutscene mid-mission and resume.
  3. Device playback unproven. Playback is emulator-proven; the Pi (SDL3/KMSDRM on the Pi) path is not yet validated.

Decision

Section titled “Decision”

Ship clip format v2 and a playback FFI, at glyph-cell fidelity (128×75, one 8×8 Code Page glyph per cell, monochrome), governed by the ADR-0047 surface contracts.

  1. The cutscene is a foreground surface kind. It is cell-addressed + timeline/playback (non-interactive, cancel-only). It obeys the same foundation as ADR-0047’s pixel tiers: system-owned rendering, invoke-only for carts (D2); composites inside Rows 1–73 with chrome persisting (D4, Spec Hygiene Rule 5); a typed surface kind in the screen router (D5); push → play → pop lifecycle (D6 — degenerate: a cutscene’s outcome is trivial, completed vs cancelled, reusing the ADR-0006/0043 path rather than extending it); and the 20 fps redraw ceiling (ADR-0036). The clip player is the system-tier renderer; the cart never touches pixels — it invokes a named clip through constrained verbs, exactly the ADR-0047 D2 posture (the built clip subsystem already worked this way; this ADR makes it formal).

  2. Grid → 128×75. Bump the .kn86clip version byte to 0x02. Region header fields stay u8 (128/75 fit). Reserve a plane/flags byte so v2.1 can add the half-block plane without another format break. The full-grid fallback and the player’s paint target move from the 80×25 text_buffer to the 128×75 cell grid, every write clamped to the declared region ∩ Rows 1–73. Regenerate all clips from their Remotion sources and drop the v1 read path — one format, one decoder.

  3. Cart-tier Lisp FFI (amends ADR-0005). Add an all-carts (Tier 1) clip-playback surface — runtime-ticked, fire-and-forget with completion — because the use case is “play a cutscene, then resume,” and runtime ticking decouples the clip’s frame clock from the cart’s event-driven redraw. These are constrained invoke verbs (ADR-0047 D2), not raw pixel access:

    (clip-play "depthcharge/active-ping"        ; named asset bundled in the .kn86
               :region '(1 0 73 128)             ; optional; default = content area, rows 1–73
               :loop   #f                        ; optional
               :on-done (fn () (resume-mission))); optional completion callback
    ;; -> clip handle
    

    (clip-playing? h)   ; poll alternative to :on-done
    (clip-stop h)       ; abort -> snap to final frame, fire on-done

    The runtime owns the frame clock at the clip’s declared fps (capped at the 20 fps ceiling), paints each frame into the cell grid, clamps to region ∩ Rows 1–73, and auto-fires firmware SFX on CUE opcodes. Cart-defined cues (0x80–0xFF → cart callback) and cart-driven manual ticking are out of scope (→ v2.1).

  4. Clips are bundled cart assets (sdk/). kn86cart build bundles *.kn86clip files into the .kn86 container as named static assets (manifest entry); the runtime loads them zero-copy (nosh_clip_load_buffer) from the mounted cart and Lisp references them by name (cart format per ADR-0006).

  5. Remotion re-flow (kn86-attract, umbrella). Move the pipeline (constants, simulator.ts, TerminalGrid, encoder.ts) from 80×25 to 128×75 and emit v2. Re-encode the scene library and reconcile the roster to the five launch titles (snake, icebreaker, depthcharge, blackledger, neongrid) — the library currently carries non-launch carts and lacks snake.

  6. Device port (hosts/device/). Validate clip → 128×75 cell grid → the SDL3/KMSDRM presentation path and the frame clock on the Pi Zero 2 W.

Options considered

Section titled “Options considered”

Placement — fold into ADR-0047 vs. a sibling ADR. Chosen: a sibling ADR under the same framework. The clip system is a pre-existing, built subsystem with its own binary format and version-migration concerns, and the cutscene interaction mode (timeline/playback) is one ADR-0047 did not enumerate — it warrants its own record, governed by ADR-0047’s contracts. Folding a format spec into the (interactive) pixel-tier ADR would blur both.

Trigger / scope. (a) a Lisp FFI over the existing 80×25 clips; (b) firmware-driven cutscenes only (load / mission-boundary / attract — works today, no cart control); (c) modernize to 128×75 + FFI (chosen). (c) is the future-proof path: cutscenes match the canonical grid and carts fire them at arbitrary beats.

Fidelity substrate. (a) glyph-cell now, half-block as v2.1 (chosen); (b) glyph + half-block together; (c) half-block-centric. Glyph-cell is the device’s native idiom (box-draw, type-on, dissolves, shaded sweeps), keeps the decoder tiny, and reuses the encoder and scenes nearly verbatim; half-block (finer “video-like” motion) is a clean additive follow-on behind the reserved plane byte — and lands on ADR-0047’s already-chartered render_bitmap pixel path. A side-by-side render of both substrates informed this.

FFI ergonomics. (a) cart-ticked (mirror the C nosh_clip_tick API — clip-load / clip-tick / clip-finished? in the cart loop); (b) runtime-ticked fire-and-forget + completion (chosen). (b) matches “play then resume,” decouples clip fps from cart redraw, and keeps carts on invoke verbs rather than a draw loop (ADR-0047 D2); (a) would couple animation rate to the cart’s render cadence.

Format versioning. Version-byte bump + regenerate-all + drop v1 (chosen) vs. dual v1/v2 read paths. All clips derive from Remotion sources, so regeneration is free and a single decoder path is simpler.

Consequences

Section titled “Consequences”

Positive. Reuses a built, tested substrate; clips stay tiny (4–8 KB); one Remotion pipeline feeds both marketing video and on-device cutscenes; carts gain cinematic beats they can’t render live; the cutscene slots cleanly into the ADR-0046/0047 framework (system-owned render, invoke-only, chrome-persisting, router-typed, objective-feeding); the reserved plane byte makes the v2.1 half-block plane purely additive and aligned with ADR-0047’s pixel blit.

Costs / risks. A format break (re-encode every clip). The player’s paint-target migration (text_buffer → 128×75 cell grid) touches clip.c and its tests. The Remotion re-flow lives in the umbrella repo — cross-repo coordination, separate PR. The router gains one more typed surface kind (cutscene). Device frame-clock behavior needs bench validation on the Pi.

Documentation Updates (REQUIRED — Spec Hygiene Rule 3)

Section titled “Documentation Updates (REQUIRED — Spec Hygiene Rule 3)”

This ADR does not touch the Canonical Hardware Specification, so no umbrella companion PR is required (cf. hardware-spec ADRs).

Action items

Section titled “Action items”

Decompose as GWP-600 sub-tasks once this ADR is approved:

  1. (docs) This ADR + clip-format.md / clip-system.md updates + the ADR-0005 amendment + the ADR-0040 router note.
  2. (runtime) Clip player: paint target text_buffer → 128×75 cell grid; v2 decode; region clamp to Rows 1–73; 20 fps clamp; extend test_clip.c.
  3. (runtime / FFI) Lisp clip-play / clip-stop / clip-playing? + completion; router cutscene surface kind (ADR-0040); new test_clip_ffi (load → play → tick → on-done, region clamp rejects Rows 0/74, stop semantics).
  4. (sdk) Bundle .kn86clip assets into .kn86 + reference-by-name; round-trip build / inspect / verify.
  5. (umbrella / kn86-attract) Re-flow the pipeline 80×25 → 128×75 + v2 encoder.
  6. (umbrella / kn86-attract) Re-encode the scene library to v2 + reconcile the roster to the launch five.
  7. (hosts/device) Validate clip → cell grid → SDL3/KMSDRM + frame clock on the Pi.
  8. (carts + runtime) Reference integration: depthcharge plays the sonar clip mid-mission as a cutscene surface; QA end-to-end (emulator → device).

Critical path: 1 → (2, 3, 4) → 8; 5–6 run in parallel; 7 validates on silicon. Implementation follows TDD + git-flow (draft PRs → Review).