ADR 0003 — The supervisor-fence seam: a per-instance HW memory boundary that binds the kernel¶

Status: Accepted (2026-08-22). Builds on ADR 0002 (multi-instance kernels / AMP islands) and ADR 0001 (the “line by axis” arch-generalization method).

Context¶

ADR 0002 generalized the singleton into N kernel instances resolved per physical core — SMP-subset and disjoint AMP islands, chosen by an owned_cores bitmask. Islands “share nothing,” but below the THREAD isolation level that sharing boundary held only by convention: nothing stopped island A’s kernel (privileged code) from storing into island B’s RAM. The per-thread MPU work (review Bucket C) fenced unprivileged threads from each other, but the privileged level — the kernel itself — ran with a full-access background on every arch. For a real freedom-from-interference story between independent AMP islands (one island faulting or going rogue must not corrupt another’s memory), the boundary has to bind the privileged level too.

C3-B is that boundary: a hardware memory fence, applied per core as it enters its instance, under which even a privileged kernel store into a neighbour island’s RAM faults. The open question was whether this is a portable mechanism or an arm_v8m trick — the arch generalization axis (ADR 0001). It is portable: the fence was proven, brick by brick, on five mechanisms spanning the two fundamental protection classes (§3), which is what lets us commit to a single arch seam rather than a per-arch special case.

Decision¶

1. One seam: a mechanism-neutral deny-list¶

A fence-capable port declares ARCH_HAS_SUPERVISOR_FENCE 1 and implements exactly one function:

int arch_instance_fence_apply(const uint32_t* deny_base, const uint32_t* deny_size, uint32_t ndeny);

Called once per core from tk_kernel_start_this_core, before the core schedules any thread, with the RAM arenas of every other registered instance (the neighbours). The port programs this core’s protection unit so a privileged store into any denied arena faults. The contract is mechanism-neutral — a deny-list of neighbour RAM ranges — because that is the one shape both protection classes can honour (§3). Addresses are uint32_t (the current 32-bit target set); widen to uintptr_t when a 64-bit port lands.

3. The generalization: two protection classes, proven on five mechanisms¶

The fence reduces to one of two hardware primitives, and both were proven to satisfy the same deny-list seam:

Class	Mechanism	Arch / port	Privilege level	Proof
Paging	RISC-V Sv32 (satp swap)	`risc_v` (S-mode)	S-mode	QEMU (in-tree; `fence_riscv`)
Paging	ARM VMSA short-descriptor (TTBR0)	Cortex-A15	PL1	QEMU (`sketches/arm_a`)
Region	ARM v8-M PMSAv8 (drop `MPU_CTRL.PRIVDEFENA`, base/limit)	`arm_v8m`	Secure priv	silicon (FRDM-MCXN947)
Region	ARM v7-M PMSAv7 (drop `PRIVDEFENA`, pow2/SRD)	`arm_v7m`	priv	silicon (nRF52832)
Region	ARM-R PMSAv8-R (no privileged background)	Cortex-R52	EL2	QEMU (`sketches/arm_r`)
Region	AURIX TriCore PRS/DPR (native bind)	`tricore`	Supervisor	silicon (in-tree; `amp_kernel_fence_tricore`)

Two paging examples and four protection-region examples (PMSAv8 base/limit, PMSAv7 pow2/SRD, PMSAv8-R, TriCore PRS/DPR), across five ISAs and four exception models (RISC-V traps, M-profile auto-stacking, classic AArch32 banked modes at EL2 and PL1, TriCore CSA). The two classes differ in how they honour the deny-list — a region arch holes the neighbour ranges out and drops its privileged-background grant (ARM v8-M PRIVDEFENA; ARM-R and AURIX have no privileged background to begin with); a paging arch maps everything except those ranges — but the kernel-facing contract is identical. That two-way evidence is the basis for committing to the single seam in §1 rather than an arch-specific API. In-tree, arm_v8m + arm_v7m (silicon), risc_v (S-mode Sv32, QEMU) and tricore (PRS/DPR, build-verified) all implement the seam — every proven mechanism except the two ARM-R/ARM-A sketches (§Open).

4. Capability + degrade ladder — each level asserts its OWN capability¶

The isolation contract is an ordered ladder an app can compare and query (tk_kernel_isolation() at runtime, TK_ISOLATION_LEVEL at compile time):

TK_ISOLATION_NONE (0)  — no HW boundary: one privilege level, shared RAM
TK_ISOLATION_THREAD (1) — HW boundary between unprivileged threads + kernel RAM
TK_ISOLATION_INSTANCE (2) — per-island fence binding the KERNEL too (this ADR)

Crucially, each level requires its OWN arch capability — not one blanket “isolation capable.” THREAD needs the unprivileged/MPU split (TK_ARCH_ISOLATION_CAPABLE = ARCH_HAS_MPU && ARCH_HAS_UNPRIV). INSTANCE needs the privileged-binding fence (TK_ARCH_SUPERVISOR_FENCE_CAPABLE = ARCH_HAS_SUPERVISOR_FENCE) and does not require an unprivileged split at all — an all-privileged AMP island can still want its kernel fenced from a neighbour. Coupling INSTANCE to the THREAD capability (as an earlier blanket invariant did — it held only because arm_v8m happens to declare all three) would wrongly reject a future fence-capable arch that lacks ARCH_HAS_UNPRIV. The two levels now assert independently (tk_config.h), which is what keeps the seam genuinely general.

5. Honest degrade — fail LOUD at compile time, fail CLOSED at runtime¶

The boundary is a safety claim, so a request it cannot meet must never silently drop to a weaker one:

Compile time. TK_CONFIG_ENABLE_INSTANCE_ISOLATION on an arch without ARCH_HAS_SUPERVISOR_FENCE, or with TK_CONFIG_MAX_KERNELS < 2 (a fence acts between instances), is a hard #error — the structurally-incapable arches (C2000, MSP430, arm_v6m, the host) are turned away at the door.
Runtime. If a capable arch cannot install the fence on a particular core — arch_instance_fence_apply returns 0 (e.g. a TrustZone Non-secure core whose NS MPU reports zero regions, or more neighbour arenas than the region budget) — the kernel calls the weak tk_instance_fence_reject(core), whose default parks the core (fail-closed). A core that asked for the fence but cannot enforce it must not run island code pretending it is fenced. An app may override the hook to log-and-continue, reassign the core, or recover.

6. Collapse safety (non-negotiable, per ADR 0002 §3)¶

The feature is present-but-dormant until opted into. No arena set → no neighbour to deny → the fence is never applied. TK_CONFIG_MAX_KERNELS == 1, or the feature off, compiles the whole path out: tk_kernel_set_ram becomes a TK_ERR_UNSUPPORTED stub, struct tk_kernel gains no fields, and a single-instance build is byte-identical to one built without C3-B.

7. Regression fence¶

tools/check_fence_matrix.sh (twin of check_arch_detect.sh) guards both halves: the degrade logic (capable+INSTANCE+≥2 builds; incapable, or <2 instances, fails loud; default collapses) portably with a host compiler, and the per-arch capability matrix (arm_v8m must declare the flag; the structurally-incapable arches must not).

Consequences¶

The islands-share-nothing invariant of ADR 0002 becomes hardware-enforced at the top rung, not convention — the missing half of the AMP freedom-from-interference story.
A new fence-capable arch plugs in by flipping one flag + writing one function, exactly like every other Tier-3 capability (arch-interface.md); the kernel, config ladder, and degrade matrix need no edit.
INSTANCE isolation is decoupled from unprivileged threads — the level-specific invariant (§4) means a fence-capable arch without an unpriv split is a first-class citizen.
The seam was widened to uintptr_t (2026-08-24, e4c95e2, once arm_v8a landed) — a mechanical, isolated change across all 7 fence arches; set_ram no longer truncates a >4 GiB arena, and 32-bit ports stay byte-identical (uintptr_t == uint32_t).

Open / deferred¶

arm_v7m wired + HW-verified (nRF52832 / Cortex-M4, examples/fence_arm_v7m): a self-contained TK_FEAT_INSTANCE_ISOLATION block that grants flash/SRAM/peripherals and denies each neighbour arena with a higher-priority no-access PMSAv7 region, then drops PRIVDEFENA. On silicon a privileged store into the fenced arena faulted (MemManage CFSR=0x82, MMFAR = the arena). Constraint: a neighbour arena must be power-of-two sized + naturally aligned (one PMSAv7 region); else arch_instance_fence_apply returns 0 → fail-closed. Validated INSTANCE-only (combining with per-thread lockdown, which reprograms regions per switch, is future work).
risc_v S-mode wired + QEMU-verified (examples/fence_riscv): the in-tree risc_v already carries an S-mode variant (TK_RISCV_SMODE, kernel in supervisor mode under an SBI shim), so the fence is a self-contained Sv32 page-table + satp-swap block gated on it — identity-map all physical space S-RWX, refine each megapage holding a neighbour arena to 4 KiB pages with the arena left INVALID, load satp. An M-mode build declares no capability (M-mode bypasses PMP) and fails loud. Constraint: a neighbour arena must be 4 KiB aligned + a 4 KiB multiple (Sv32 page granularity).
tricore wired + HW-verified (TC275 Lite Kit, 2026-09-02): the full arch was promoted from _legacy to root (arch/tricore/kernel_arch.{c,h} + switch.S — the return-to-root migration), and arch_instance_fence_apply added (PRS 0 permissive, PRS 1 grants all data except neighbour arenas via bracketing DPR ranges, SYSCON.PROTEN; recipe from sketches/amp/tc275/amp_mpu.c). TriCore ranges are arbitrary [lo,hi) — no alignment/pow2 constraint (unlike ARM/RISC-V). Binding the KERNEL: an island’s contexts run in PRS 1 (per-context PSW.PRS=1, baked in arch_context_init), so a scheduled island thread — and the kernel code it calls by direct Supervisor call — is bound; only trap/ISR entry drops to the permissive PRS 0. ✅ examples/fence_tricore (single-core: tree arch_instance_fence_apply + PRS 1 + class-1 MPW trap) and examples/amp_kernel_fence_tricore (SMP=2: a scheduled island-A thread storing into island B’s arena took a class-1 MPW trap, core 0 halted, core 1 kept scheduling — fence bound the kernel + fault isolation held). ARM-R / ARM-A are QEMU-proven sketches only.
Secondary-core limitation (HW-confirmed on FRDM-MCXN947, 2026-08-24): the per-core fence works only on a core that HAS a privilege-binding MPU. On this asymmetric dual-M33 the primary CPU0 has an 8-region MPU (fence works), but the secondary CPU1 is a reduced Cortex-M33 with NO MPU (and no TrustZone/SAU — ID_PFR1[7:4]=0, MPU_TYPE=0, SAU_TYPE=0 read on-chip). So “run core1 Secure” is impossible (no security extension), and there is no per-core MPU to program: arch_instance_fence_apply correctly fails LOUD (§5, tk_instance_fence_reject). Fencing CPU1 requires a board-specific, system-side mechanism (the AHB Secure Controller — a bus-master-ownership class). That was attempted on HW and found not effective (the AHBSC per-region rules are runtime-writable, but enabling the global secure-checker resets the chip — a vendor register-write protocol the device header doesn’t document); the attempt is documented and its implementation deliberately kept out of the kernel/arch tree. Detail + register reference: examples/frdm_mcxn947/README.md “E1 — AHBSC TrustZone-AMP fence”. RESOLVED in principle (2026-08-25): the system-side fence is HW-PROVEN and designed in ADR 0005 — a third (bus-master-ownership) fence class for a unit-less secondary core; the kernel graft is a board-gated follow-on.
Ownership-based fences (dual-core C2000 F2837xD master/slave RAM protection) are a third class — not privilege-based — deferred until the board is on the bench; the seam may need to express “owner core” as well as “deny range” then.
~~64-bit widening of the seam signature when an AArch64 / RV64 port lands.~~ DONE 2026-08-24 (e4c95e2, Epic B) — arm_v8a triggered it; deny_base/size + fence_base/size are now uintptr_t.