Per-thread stack isolation (full-lockdown, part 2 of 2)

Status: shipped mechanism, arch-generalized. Design record, not a phasing plan. Second half of full lockdown — read kernel-ram-isolation.md §1 first for the two-knob split (TK_CONFIG_UNPRIV_THREADS vs TK_CONFIG_ENABLE_MPU_ISOLATION); this doc doesn’t repeat it. Companion: kernel-owned-sync.md.

Goal: an unprivileged thread must not be able to read or write another thread’s stack — call frames, saved registers, locals. Confinement alone (TK_CONFIG_UNPRIV_THREADS) does not give you this: on the SRD/PMP-denylist arches (§5) the unprivileged map still grants all of RAM, sub-region-carved only for the kernel singleton and MSP, so every thread’s stack is one shared RW grant peers can reach into.

Non-goal: isolating private globals between threads — a thread can still touch another thread’s static data. Stacks hold the live working state; that is the high-value target.

1. The mechanism: a per-thread stack region, reprogrammed every switch

One MPU/PMP slot is dedicated to “the currently-running unprivileged thread’s own stack,” and the kernel reprograms it on every context switch — both the cooperative yield path and the preemptive tick path, unconditionally when TK_FEAT_MPU_GUARD is on:

/* kernel/tk_kernel.c — cooperative switch (:344-350) and preempt-select (:990-996), identical shape */
#if TK_FEAT_MPU_GUARD
arch_stack_guard_set(tcb->stack_mem);
arch_thread_region_set(tcb->stack_mem, (uint32_t)tcb->stack_cells,
                       arch_context_is_unprivileged(&tcb->context));
#endif

arch_thread_region_set() grants exactly [stack_mem, stack_mem + stack_cells) RW to the incoming thread when it is unprivileged, and disables the slot (or a no-op, on v8-M — §5) for a privileged thread, whose stack the background/whitelist already covers appropriately. Two other threads’ stacks sit in the same denied zone as everything else not currently granted — reachable by neither the previous thread (region reprogrammed away) nor a peer (never granted at all).

Overflow for free on the denylist arches: because the region is sized to the exact stack bounds, its lower edge doubles as the overflow limit — a full-descending overflow steps below the granted region and faults at the offending instruction, same class of fault as isolation (TK_STACK_FAULT_PROTECTION) vs. overflow (TK_STACK_FAULT_MPU) is disambiguated by the arch’s fault decode, not by this mechanism.

2. Declaring an isolatable stack — TK_THREAD_STACK

/* kernel/tk_kernel.h:69-75 */
#if TK_FEAT_ISOLATION
#define TK_THREAD_STACK(name, size) \
    static tk_stack_cell_t name[(size)] __attribute__((aligned(size), section(".thread_stacks")))
#else
#define TK_THREAD_STACK(name, size) \
    static tk_stack_cell_t name[(size)]
#endif

size in addressable cells (tk_stack_cell_t, ADR 0001 — kernel/tk_types.h), not bytes; on every arch currently in the tree a cell is a byte, so this is presently the same number either way, but the type keeps the macro honest for a future word-addressed target. Under full lockdown the stack is placed in the .thread_stacks linker zone and aligned to its own size; without it, TK_THREAD_STACK is a plain array — the same call site compiles either way. A stack declared any other way (a bare array, a malloc) lands in the ordinary shared grant and is not isolated — reachable by every peer, exactly as if the feature were off.

3. Worked example: ARMv7-M power-of-two stacks (nRF52832 / PCA10040)

Arch-specific. Skip to §5 for the general contract.

ARMv7-M MPU regions must be power-of-two sized and naturally aligned to cover a stack exactly with one region:

/* arch/arm_v7m/kernel_arch.c:408-426 */
void arch_thread_region_set(const void* stack_mem, uint32_t stack_size, int unprivileged) {
#if TK_FEAT_ISOLATION
    uint32_t base = (uint32_t)(uintptr_t)stack_mem;
    MPU_RNR = STACK_REGION;
    if (unprivileged && stack_size >= 32u && (stack_size & (stack_size-1u)) == 0u &&
        (base & (stack_size-1u)) == 0u) {
        uint32_t log2 = __builtin_ctz(stack_size);
        MPU_RBAR = base; MPU_RASR = MPU_RASR_ENABLE | ((log2-1u)<<1) | MPU_RASR_AP_RW | MPU_RASR_XN;
    } else {
        MPU_RASR = 0u;   /* non-conforming stack: not isolated, degrades to shared grant */
    }
#endif
}

A non-power-of-two or misaligned stack simply doesn’t get isolated — arch_stack_isolatable() (arch/arm_v7m/kernel_arch.c:429-441) reports whether a given (stack_mem, stack_size) pair would be isolatable (power-of-two, aligned, and inside the .thread_stacks zone bounds _sstacks/_estacks), so the kernel can check before trusting isolation — see §6 for whether that check is actually wired up today.

The zone itself is carved the same way as kernel-ram-isolation.md §4’s .privileged_bss: boards/nrf52832_pca10040/nrf52832_pca10040.ld:79-89 reserves an 8 KB-aligned .thread_stacks region between .bss and the MSP reservation, and arch_mpu_init()’s SRD mask denies it in region 1 the same way it denies the kernel-RAM zone (both zones OR into one SRD mask, arch/arm_v7m/kernel_arch.c:371-375).

4. Region budget (ARMv7-M)

After full lockdown, region use on the 8-region ARMv7-M MPU: 0 flash, 1 shared RAM (SRD-carved), 2 peripherals, 3 = per-thread stack (reprogrammed every switch), 7 = the 32-byte overflow guard (kept for privileged threads, which get no isolation region — the background covers all RAM for them, so only the guard band gives them overflow detection). Slots 4–6 are free. This is unchanged from the pre-reshape design and still accurate against arch/arm_v7m/kernel_arch.c’s GUARD_REGION 7u / STACK_REGION 3u defines.

5. The general contract, and multi-arch generalization

Strip the power-of-two/SRD specifics and the contract is: on every switch, grant the incoming unprivileged thread’s own stack and nothing else in the stacks zone; a privileged thread or a non-conforming stack falls back to whatever the background/whitelist already grants it.

Arch

Mechanism

Alignment constraint

Needs the .thread_stacks linker zone?

arm_v7m

MPU region + SRD, reprogrammed per switch (§3)

power-of-two, naturally aligned

yes

arm_v8m

No per-switch region call at all. arch_thread_region_set() is a documented no-op (arch/arm_v8m/kernel_arch.c:498-507) — v8-M’s whitelist region 1 is instead programmed inside the PendSV/SVC restore asm itself, from g_arch_to’s precomputed mpu_rbar/mpu_rlar fields (arch/arm_v8m/kernel_arch.h:62-67), because on v8-M a schedule-time region write can race a syscall return to the current thread. Base/LIMIT regions need no power-of-two size — any 32-byte-aligned extent works (arch_stack_isolatable, kernel_arch.c:517-518)

no — no TK_FEAT_ISOLATION, no zone; any stack anywhere is isolatable as long as it’s 32-byte aligned

risc_v (M-mode)

PMP pmp0, NAPOT, reprogrammed per switch (arch/risc_v/kernel_arch.c:641-664) — same shape as arm_v7m

power-of-two, naturally aligned (NAPOT encoding)

yes

risc_v (S-mode/SBI)

Sv32 MMU: the whole .thread_stacks zone is U-accessible in one shared page table; no per-thread peer-stack isolation yet — arch_thread_region_set() is a no-op here too, but for the opposite reason from v8-M: the mechanism is deliberately incomplete, not unneeded (comment: “a later step”)

4 KiB page granularity, no power-of-two requirement

yes (zone-level only, not per-thread)

arm_v6m

none (no MPU)

—

n/a

The v8-M and RISC-V S-mode rows both make arch_thread_region_set() a no-op, but for opposite reasons — v8-M because the restore asm already handles it more precisely; RISC-V S-mode because per-thread isolation isn’t built yet there. Don’t conflate “no-op” with “unimplemented” without checking which case a given arch is in.

6. Root status: the guard rail is wired (fail-closed at creation)

The pre-reshape design’s phase “4.3” was a fail-closed guard rail: check every unprivileged thread’s stack with arch_stack_isolatable() and reject a non-conforming one, so it is loud instead of silently degrading to “shared and unisolated.”

arch_stack_isolatable() is implemented on every ARCH_HAS_MPU arch (arm_v7m, arm_v8m, risc_v) and defaulted to “always isolatable” where there’s no MPU (arch/arch_defaults.h:74). Each denylist-arch impl checks the full condition — power-of-two + naturally aligned and inside the .thread_stacks zone (arm_v7m kernel_arch.c:437, risc_v :665); arm_v8m needs only 32-byte alignment (its base/limit region isolates any extent, no zone). It is now wired: create_core (kernel/tk_kernel.c:902, review D2) calls it under #if TK_FEAT_ISOLATION for every created thread and returns NULL + TK_DET(TK_ERR_PARAM) for a non-conforming stack — so a bad stack is rejected at the point of the offending tk_thread_create*/tk_thread_spawn* call, not silently run unisolated. The check is placed at creation rather than a tk_kernel_start() sweep (the original 4.3 site) so the failure names the specific offending call. No TK_CONFIG_UNPRIV_REQUIRE_ISOLATABLE knob is needed — under full lockdown it is unconditionally fail-closed. Proven by examples/isolation_reject (§7 item 3).

7. Validation shape

  1. Inter-thread isolation: an unprivileged thread writes into a peer’s stack buffer → fault, faulting address inside the peer’s stack zone; the same thread writing its own stack works. Proven by examples/isolation (HW, nRF52).

  2. Overflow still caught: on the denylist arches (§5), the per-thread region’s lower edge is the overflow trip; on v8-M, PSPLIM is (see kernel-ram-isolation.md — out of scope here).

  3. Fail-closed on non-conforming input: a mis-aligned / out-of-zone stack is rejected at creation (NULL + TK_ERR_PARAM), not run un-isolated — the loud replacement for the old silent degrade (§6). Proven by examples/isolation_reject (runtime on qemu RISC-V; builds on nRF52) alongside a conforming TK_THREAD_STACK thread that is accepted and runs.