Return-to-root roadmap — landing the reshaped kernel as THE kernel

Status: LIVING migration state (reconciled 2026-08-19). Return-to-root has landed: the reshaped core is the repository root, and the pre-reshape tree is archived read-only under _legacy/. This document tracks what has re-attached and what has not yet been ported forward — it is the companion to docs/requirements.md (the north star) and the two accepted decisions docs/decisions/0001 / 0002.

1. Why this document

The reshape was built in isolation in sketches/kernel-reshape/ deliberately — a clean rebuild of the kernel core (arch / boards / kernel / make / profiles / examples) whose thesis is discharged: portable across two switch models (deferred-PendSV + trap-frame), privilege-parametric (M/S mode), SMP on two arches, and freedom-from-interference proven on three mechanisms (ARM MPU, RISC-V PMP, RISC-V Sv32 paging).

That sketch has now become the root. The migration is not more kernel design — it is a re-attachment effort: the reshaped core is promoted to <root>/kernel + <root>/arch, and the reuse layers (HAL, devices, drivers, tools, tests) are being reconnected on top, forward- ported against the reshaped API rather than moved as-is. This is integration work, and the real finish line.

The reuse layers listed below carry file/LOC counts from the _legacy/ tree — they measure the size of the port-forward job, not what is at root today (most of these are not yet at root; see §5):

_legacy layer	Files / LOC	Nature	Verdict	At root?
hal/	40 / 2701	MCAL + HAL (Option-C resolver, instance model)	Reuse (composability I1)	Partial — a slim hal/ (9 files: board/console/dio/mcu/port/uart) landed; crypto/wdg/error/compiler headers not ported
tools/	13 / 9174	tkgen pinmux, composability matrix, loaders	Reuse — kernel-agnostic	No — root tools/ has only SEGGER/ + bin2uf2.py
tests/	19 / 1670	host unit tests	Reuse + extend	No — no tests/ dir at root
devices/	12 / 1122	PN532, sensors, device drivers	Reuse — sit on HAL	No — no devices/ dir at root
drivers/	6 / 391	board-agnostic tk_button/tk_led/gpiote	Reuse — one kernel touchpoint	No — no drivers/ dir at root (ISR→wake shown via examples/button_irq)
system/	5 / 370	trust-chain / heartbeat / secure-boot glue	Reuse	Partial — only system/tk_sysview.c landed
boards/	203 / 13613	every board port	Selective	Partial — 4 reshaped boards landed
kernel/	18 / 4349	the pre-reshape kernel	Superseded by the reshaped core	Superseded — reshaped kernel/ is at root
arch/	19 / 4298	pre-reshape arch ports	Superseded where a reshaped port exists	Superseded — reshaped arch/ is at root

2. The load-bearing risk: API drift

The reuse layers were written against the pre-reshape kernel API. The reshaped kernel deliberately has a different, tighter surface (the [rt]/[setup] boundary settled in ADR 0002) and is missing some features the pre-reshape kernel exposed. Every reuse layer crosses one of these seams:

• Sync/thread API shape — drivers/devices that block, wake from ISR, or use sems/mutexes call kernel runtime APIs. Names/semantics mostly align (both tk_-prefixed, pthread-shaped), but each call site must be audited, not assumed.

• Features present in _legacy/ but NOT yet in the reshaped core — must be ported forward before the layers that need them can re-attach. Landing state as of this reconciliation:

  • tk_introspect.{c,h} — remote-monitor / RO introspection region — NOT ported (a lighter tk_kernel_thread_snapshot + struct tk_thread_info exist under TK_CONFIG_ENABLE_INTROSPECT, but not the seqlock region).

  • tk_trace.h + tk_trace_ring.c + tk_sysview.h + system/tk_sysview.c — flight-recorder + SEGGER SystemView — ✅ ported (R2 half done).

  • tk_time.c — the reshape folds the kernel-owned tick inline; root has only kernel/tk_time.h (no .c). Parity confirmed (inlined), not a gap.

  • tk_stack.c — root has kernel/tk_stack.h as a placeholder no-op (tk_stack_prepare/ tk_stack_check are stubs; fill + canary + high-water “land here when enabled”). The L1 software-canary implementation + the tk_thread_stack_peak/free + overflow-hook API are not yet re-hosted — reconcile against the arch canary.

• HAL↔kernel independence must stay intact — composability invariants I1/I2/I3 (HAL never depends on kernel). The enforcing tool _legacy/tools/composability_matrix.sh is not yet at root; re-porting it (R4) is what restores the CI gate — until then the invariant is honored by convention, not enforced automatically.

3. Phased sequence & status

Each phase is independently landable and leaves the tree building + tests green.

• R0 — Freeze & diff. ✅ Done (implicit in the promotion). The reshaped core was snapshotted and promoted; the pre-reshape tree is frozen under _legacy/.

• R1 — Promote the core. ✅ Done. Reshaped kernel/ + arch/ + make/ are the root build. Host, nRF52 (arm_v7m), and RV32-QEMU (risc_v) build; the reshaped examples build from their root home. The make system wires arm_v6m/arm_v7m/arm_v8m/pthread/risc_v and boards host/nrf52832_pca10040/qemu_riscv32_virt(+_sbi)/rp2040_pico(+_flash)/mps2_an505.

• R2 — Port forward the missing kernel features. 🟡 Half done. Trace / trace-ring / SystemView are landed (kernel/tk_trace.h, kernel/tk_trace_ring.c, kernel/tk_sysview.h, system/tk_sysview.c; demos examples/{trace,sysview}). Not done: tk_introspect

  • examples/monitor; the L1-canary implementation behind tk_stack.h. tk_time.c reconciled (inlined).

• R3 — Re-attach HAL + devices + drivers. 🔴 Not started. A slim hal/ is present, but devices/ and drivers/ do not exist at root. Re-point them at the reshaped runtime API, fix the drift (expected: a handful of renames + the ISR→wake path), then re-assert the composability matrix gate.

• R4 — Retarget tests + tools. 🔴 Not started. tests/ does not exist at root; tools/ has only SEGGER/ + bin2uf2.py — tkgen (pinmux) and composability_matrix.sh are not ported. Repoint host unit tests + tkgen/matrix at the promoted core.

• R5 — Board sweep. 🟡 Partial. Four reshaped boards landed (host, nrf52832_pca10040, qemu_riscv32_virt, rp2040_pico). Reshaped ports drop in directly; HAL-only boards (MSP430, FR2433, STM32H723) re-attach via R3’s HAL. RA4M1, TC275, STM32, ESP32, AVR boards are not yet at root. Retire the (already archived) pre-reshape kernel/+arch/ once every live board is on the new core. Do this board-by-board, not big-bang.

Boundary — what is explicitly NOT in this plan (so scope stays honest): new arch ports. The remaining reshape arches are mostly confirmatory and should be done lazily, pulled by a board an integration target actually needs, reusing the HW-verified _legacy/ code rather than re-proving the pattern:

• RISC-V S-standalone / SMP+U-mode — confirmatory (S-mode, U-mode, SMP each already proven; both the M-mode and S-mode/SBI QEMU targets are already at root).

• C2000 / TriCore / MSP430 full reshaped ports — AXIS + CSA + small-RAM already validated. Ahead of the plan: arch/c2000/ and arch/tricore/ already landed as arch directories (C2000 core compiles under cl2000; TriCore has switch + SMP + lazy-CSA) — but neither is make-wired and neither has an in-root board yet. Finish them at R5 when their board is next.

• Cortex-M33 / ARMv8-M + TrustZone — the one genuinely-NEW capability (secure/non-secure world isolation). arch/arm_v8m + the mps2-an505 QEMU target are already at root (QEMU-proven); the TrustZone world dimension and HW bring-up (FRDM-MCXN947) remain the deliberate slot.

4. Rough total & the honest caveat

Order-of-magnitude: R0–R2 are largely behind us; R3–R4 ≈ 1 focused week; R5 is open-ended (203 _legacy board files, done incrementally and only for boards that stay alive). The dominant cost is not writing code — it is the drift audit and keeping the composability gate green so the HAL/kernel independence the reshape earned is not lost on the way back in. The highest-value next slice that still delivers a coherent result is R2 (finish introspect) → R3 on one reference board (nRF52), proving the reshaped core hosts the full reuse stack on real silicon.

5. Not yet ported to root (the standing carry-forward list)

The concrete backlog, reconciled against the root tree on 2026-08-19. Each item is absent at root and present (proven) in _legacy/ unless noted.

Kernel features / observability

• tk_introspect.{c,h} + examples/monitor — the seqlock-guarded RO ps/top region and its unprivileged reader. (The TK_CONFIG_ENABLE_INTROSPECT knob and tk_kernel_thread_snapshot exist; the region + monitor do not.) → R2.

• L1 software-canary implementation — kernel/tk_stack.h is a no-op placeholder; the fill / canary / high-water scan and tk_thread_stack_peak/free + overflow-hook API are not re-hosted. Note the placeholder’s stale comment says “default ENABLE_STACK_CHECK=0” while tk_config.h defaults it to 1 — reconcile when the canary lands. → R2.

• Error / DET subsystem — kernel/tk_det.h is a no-op stub (TK_DET(...) → ((void)0)); the tk_error.* counters/hooks/28-module-ID machinery is _legacy/hal/ only. → R3/R4.

• tk_time_us() carries only millisecond resolution — it ships as a flat `tk_kernel_now_ms()

  • 1000rather than reading a sub-tick source, so even the host's nanosecondCLOCK_MONOTONIC` is truncated to ms before it. A real per-board sub-tick surface is a later add. → R2.

Tooling (tools/)

• tkgen pinmux registry (_legacy/tools/tkgen/pingen.py) — YAML pin registry → validated const C tables. → R4.

• composability_matrix.sh — the HAL-never-depends-on-kernel CI gate. Not at root. See the review carry-forward below. → R4.

Reuse layers (whole directories absent at root)

• tests/ — 19 host unit test files; retarget at the promoted core. → R4.

• devices/ — PN532 + sensors (sit on HAL, not kernel). → R3.

• drivers/ — tk_button / tk_led / gpiote (one kernel touchpoint: ISR→wake). → R3.

Unported subsystems (safety/security — staged in docs/safety-and-security.md)

• Crypto + verified boot — SHA-256 / HMAC / AES-CBC / P-256 ECDSA, tk_vboot trust chain + signer pipeline. No crypto/vboot headers at root.

• Watchdog — tk_wdg (mode-based HW watchdog). No hal/tk_wdg.h at root.

• Cross-core heartbeat — system/tk_heartbeat + TK_CONFIG_ENABLE_HEARTBEAT. Not at root.

• Reset-reason / brownout — tk_mcu_reset_reason / BOD warning backends. Not at root.

• Persistence — the .noinit-survives-reset story: the demo examples/noinit is at root (with its own linker script), but any broader persistence subsystem is not tracked here.

API-boundary annotations & type hygiene

• [rt] / [setup] header annotations — ADR 0002 §4 fixes the runtime-vs-setup API split; the annotations ARE present in kernel/tk_kernel.h (done). ⚠ Open: [rt]-tagged tk_kernel_self/tk_kernel_name read privileged data → a fault under TK_CONFIG_UNPRIV_THREADS (see docs/reviews/amp-smp-review-0001.md L2).

• Residual uint8_t leak — kernel/tk_kernel.h’s public struct tk_thread_info still declares uint8_t base_priority / uint8_t state — the kind-1-as-uint8_t anti-pattern ADR 0001 (Addendum 2) forbids. Fix to unsigned char when the introspection surface is next touched.

Isolation — implemented in arch/+kernel/ but not wired at root

• No root board provides the MPU zone symbols. The arm_v7m (and arm_v8m/risc_v) isolation path references board-supplied linker symbols — tk_mpu_flash_base/_size, tk_mpu_ram_base/_size, plus the .privileged_bss / .thread_stacks zones — and no root board defines them (verified: arch/arm_v7m/kernel_arch.c:295 externs them; nothing in boards/ supplies them). So a build with TK_CONFIG_ENABLE_MPU_ISOLATION or TK_CONFIG_UNPRIV_THREADS=1 will not link on any root board today — the machinery is present, the board wiring is not. Wire nRF52 first (its arm_v7m MPU path is the most complete). → R3/R5.

• arch_stack_isolatable() has no caller. The fail-closed guard — reject/DET an unprivileged thread whose stack can’t be MPU-isolated — is implemented on every ARCH_HAS_MPU arch (arm_v7m/arm_v8m/risc_v) but kernel/tk_kernel.c never calls it (verified). Add the check where unprivileged threads are admitted; until then the guard the isolation design assumes is absent. → R2/R3.

Review carry-forwards (re-verify; do not reopen the design)

• SEC-4 — hard-coded MPU sub-region mask. A prior review flagged a hard-coded sub-region disable mask in the MPU region programming. Re-verify it against the reshaped arch/arm_v7m / arch/arm_v8m MPU code; confirm it is still correct (or parameterized) — this is a re-verification, not a reopened design question.

• HAL-only CI build. The follow-up to prove HAL builds standalone (no kernel) was expected to be satisfied by composability_matrix.sh. Correction: that tool is not yet at root (see Tooling above), so the automated gate is not currently satisfied here — the design is settled (the _legacy matrix), the re-port is the open action (R4). Confirm the invariant still holds; do not reopen the design.