# Porting the tk kernel to a new CPU (`arch/`) **Audience:** anyone adding a CPU port or reviewing an existing one. **Scope:** the contract between the portable kernel (`kernel/`) and a CPU port (`arch//kernel_arch.{h,c}`). Field-by-field reference: `docs/architecture/arch-interface.md`. The byte-width contract this doc leans on throughout is `docs/decisions/0001-foreign-architecture-policy.md` (ADR 0001). The HAL/MCAL contract is separate — `docs/architecture/hal-architecture.md`. Board-level sequencing (where this port slots into a board's bring-up) is `docs/guides/board-bringup.md`. A port is one header + one source file selected by `TK_ARCH` in the board fragment (`make/board/.mk`, via `kernel/tk_arch.h`'s front door). It owns three mandatory things and nothing else: **context switching, the interrupt-state primitive, and the timer tick**. It may additionally opt into SMP, an MPU/stack-guard, unprivileged threads, and context-reclaim/lazy-materialization (§1b) — but it must not know about peripherals, devices, or the application. Existing ports to copy from, by switching mechanism: - **Deferred switch via PendSV** — `arm_v6m`, `arm_v7m`, `arm_v8m` (a board tick calls `tk_kernel_tick_preempt()`; the actual register save/restore happens later, in PendSV). - **Trap-frame switch** (the timer trap saves the full frame itself) — `risc_v` (CLINT → `mtvec`), `c2000` (CPU-timer ISR → `switch.asm`); `tricore` declares this shape too but its `kernel_arch.c` has not been grafted into the tree yet (see §7's status table). - **Cooperative host sim** — `pthread` (a mutex+condvar baton; `ARCH_CONTEXT_SWITCH_SUPPORTED` 1, `ARCH_PREEMPTION_SUPPORTED` 0). - **No switching** — `flat` (run-to-completion, for `host_test`). - **Tier 0 only, no kernel** — `msp430` (declares nothing beyond the IRQ pair; a HAL/MCAL superloop links it and never pulls in the scheduler). --- ## 1. Capability macros (in `kernel_arch.h`) | Macro | Meaning | |-------|---------| | `ARCH_CONTEXT_SWITCH_SUPPORTED` | `1` if the port can switch contexts at all. `0` selects the kernel's run-to-completion branch (`flat`). | | `ARCH_PREEMPTION_SUPPORTED` | `1` if the port drives a periodic timer that *forces* a switch. **Distinct from the above:** a port can switch but be cooperative-only (`pthread` = switch yes, preempt no). | | `ARCH_MIN_STACK_SIZE` | Smallest thread stack the port can run, in **cells** (§5) — it must hold the saved context frame **plus** the C dispatch chain that runs on the thread's own stack during a switch/preemption. `tk_thread_create()` rejects a smaller stack. | The two capability flags are independent on purpose. The kernel default preemption mode is `TK_PREEMPTIVE`, but it is **inert** where `ARCH_PREEMPTION_SUPPORTED == 0` (nothing calls the tick), so threads there switch only at yield/sleep/block. Make the macro honest; do not set it to 1 hoping the kernel will preempt for you. Status of the nine in-tree ports (`arch/*`): | port | CONTEXT_SWITCH | PREEMPTION | mechanism | status at root | |------|:--:|:--:|-----------|-----------------| | `arm_v6m` | 1 | 1 | board tick → PendSV (deferred) | complete; SMP real when built `TK_SMP_CORES>1` (rp2040) | | `arm_v7m` | 1 | 1 | board tick → PendSV (deferred) | complete; real MPU + unprivileged threads | | `arm_v8m` | 1 | 1 | board tick → PendSV (deferred) | complete; PMSAv8 MPU + unprivileged + PSPLIM hw stack-limit; SMP complete on multi-core M33 | | `risc_v` | 1 | 1 | CLINT mtimer → `mtvec` trap (trap-frame) | complete; PMP MPU + U-mode unprivileged; SMP real multi-hart | | `c2000` | 1 | 1 | CPU-timer ISR → inline switch, `switch.asm` (trap-frame) | implemented (`kernel_arch.c`+`switch.asm`); no board or `make/arch/c2000.mk` yet | | `tricore` | 1 | 1 | STM compare IRQ → inline switch (trap-frame) | **declared only** — `kernel_arch.h` has full prototypes incl. `ARCH_HAS_CONTEXT_RECLAIM`, but no `kernel_arch.c` at root yet | | `pthread` (host) | 1 | 0 | cooperative baton (mutex+condvar) | complete; `ARCH_OWNS_TIMEBASE 1` (reads the OS clock) | | `flat` (host_test) | 0 | 0 | run-to-completion | complete; no `arch_context_start`/`arch_idle` (mode 0 never launches or idles) | | `msp430` | 0 | 0 | — Tier 0 only, no kernel on this target | complete by design; no board paired yet | --- ## 1b. Optional tiers — SMP, MPU, unprivileged, context-reclaim Beyond the three mandatory macros above, a port may declare any of these; each is backed by a **central no-op default** in `arch/arch_defaults.h`, included at the *end* of every `kernel_arch.h`. A port `#define`s ONLY the capability flag for a tier it actually implements and supplies the real primitives; every tier it leaves undeclared compiles out to the no-op group instead — so e.g. `arm_v6m` (no MPU, no unprivileged split) is ~80 lines instead of hand-copying ~20 stub functions. This is the "stub tax" `arch_defaults.h` exists to kill. | Tier | Capability macro | Undeclared default | Real in-tree today | |---|---|---|---| | 2 — SMP | `ARCH_HAS_SMP` + `ARCH_CORE_COUNT` | 1 core; no-op spinlock/IPI (IRQ masking already serializes) | `arm_v6m` (rp2040, `SMP=2`), `arm_v8m` (multi-core M33), `risc_v` (multi-hart); `tricore` declares it, gated the same way | | 3a — MPU / stack-guard | `ARCH_HAS_MPU` | software canary only; `arch_stack_isolatable` reports "always isolatable" | `arm_v7m`, `arm_v8m` (PMSAv8), `risc_v` (PMP) | | 3b — unprivileged / syscall | `ARCH_HAS_UNPRIV` | every thread stays privileged; syscall-return-set is a no-op | `arm_v7m`, `arm_v8m`, `risc_v` (U-mode) | | 3c — context reclaim / lazy materialize | `ARCH_HAS_CONTEXT_RECLAIM` / `ARCH_HAS_LAZY_CONTEXT` | no default needed — the kernel guards the *call* on the macro, so absence compiles the call out entirely | `tricore` only: register context lives in hardware CSAs, not the stack, so a dead thread's CSA chain must be explicitly reclaimed, and under SMP a thread's CSAs must come from its home core's pool (`ARCH_HAS_LAZY_CONTEXT` ⇒ affinity becomes mandatory — `docs/decisions/0002-multi-instance-kernel.md`) | | 3d — supervisor fence | `ARCH_HAS_SUPERVISOR_FENCE` | no default needed — the kernel guards the *call* on `TK_FEAT_INSTANCE_ISOLATION` (derived from the flag), so absence compiles it out | `arm_v8m` + `arm_v7m` (silicon, drop `PRIVDEFENA`) + `risc_v` S-mode (QEMU, Sv32 satp swap; `TK_RISCV_SMODE`) + `tricore` (build-verified, PRS/DPR native bind — no `[lo,hi)` alignment constraint). Implement `arch_instance_fence_apply(deny_base, deny_size, ndeny)` → 1 if installed on this core, 0 → kernel fails closed. Mechanism-neutral deny-list: region arches hole/deny out, paging arches map-except (`docs/decisions/0003-supervisor-fence-seam.md`) | | property — stack direction | `ARCH_STACK_GROWS_UP` | `0` (grows down) | `c2000` (`1` — the C28x SP post-increments on PUSH) | | property — timebase owner | `ARCH_OWNS_TIMEBASE` | `0` (kernel tick counter owns it) | `pthread` (`1` — reads `CLOCK_MONOTONIC`, no kernel tick) | A port does **not** declare an isolation capability directly. `tk_config.h` DERIVES `TK_ARCH_ISOLATION_CAPABLE` = `defined(ARCH_HAS_MPU) && defined(ARCH_HAS_UNPRIV)` — a real HW isolation boundary needs both a memory-protection unit (to fence kernel RAM + per-thread stacks) and the unprivileged split (a thread the fence acts against). It reads **1** on `arm_v7m`/`arm_v8m`/`risc_v`, **0** on `arm_v6m` (M0+), `c2000`, and `pthread`. Apps test it (and the degrading `TK_ISOLATION_LEVEL` ladder / runtime `tk_kernel_isolation()`) for an honest per-arch answer to "is my island/thread boundary HW-backed here?" (review QC3-2/3); a port gets it for free by declaring 3a + 3b. The top rung, `TK_ISOLATION_INSTANCE` (the C3-B supervisor fence), is DERIVED separately from `TK_ARCH_SUPERVISOR_FENCE_CAPABLE` = `defined(ARCH_HAS_SUPERVISOR_FENCE)` (tier 3d) — **not** from 3a+3b: fencing the *kernel* between islands needs no unprivileged split, so a port may declare 3d without 3b. An app requesting `TK_CONFIG_ENABLE_INSTANCE_ISOLATION` on an arch that lacks 3d fails loud at compile time (`tools/check_fence_matrix.sh` guards the matrix). See `kernel/tk_config.h`. Full field-by-field prototypes for each tier: `docs/architecture/arch-interface.md`. --- ## 2. Required types + functions ```c struct arch_context { /* opaque to the kernel; e.g. one saved stack pointer as the FIRST word */ }; typedef arch_irq_state_t; /* saved interrupt-mask state */ static inline void arch_context_clear(struct arch_context*); /* mark unused */ void arch_context_init(struct arch_context*, void* stack_mem, size_t stack_cells, void* (*fn)(void*), void* arg); /* build first frame */ void arch_context_switch(struct arch_context* from, struct arch_context* to); void arch_context_start(struct arch_context* to); /* launch first thread */ void arch_idle(void); /* idle-thread body: wait for irq */ void arch_timer_init(void); /* arm the 1 ms tick */ uint32_t arch_timer_now_ms(void); /* only consumed when ARCH_OWNS_TIMEBASE */ /* Interrupt state — TWO distinct pairs (see §3) */ static inline void arch_irq_disable(void); /* flat mask: kernel cooperative yield */ static inline void arch_irq_enable(void); static inline arch_irq_state_t arch_irq_save(void); /* nesting-safe: HAL/DET */ static inline void arch_irq_restore(arch_irq_state_t); ``` `stack_mem`/`stack_cells` are a `void*` region and a **count of addressable cells** (`tk_stack_cell_t = unsigned char`, `kernel/tk_types.h`) — never `uint8_t*`/bytes. This is ADR 0001's CHAR_BIT-neutral stack contract: a cell is one `sizeof`-unit, 8 bits on every ARM/RISC-V/host port in the tree and 16 bits on C2000 (`CHAR_BIT == 16` there, no `uint8_t` exists). `ARCH_MIN_STACK_SIZE` is a cell count too, and `tk_thread_create()` (`kernel/tk_kernel.c`) rejects a stack smaller than it. `arch_timer_init()` is entirely board-owned — the port declares it but must not hardcode a rate. What "board-owned" means varies by port in the tree today: the ARM ports document it explicitly (`arm_v7m`'s header: *"arch_timer_init() is provided by the BOARD (tick source is board-specific: RTC1 on nRF52, where SysTick can't survive sleep)"*), while `risc_v`'s `arch_timer_init` reads a board `.mk` compile define (`TK_RISCV_MTIME_HZ`) rather than any HAL call. Either way, the arch never assumes a number. The tick *period* is 1 ms. `arch_idle()` is the kernel idle-thread body (called from `tk_kernel.c`); it should park the core until the next interrupt (e.g. `wfi` / `WFE`, or a `pause`). It is **required for any `ARCH_CONTEXT_SWITCH_SUPPORTED == 1` port** (the `flat` no-switch port omits it). Note that `arch_sched_init()` and `arch_kernel_tick()` are port-internal board glue (PendSV priority setup, the tick-ISR core) shared by all three deferred-switch ARM ports (`arm_v6m`/`arm_v7m`/`arm_v8m`) and called by each board's own tick driver — they are **not** part of this kernel-facing contract, so a new port need not provide them by name. --- ## 3. The interrupt-state contract (get this right) There are **two** masking pairs and they are not interchangeable: - **`arch_irq_disable()` / `arch_irq_enable()`** — a *flat* mask the kernel uses around its own scheduler bookkeeping. `enable()` turns interrupts on **unconditionally**; it is the cooperative-yield mask, not save/restore. Only the kernel uses it, and only where it knows interrupts should end up on. - **`arch_irq_save()` / `arch_irq_restore()`** — **nesting-safe**: `save()` returns the prior mask and disables; `restore()` puts that exact state back (re-enabling only if it was enabled). The HAL and the DET/error module use these so they are correct when called from any context, including nested critical sections. A new port must provide both pairs even if one is a no-op — true of every single-core build in the tree, including `flat`/`pthread` (no hardware interrupts at all) and `tricore`/`msp430` (single-core by default; `tricore` gains a real spinlock only once built `TK_SMP_CORES>1`). --- ## 4. Which preemption hook to call The scheduler decision lives in the kernel and is shared by yield and preemption so they always agree. The kernel-provided entry points a port calls are the Tier-B contract in `kernel/tk_kernel_port.h` (never included by application code). A preemptive port wires its timer to **one** of two hooks depending on *how* it switches: - **Deferred-switch ports** (the switch happens later, in a separate lowest-priority handler) call **`tk_kernel_tick_preempt()`** from the timer ISR. It runs the scheduler and *requests* a switch; the actual register save/restore happens in the deferred handler. → `arm_v6m`/`arm_v7m`/`arm_v8m`: the board tick calls `tk_kernel_tick_preempt()` (usually via the friendly wrapper below), which pends PendSV; PendSV does the switch and tail-chains after the ISR. - **Trap-frame ports** (the timer trap itself saves the full frame) call **`tk_kernel_preempt_select(current_sp)`** from the trap, which returns the next thread's stack pointer to restore in place. → `risc_v` (`mtvec` trap) and `c2000` (`switch.asm`) both do this; `tricore` is declared to when its body lands. The **friendly integration entry point** a board's 1 ms timer callback should actually call is `tk_tick()` (also in `tk_kernel_port.h`): it advances the kernel's millisecond base (`tk_tick_advance()`) *and* drives preemption via `tk_kernel_tick_preempt()` — this is what the ARM boards wire into their tick ISR. It is pre-launch-safe (a no-op before the first thread runs and during the launch-commit window), so an SDK that already runs its own timer before `tk_kernel_start()` may call it unconditionally. Both `tk_kernel_tick_preempt()` and `tk_kernel_preempt_select()` funnel into the same internal `tk_kernel_tick()` (wake due sleepers, refresh the quantum, pick the next thread) — `flat` and `pthread` aside, every real port agrees here. In `TK_COOPERATIVE` mode it still wakes sleepers but never forces a switch. **SMP adds a third hook**, only compiled when `ARCH_CORE_COUNT > 1`: `tk_kernel_switch_select(current_sp)` — the cooperative-yield counterpart for a trap-frame SMP port (records the outgoing SP, picks next, releases the kernel lock, returns the next SP; the lock is never held across the resume). A select-in-trap port whose restore must also reprogram per-thread hardware state the SP alone doesn't carry (PSPLIM, `CONTROL.nPRIV`, an MPU region) calls `tk_kernel_current_context()` right after `*_select` to fetch it. --- ## 5. Stacks and the trap frame The preemptive trap frame is **large and lives on the interrupted thread's own stack**, and its size and *unit* vary by port: - `risc_v` pushes a full 128-byte integer frame (cells = 8-bit bytes) — the reason its `ARCH_MIN_STACK_SIZE` floor is 512 cells. - `c2000` saves a smaller register set, but its cells are **16-bit words** (`CHAR_BIT == 16`, `ARCH_STACK_GROWS_UP 1`) — `ARCH_MIN_STACK_SIZE` (256) is a *word* count, not a byte count. Confusing the two on a word-addressed target is exactly the class of bug ADR 0001 exists to prevent. - `tricore` is the outlier shape: the register context lives in hardware Context Save Areas (CSAs), not on the C stack at all, so its `ARCH_MIN_STACK_SIZE` (512, 8-bit cells) is pure C call-chain headroom, not a register-frame floor. This is why it is the one port with `ARCH_HAS_CONTEXT_RECLAIM` (§1b) — freeing a dead thread means returning its CSA chain, not just its stack memory. Therefore, for any port: - Size every thread stack ≥ `ARCH_MIN_STACK_SIZE`, in that port's cells (the kernel enforces this in `tk_thread_create()`); size it generously for threads with deep call chains or FPU use. - Honour the CPU's stack alignment in `arch_context_init()` (e.g. 8-byte on Cortex-M, 16-byte on RISC-V). - Under-sizing manifests as the trap's stores running off the end of the stack and overwriting adjacent memory/code — on QEMU this shows up as illegal-instruction loops, not a clean fault. --- ## 6. CPU-architecture vs board-specific `arch/` is the *CPU* port; anything board-specific belongs in `boards/`. Two current seams to be aware of when adding a second board of an existing CPU: - **RISC-V CLINT/PLIC addresses** are presently hardcoded in `arch/risc_v` (legacy SiFive layout, base `0x02000000`) for the QEMU `virt` machine. A second RISC-V board would move these to a per-board constant the port reads (only `TK_RISCV_MTIME_HZ`, the timebase frequency, is already externalized to the board `.mk`). - **Timer source and rate are never hardcoded in the arch** — see §2. Keep it that way when adding a second board of an existing CPU: `arch_timer_init()` stays a board-provided function, even if two boards of the same CPU pick the source differently. - `arch_trap_init()` (RISC-V) sets `mtvec` *without* arming the timer, so a HAL-only superloop (no kernel) can take interrupts without a kernel tick calling into a scheduler that isn't running. --- ## 7. New-port checklist 1. `ARCH_CONTEXT_SWITCH_SUPPORTED`, `ARCH_PREEMPTION_SUPPORTED`, `ARCH_MIN_STACK_SIZE` (§1). 2. `struct arch_context` + `arch_context_clear`; decide the stack-cell width and `ARCH_STACK_GROWS_UP` if it isn't 0 (§1b, §5). 3. Both interrupt pairs (§3) — no-ops if single-threaded. 4. `arch_context_init` (build the first frame in `stack_cells`, honour alignment), `arch_context_switch`, `arch_context_start`. 5. `arch_timer_init` — board-provided rate, never hardcoded (§2); wire the timer to the right preempt hook (§4), normally via `tk_tick()` — or skip it entirely for a cooperative port (`ARCH_PREEMPTION_SUPPORTED 0`). 6. Add `make/arch/.mk` (toolchain + flags) and a `make/board/.mk` fragment — the two currently-missing pieces for `c2000`/`tricore`/`msp430` at root (§1's status table). 7. Verify: build and run the cross-board example apps against the new board — `examples/threads` (`make BOARD= run`/`flash`) for the single-core path, and, if the port declares `ARCH_HAS_SMP`, `examples/threads_smp` with `SMP=2` (`qemu_riscv32_virt` and `rp2040_pico` are the two in-tree boards that already prove this path). There is no standalone multi-arch build harness at root yet — the example apps are it.