Porting the tk kernel to a new CPU (arch/<cpu>)

Audience: anyone adding a CPU port or reviewing an existing one. Scope: the contract between the portable kernel (kernel/) and a CPU port (arch/<cpu>/kernel_arch.{h,c}). Field-by-field reference: docs/architecture/arch-interface.md. The byte-width contract this doc leans on throughout is docs/decisions/0001-foreign-architecture-policy.md (ADR 0001). The HAL/MCAL contract is separate — docs/architecture/hal-architecture.md. Board-level sequencing (where this port slots into a board’s bring-up) is docs/guides/board-bringup.md.

A port is one header + one source file selected by TK_ARCH in the board fragment (make/board/<board>.mk, via kernel/tk_arch.h’s front door). It owns three mandatory things and nothing else: context switching, the interrupt-state primitive, and the timer tick. It may additionally opt into SMP, an MPU/stack-guard, unprivileged threads, and context-reclaim/lazy-materialization (§1b) — but it must not know about peripherals, devices, or the application.

Existing ports to copy from, by switching mechanism:

  • Deferred switch via PendSV — arm_v6m, arm_v7m, arm_v8m (a board tick calls tk_kernel_tick_preempt(); the actual register save/restore happens later, in PendSV).

  • Trap-frame switch (the timer trap saves the full frame itself) — risc_v (CLINT → mtvec), c2000 (CPU-timer ISR → switch.asm); tricore declares this shape too but its kernel_arch.c has not been grafted into the tree yet (see §7’s status table).

  • Cooperative host sim — pthread (a mutex+condvar baton; ARCH_CONTEXT_SWITCH_SUPPORTED 1, ARCH_PREEMPTION_SUPPORTED 0).

  • No switching — flat (run-to-completion, for host_test).

  • Tier 0 only, no kernel — msp430 (declares nothing beyond the IRQ pair; a HAL/MCAL superloop links it and never pulls in the scheduler).

1. Capability macros (in kernel_arch.h)

Macro

Meaning

ARCH_CONTEXT_SWITCH_SUPPORTED

1 if the port can switch contexts at all. 0 selects the kernel’s run-to-completion branch (flat).

ARCH_PREEMPTION_SUPPORTED

1 if the port drives a periodic timer that forces a switch. Distinct from the above: a port can switch but be cooperative-only (pthread = switch yes, preempt no).

ARCH_MIN_STACK_SIZE

Smallest thread stack the port can run, in cells (§5) — it must hold the saved context frame plus the C dispatch chain that runs on the thread’s own stack during a switch/preemption. tk_thread_create() rejects a smaller stack.

The two capability flags are independent on purpose. The kernel default preemption mode is TK_PREEMPTIVE, but it is inert where ARCH_PREEMPTION_SUPPORTED == 0 (nothing calls the tick), so threads there switch only at yield/sleep/block. Make the macro honest; do not set it to 1 hoping the kernel will preempt for you.

Status of the nine in-tree ports (arch/*):

port

CONTEXT_SWITCH

PREEMPTION

mechanism

status at root

arm_v6m

1

1

board tick → PendSV (deferred)

complete; SMP real when built TK_SMP_CORES>1 (rp2040)

arm_v7m

1

1

board tick → PendSV (deferred)

complete; real MPU + unprivileged threads

arm_v8m

1

1

board tick → PendSV (deferred)

complete; PMSAv8 MPU + unprivileged + PSPLIM hw stack-limit; SMP complete on multi-core M33

risc_v

1

1

CLINT mtimer → mtvec trap (trap-frame)

complete; PMP MPU + U-mode unprivileged; SMP real multi-hart

c2000

1

1

CPU-timer ISR → inline switch, switch.asm (trap-frame)

implemented (kernel_arch.c+switch.asm); no board or make/arch/c2000.mk yet

tricore

1

1

STM compare IRQ → inline switch (trap-frame)

declared only — kernel_arch.h has full prototypes incl. ARCH_HAS_CONTEXT_RECLAIM, but no kernel_arch.c at root yet

pthread (host)

1

0

cooperative baton (mutex+condvar)

complete; ARCH_OWNS_TIMEBASE 1 (reads the OS clock)

flat (host_test)

0

0

run-to-completion

complete; no arch_context_start/arch_idle (mode 0 never launches or idles)

msp430

0

0

— Tier 0 only, no kernel on this target

complete by design; no board paired yet

1b. Optional tiers — SMP, MPU, unprivileged, context-reclaim

Beyond the three mandatory macros above, a port may declare any of these; each is backed by a central no-op default in arch/arch_defaults.h, included at the end of every kernel_arch.h. A port #defines ONLY the capability flag for a tier it actually implements and supplies the real primitives; every tier it leaves undeclared compiles out to the no-op group instead — so e.g. arm_v6m (no MPU, no unprivileged split) is ~80 lines instead of hand-copying ~20 stub functions. This is the “stub tax” arch_defaults.h exists to kill.

Tier

Capability macro

Undeclared default

Real in-tree today

2 — SMP

ARCH_HAS_SMP + ARCH_CORE_COUNT

1 core; no-op spinlock/IPI (IRQ masking already serializes)

arm_v6m (rp2040, SMP=2), arm_v8m (multi-core M33), risc_v (multi-hart); tricore declares it, gated the same way

3a — MPU / stack-guard

ARCH_HAS_MPU

software canary only; arch_stack_isolatable reports “always isolatable”

arm_v7m, arm_v8m (PMSAv8), risc_v (PMP)

3b — unprivileged / syscall

ARCH_HAS_UNPRIV

every thread stays privileged; syscall-return-set is a no-op

arm_v7m, arm_v8m, risc_v (U-mode)

3c — context reclaim / lazy materialize

ARCH_HAS_CONTEXT_RECLAIM / ARCH_HAS_LAZY_CONTEXT

no default needed — the kernel guards the call on the macro, so absence compiles the call out entirely

tricore only: register context lives in hardware CSAs, not the stack, so a dead thread’s CSA chain must be explicitly reclaimed, and under SMP a thread’s CSAs must come from its home core’s pool (ARCH_HAS_LAZY_CONTEXT ⇒ affinity becomes mandatory — docs/decisions/0002-multi-instance-kernel.md)

3d — supervisor fence

ARCH_HAS_SUPERVISOR_FENCE

no default needed — the kernel guards the call on TK_FEAT_INSTANCE_ISOLATION (derived from the flag), so absence compiles it out

arm_v8m + arm_v7m (silicon, drop PRIVDEFENA) + risc_v S-mode (QEMU, Sv32 satp swap; TK_RISCV_SMODE) + tricore (build-verified, PRS/DPR native bind — no [lo,hi) alignment constraint). Implement arch_instance_fence_apply(deny_base, deny_size, ndeny) → 1 if installed on this core, 0 → kernel fails closed. Mechanism-neutral deny-list: region arches hole/deny out, paging arches map-except (docs/decisions/0003-supervisor-fence-seam.md)

property — stack direction

ARCH_STACK_GROWS_UP

0 (grows down)

c2000 (1 — the C28x SP post-increments on PUSH)

property — timebase owner

ARCH_OWNS_TIMEBASE

0 (kernel tick counter owns it)

pthread (1 — reads CLOCK_MONOTONIC, no kernel tick)

A port does not declare an isolation capability directly. tk_config.h DERIVES TK_ARCH_ISOLATION_CAPABLE = defined(ARCH_HAS_MPU) && defined(ARCH_HAS_UNPRIV) — a real HW isolation boundary needs both a memory-protection unit (to fence kernel RAM + per-thread stacks) and the unprivileged split (a thread the fence acts against). It reads 1 on arm_v7m/arm_v8m/risc_v, 0 on arm_v6m (M0+), c2000, and pthread. Apps test it (and the degrading TK_ISOLATION_LEVEL ladder / runtime tk_kernel_isolation()) for an honest per-arch answer to “is my island/thread boundary HW-backed here?” (review QC3-2/3); a port gets it for free by declaring 3a + 3b. The top rung, TK_ISOLATION_INSTANCE (the C3-B supervisor fence), is DERIVED separately from TK_ARCH_SUPERVISOR_FENCE_CAPABLE = defined(ARCH_HAS_SUPERVISOR_FENCE) (tier 3d) — not from 3a+3b: fencing the kernel between islands needs no unprivileged split, so a port may declare 3d without 3b. An app requesting TK_CONFIG_ENABLE_INSTANCE_ISOLATION on an arch that lacks 3d fails loud at compile time (tools/check_fence_matrix.sh guards the matrix). See kernel/tk_config.h.

Full field-by-field prototypes for each tier: docs/architecture/arch-interface.md.

2. Required types + functions

struct arch_context { /* opaque to the kernel; e.g. one saved stack pointer as the FIRST word */ };
typedef <uint> arch_irq_state_t;   /* saved interrupt-mask state */

static inline void arch_context_clear(struct arch_context*);   /* mark unused */
void arch_context_init(struct arch_context*, void* stack_mem, size_t stack_cells,
                       void* (*fn)(void*), void* arg);         /* build first frame */
void arch_context_switch(struct arch_context* from, struct arch_context* to);
void arch_context_start(struct arch_context* to);              /* launch first thread */
void arch_idle(void);                                          /* idle-thread body: wait for irq */
void arch_timer_init(void);                                    /* arm the 1 ms tick */
uint32_t arch_timer_now_ms(void);                               /* only consumed when ARCH_OWNS_TIMEBASE */

/* Interrupt state — TWO distinct pairs (see §3) */
static inline void arch_irq_disable(void);    /* flat mask: kernel cooperative yield */
static inline void arch_irq_enable(void);
static inline arch_irq_state_t arch_irq_save(void);    /* nesting-safe: HAL/DET */
static inline void arch_irq_restore(arch_irq_state_t);

stack_mem/stack_cells are a void* region and a count of addressable cells (tk_stack_cell_t = unsigned char, kernel/tk_types.h) — never uint8_t*/bytes. This is ADR 0001’s CHAR_BIT-neutral stack contract: a cell is one sizeof-unit, 8 bits on every ARM/RISC-V/host port in the tree and 16 bits on C2000 (CHAR_BIT == 16 there, no uint8_t exists). ARCH_MIN_STACK_SIZE is a cell count too, and tk_thread_create() (kernel/tk_kernel.c) rejects a stack smaller than it.

arch_timer_init() is entirely board-owned — the port declares it but must not hardcode a rate. What “board-owned” means varies by port in the tree today: the ARM ports document it explicitly (arm_v7m’s header: “arch_timer_init() is provided by the BOARD (tick source is board-specific: RTC1 on nRF52, where SysTick can’t survive sleep)”), while risc_v’s arch_timer_init reads a board .mk compile define (TK_RISCV_MTIME_HZ) rather than any HAL call. Either way, the arch never assumes a number. The tick period is 1 ms.

arch_idle() is the kernel idle-thread body (called from tk_kernel.c); it should park the core until the next interrupt (e.g. wfi / WFE, or a pause). It is required for any ARCH_CONTEXT_SWITCH_SUPPORTED == 1 port (the flat no-switch port omits it). Note that arch_sched_init() and arch_kernel_tick() are port-internal board glue (PendSV priority setup, the tick-ISR core) shared by all three deferred-switch ARM ports (arm_v6m/arm_v7m/arm_v8m) and called by each board’s own tick driver — they are not part of this kernel-facing contract, so a new port need not provide them by name.

3. The interrupt-state contract (get this right)

There are two masking pairs and they are not interchangeable:

  • arch_irq_disable() / arch_irq_enable() — a flat mask the kernel uses around its own scheduler bookkeeping. enable() turns interrupts on unconditionally; it is the cooperative-yield mask, not save/restore. Only the kernel uses it, and only where it knows interrupts should end up on.

  • arch_irq_save() / arch_irq_restore() — nesting-safe: save() returns the prior mask and disables; restore() puts that exact state back (re-enabling only if it was enabled). The HAL and the DET/error module use these so they are correct when called from any context, including nested critical sections.

A new port must provide both pairs even if one is a no-op — true of every single-core build in the tree, including flat/pthread (no hardware interrupts at all) and tricore/msp430 (single-core by default; tricore gains a real spinlock only once built TK_SMP_CORES>1).

4. Which preemption hook to call

The scheduler decision lives in the kernel and is shared by yield and preemption so they always agree. The kernel-provided entry points a port calls are the Tier-B contract in kernel/tk_kernel_port.h (never included by application code). A preemptive port wires its timer to one of two hooks depending on how it switches:

  • Deferred-switch ports (the switch happens later, in a separate lowest-priority handler) call tk_kernel_tick_preempt() from the timer ISR. It runs the scheduler and requests a switch; the actual register save/restore happens in the deferred handler. → arm_v6m/arm_v7m/arm_v8m: the board tick calls tk_kernel_tick_preempt() (usually via the friendly wrapper below), which pends PendSV; PendSV does the switch and tail-chains after the ISR.

  • Trap-frame ports (the timer trap itself saves the full frame) call tk_kernel_preempt_select(current_sp) from the trap, which returns the next thread’s stack pointer to restore in place. → risc_v (mtvec trap) and c2000 (switch.asm) both do this; tricore is declared to when its body lands.

The friendly integration entry point a board’s 1 ms timer callback should actually call is tk_tick() (also in tk_kernel_port.h): it advances the kernel’s millisecond base (tk_tick_advance()) and drives preemption via tk_kernel_tick_preempt() — this is what the ARM boards wire into their tick ISR. It is pre-launch-safe (a no-op before the first thread runs and during the launch-commit window), so an SDK that already runs its own timer before tk_kernel_start() may call it unconditionally.

Both tk_kernel_tick_preempt() and tk_kernel_preempt_select() funnel into the same internal tk_kernel_tick() (wake due sleepers, refresh the quantum, pick the next thread) — flat and pthread aside, every real port agrees here. In TK_COOPERATIVE mode it still wakes sleepers but never forces a switch.

SMP adds a third hook, only compiled when ARCH_CORE_COUNT > 1: tk_kernel_switch_select(current_sp) — the cooperative-yield counterpart for a trap-frame SMP port (records the outgoing SP, picks next, releases the kernel lock, returns the next SP; the lock is never held across the resume). A select-in-trap port whose restore must also reprogram per-thread hardware state the SP alone doesn’t carry (PSPLIM, CONTROL.nPRIV, an MPU region) calls tk_kernel_current_context() right after *_select to fetch it.

5. Stacks and the trap frame

The preemptive trap frame is large and lives on the interrupted thread’s own stack, and its size and unit vary by port:

  • risc_v pushes a full 128-byte integer frame (cells = 8-bit bytes) — the reason its ARCH_MIN_STACK_SIZE floor is 512 cells.

  • c2000 saves a smaller register set, but its cells are 16-bit words (CHAR_BIT == 16, ARCH_STACK_GROWS_UP 1) — ARCH_MIN_STACK_SIZE (256) is a word count, not a byte count. Confusing the two on a word-addressed target is exactly the class of bug ADR 0001 exists to prevent.

  • tricore is the outlier shape: the register context lives in hardware Context Save Areas (CSAs), not on the C stack at all, so its ARCH_MIN_STACK_SIZE (512, 8-bit cells) is pure C call-chain headroom, not a register-frame floor. This is why it is the one port with ARCH_HAS_CONTEXT_RECLAIM (§1b) — freeing a dead thread means returning its CSA chain, not just its stack memory.

Therefore, for any port:

  • Size every thread stack ≥ ARCH_MIN_STACK_SIZE, in that port’s cells (the kernel enforces this in tk_thread_create()); size it generously for threads with deep call chains or FPU use.

  • Honour the CPU’s stack alignment in arch_context_init() (e.g. 8-byte on Cortex-M, 16-byte on RISC-V).

  • Under-sizing manifests as the trap’s stores running off the end of the stack and overwriting adjacent memory/code — on QEMU this shows up as illegal-instruction loops, not a clean fault.

6. CPU-architecture vs board-specific

arch/<cpu> is the CPU port; anything board-specific belongs in boards/<board>. Two current seams to be aware of when adding a second board of an existing CPU:

  • RISC-V CLINT/PLIC addresses are presently hardcoded in arch/risc_v (legacy SiFive layout, base 0x02000000) for the QEMU virt machine. A second RISC-V board would move these to a per-board constant the port reads (only TK_RISCV_MTIME_HZ, the timebase frequency, is already externalized to the board .mk).

  • Timer source and rate are never hardcoded in the arch — see §2. Keep it that way when adding a second board of an existing CPU: arch_timer_init() stays a board-provided function, even if two boards of the same CPU pick the source differently.

  • arch_trap_init() (RISC-V) sets mtvec without arming the timer, so a HAL-only superloop (no kernel) can take interrupts without a kernel tick calling into a scheduler that isn’t running.

7. New-port checklist

  1. ARCH_CONTEXT_SWITCH_SUPPORTED, ARCH_PREEMPTION_SUPPORTED, ARCH_MIN_STACK_SIZE (§1).

  2. struct arch_context + arch_context_clear; decide the stack-cell width and ARCH_STACK_GROWS_UP if it isn’t 0 (§1b, §5).

  3. Both interrupt pairs (§3) — no-ops if single-threaded.

  4. arch_context_init (build the first frame in stack_cells, honour alignment), arch_context_switch, arch_context_start.

  5. arch_timer_init — board-provided rate, never hardcoded (§2); wire the timer to the right preempt hook (§4), normally via tk_tick() — or skip it entirely for a cooperative port (ARCH_PREEMPTION_SUPPORTED 0).

  6. Add make/arch/<cpu>.mk (toolchain + flags) and a make/board/<board>.mk fragment — the two currently-missing pieces for c2000/tricore/msp430 at root (§1’s status table).

  7. Verify: build and run the cross-board example apps against the new board — examples/threads (make BOARD=<board> run/flash) for the single-core path, and, if the port declares ARCH_HAS_SMP, examples/threads_smp with SMP=2 (qemu_riscv32_virt and rp2040_pico are the two in-tree boards that already prove this path). There is no standalone multi-arch build harness at root yet — the example apps are it.