# ADR 0002 — Multi-instance kernel: un-globaling the singleton, AMP islands in scope ("one model, N instances")

**Status:** Accepted (2026-08-19). **Supersedes** (narrowly): the *"separate-kernel AMP
out of scope"* stance in `smp-design.md` §8 and the `smp-model.md` (Q3) "SMP + AMP"
resolution — the scope limit only; **every scheduler mechanic those docs settled is
preserved.**

**Context.** Q3 (`smp-model.md`, 2026-08-16) settled the SMP model as a **giant-lock
singleton** — one `struct tk_kernel`, one `g_sched_lock`, per-core `current/idle/quantum/
entering`, affinity subsuming pinning, select-in-trap with the lock held only inside
`*_select`. It deliberately put **separate-kernel AMP out of scope** *for lack of proof*,
and modeled cross-core comms as "SMP + an optional mailbox," explicitly *"not a second
scheduling model … without reopening the separate-kernel design."*

That deferral was evidence-gated, and the evidence now exists. Between 2026-08-18 and
-08-19 separate-kernel AMP was **HW-proven twice**, brick-by-brick, on the user's bench:

- **rp2040 1 + 1** (`sketches/amp/rp2040/`): two `amp_kernel` instances, one per M0+ core,
  disjoint `owned_cores`, sharing nothing but a mailbox.
- **AURIX TC275 "2 SMP + 1 aside"** (`sketches/amp/tc275/`): **one** instance owning two
  cores (`kSMP` {CPU0,CPU1}, a real SMP domain) beside a **disjoint one-core AMP island**
  (`kAside` {CPU2}) — the exact shape this ADR generalizes.

plus the `temp/multicore.c` whiteboard (rounds 1–6). Both proofs kept the Tier-B port
entry-point *names* identical to the real kernel's (`tk_kernel_switch_select`,
`tk_kernel_preempt_select`, `tk_kernel_current_context`, `tk_kernel_start_this_core`), so
the un-globaling is confined to `tk_kernel.c` + its internal header — no arch/board edits.

The reshape's layering invariant frames the whole change: **the kernel depends only on
`arch` (`kernel_arch.h`); `board + HAL = SDK`; the kernel is a guest.** So an API reshape
may touch the kernel↔arch contract and ripple into the **example apps** (acceptable,
mechanical) but must not require SDK edits.

## Decision

Generalize the kernel singleton into **per-instance kernels resolved per physical core**,
bringing separate-kernel AMP into scope. One scheduling model, instantiated N times.

### 1. Un-global: what moves, what stays global-by-physical-core

| today (singleton) | becomes | why |
|---|---|---|
| `active_kernel` (1 ptr) | `g_core_kernel[ARCH_CORE_COUNT]` + `this_kernel()` | the linchpin — a core reads its kernel by `arch_core_id()` |
| `g_sched_lock` (1) | `k->lock` (into `struct tk_kernel`) | each instance schedules independently |
| — | **new** `k->owned_cores` (bitmask), `k->name` | the one knob: 2-bit set = SMP, disjoint 1-bit = AMP island |
| `g_sched_entering[]`, `g_sched_irq[]`, `g_quantum_remaining[]` | **stay global**, indexed by `arch_core_id()` | physical-HW reentrancy facts — a core has one each regardless of which hat it wears |
| `g_ticks_ms`, `g_time_base` | **now per-instance** — `k->ticks_ms`, `k->time_base` in `struct tk_kernel` (review C4 superseded the original "stay global") | §6 |
| `struct tk_kernel.current_thread[N]`, `idle_index[N]` | unchanged width `[ARCH_CORE_COUNT]`; an instance uses only the slots for cores it owns | preserves collapse (N=1 uses all) |

### 2. The same scheduler model, N times — not a new model

Every Q3 mechanic is preserved *per instance*: giant lock, first-class affinity
(subsumes pinning), per-core `current/idle/quantum/entering`, select-in-trap (lock only
inside `*_select`, never across a switch), the `g_sched_entering` re-entry guard,
phase-offset first slice, TTAS lock, `ARCH_HAS_LAZY_CONTEXT ⇒ mandatory affinity`. We do
**not** introduce a second scheduling model — we replicate the proven one. This honors
Q3's actual concern; only the *singleton / AMP-scope* limit is lifted.

### 3. The collapse property (non-negotiable safety contract)

**N = 1 instance owning all `ARCH_CORE_COUNT` cores == the Q3 singleton, byte-for-byte.**
`tk_kernel_init()` with no topology declared **auto-registers one all-cores instance**
bound to every core, so `this_kernel()` returns it everywhere and every existing
singleton call resolves unchanged. The multi-instance machinery is **present but dormant**
until an app opts into a second instance. Consequence: **existing arch ports and the SDK
are untouched; only example apps may ripple.** This is what lets the graft land without a
regression sweep of the ports.

### 4. API surface: `[rt]` stays pointer-free, `[setup]` targets an instance

- **`[rt]`** (`yield`/`sleep`/`exit`/`self`/`join`) keep the "no kernel pointer" promise —
  they resolve `this_kernel()` from the calling core.
- **`[setup]`** calls that target a *specific, not-yet-running* instance gain a handle. As built,
  `tk_kernel_define` is an ALLOCATOR that returns an opaque handle from a kernel-owned pool —
  `tk_kernel_t* tk_kernel_define(uint32_t owned_cores, const char* name)`. (The originally-sketched
  `struct tk_kernel* k` first arg was un-allocatable: `struct tk_kernel` is Tier-D opaque, so the app
  cannot supply storage.) Plus an instance-targeted thread create, `tk_thread_create_in`. The bare
  forms target the default instance → every existing app is source-compatible.
- **`tk_kernel_start()` stays arg-less.** As built, `g_core_kernel[]` is bound in `tk_kernel_init`
  (every slot → the default all-cores instance) and rebound incrementally by each `tk_kernel_define`;
  `tk_kernel_start` sets nothing here — it arms the timer, releases secondaries, and starts
  `this_kernel()` on the boot core. The invariant "`g_core_kernel[]` fully populated before
  `arch_smp_start()`" still holds because `init` (and any `define`) precede `start`.
- **Fork 1 (settled):** runtime registration is the primary mechanism (HW-proven twice);
  single-instance is the zero-config default; a compile-time/link-time topology table is
  **later sugar over** the runtime API, not the primary path.
- **Fork 2 (settled, option a):** a thread handle resolves to its instance by validating
  against `this_kernel()` — zero-cost, matches the islands-share-nothing invariant. A TCB
  back-pointer is a later add *iff* a real cross-island join case ever appears.

### 5. Bring-up keeps the SoC glue untouched

`g_core_kernel[]` is **fully populated before `arch_smp_start()`** — as built, bound in
`tk_kernel_init` (default all-cores) and rebound by `tk_kernel_define`, both of which run on the boot
core before `tk_kernel_start` releases the secondaries. Each secondary then calls the unchanged
`tk_kernel_start_this_core()`, `this_kernel()` already resolves, and **no secondary entry
(no SoC glue) needs a bind edit.** The layering invariant actively shapes the graft.

### 6. Timebase: per-instance (revised by review C4)

> **Superseded.** This ADR originally read: *"`g_ticks_ms` stays a single global wall clock; only
> the timebase-owning core advances it. Per-instance clocks are out of scope."* Review C4 reversed
> that: a single global wall clock advanced by one core is a cross-island **SPOF** — a fault/halt on
> that core freezes every other island's sleeps and timeouts. Per-instance clocks are now the model.

Each instance owns its own `k->ticks_ms` + `k->time_base` in `struct tk_kernel`, advanced only by its
own `k->ticker_core` (the lowest core it owns). The four arch tick handlers now call `tk_tick_advance()`
on **every** core's tick; the kernel self-selects which core advances, per instance — so a halt on one
island's ticker core cannot stop another island's clock. Het-SoC caveat: a SoC with a single *shared*
hardware timer makes that timer a HW SPOF regardless — a board-level fact to document, not kernel-fixable;
normal multicore has per-core timers (ARM SysTick, RISC-V per-hart CLINT). Collapse (N=1, one instance
owning all cores): `ticker_core == 0`, only core 0 advances — byte-identical timing to the old global.

### 7. SoC / multicore glue belongs in a per-arch multicore tier, not `boards/`

`arch_core_id` / `arch_spin_*` / `arch_ipi` / `arch_smp_start` + the secondary entry are
**arch-contract** code — but on some SoCs they are *implemented with SoC peripherals*
(RP2040 SIO, AURIX GPSR) that vanilla CPU-arch lacks. On ARMv6-M this is stark: no
LDREX/STREX, and `WFE`/`SEV` inter-core event semantics are IMPLEMENTATION-DEFINED, so the
SIO hardware spinlock is the *only* atomic. Today `boards/rp2040_pico/tk_smp_rp2040.c`
implements this contract **and `#include`s `tk_kernel.h`** — a board file reaching into the
kernel (the invariant's one leak, self-documented in the file).

**Decision:** such code moves to a per-arch multicore tier — `arch/<cpu>/<mp>/<soc>/`, e.g.
`arch/arm_v6m/<mp>/rp2040/`. The umbrella word is **config-neutral** (`mp` /
`multicore`, *not* `amp` — the same glue serves SMP too); the exact word is finalized at
move-time. This is a **decoupled follow-up refactor, not a graft prerequisite** (the graft
is pure `tk_kernel.c`). Relates to `_legacy/docs/notes/smp-bringup-centralization.md`.

### 8. Naming

- **Rename rule:** rename a legacy name only when it is wrong about its **layer or role**
  (`tk_board_log` → `tk_print`: the console is an *application* choice, not board logging);
  **keep** a name that is already consistent with its tier's convention and honest.
  Renames are driven by correctness, not modernization — each is an API break that churns
  the examples.
- **The one ergonomic exception:** add `tk_sleep_ms` as a `static inline` forwarder over
  the canonical `tk_thread_sleep_ms` (no ABI symbol, no macro blast radius, `-Werror`
  clean, flattens to zero cost, UNPRIV syscall path untouched). Examples adopt the short
  form so it is what people copy; the canonical stays for the browsable `tk_thread_*`
  family. **Guard rail:** this is a deliberate *singular* exception — we do **not** alias
  `tk_thread_yield`/`_exit`/`_join`, so the exception cannot metastasize.

## Consequences

- **One bitmask spans the whole space:** SMP-all, SMP-subset (2-of-4), bare-core-0 +
  kernel-elsewhere, and disjoint AMP islands — chosen by `owned_cores`, no new model.
- **Staged, always-green sequencing:**
  1. Un-global `tk_kernel.c` — collapse-verified on the existing single-instance boards
     (host / nRF52 / RV32-qemu / rp2040-SMP) before any second instance exists.
  2. Multi-instance enablement — registry, `tk_kernel_define`, per-instance init +
     per-instance idle-thread reservation (each instance reserves idles only for the cores
     it owns).
  3. SoC/multicore-tier refactor (§7) — independent.
  4. `tk_sleep_ms` alias (§8) — independent.
- **The AURIX↔rp2040 delta remains exactly `ARCH_HAS_LAZY_CONTEXT`** (Q3's payoff): the
  un-globaling adds no per-arch variance.
- **Supersedes narrowly:** `smp-design.md` §8 and `smp-model.md`'s "SMP + AMP" section are
  superseded on the *scope* question only; their scheduler mechanics stand and are carried
  forward here.

## Open / deferred

- ~~The `<mp>`-tier umbrella word and the physical move of the rp2040 glue (§7).~~ **DONE** — umbrella
  word = `mp`; glue moved to `arch/arm_v6m/mp/rp2040/` (the board fragment still pulls it into
  `TK_BOARD_SRCS`, so only that SoC compiles it).
- Compile-time / link-time topology declaration as sugar over the runtime registry (§4).
- Cross-island thread join / handle resolution beyond `this_kernel()` (§4, fork 2).
- Per-instance timebase for a hosted third-party RTOS (§6).
- **Post-graft review (2026-08-20):** `docs/reviews/amp-smp-review-0001.md` — a 5-lens as-built audit.
  Bucket A (lifecycle-phase guard, carve-idle release, `owned_cores` IPI filter, handle validation,
  this reconciliation, core-count assert) applied in the review cycle. Buckets B (error-reporting
  convention, `_in` API variants) and C (per-instance memory isolation, timebase SPOF, fault
  containment, privileged-tagging the linchpin globals) are the memory/safety-AMP session agenda.

## Follow-through (north star — code lands in the staged order above)

- **Touched by the un-globaling:** `kernel/tk_kernel.c`, `kernel/tk_kernel_internal.h`
  (struct `tk_kernel` gains `owned_cores`/`name`/`lock`; `active_kernel` → `g_core_kernel[]`).
  `kernel/tk_kernel.h` gains the `tk_sleep_ms` inline. **No `arch/` or `boards/` edits** for
  the graft itself.
- **Examples** may need mechanical updates when the multi-instance surface + `tk_sleep_ms`
  land (the accepted, bounded trade-off).
- This ADR is referenced by the pending corpus migration (it marks the superseded
  sections of `smp-design.md` / `smp-model.md` when those move into root `docs/`).