UQO Working Papers · WP31 · v1.0
PRISME: Intervention-Supervised Sparse Workspace Editing for Small Language Models
Design, controlled evaluation, and negative and positive evidence at the 100M-parameter scale
- Year
- 2026
- First release
- October 2026
- Latest revision
- October 2026
- Publication status
- Working paper
Learned spatial routing in a sparse latent workspace does not pay
Sparse one-or-two-slot edits cut a frozen backbone's held-out loss by 2.8%, yet a global-recurrence adapter trained identically matches it at every budget, the primary effect is +0.9 points against a 3-point threshold, and the learned controller ranks actions near chance.
Abstract
PRISME augments a frozen causal decoder with a read-only source bank, an ephemeral sixteen-slot latent workspace re-created at every position, a shared sparse editor that rewrites one or two slots per action, a value controller trained on counterfactual intervention gains measured by executing paired edit/no-edit branches, and a differential readout with an exact zero-budget bypass. The paper reproduces the proposal's exact backbone parameter counts and analytic operation table, proves its structural properties (causality, bitwise identity, exact locality, bounded displacement, one-step selection regret), and tests seven pre-registered hypotheses with a complete implementation, a fifteen-invariant test suite, an incremental inference engine and seeded comparisons against a parameter-matched plain decoder, a dense-slot editor, a global-recurrence adapter trained with the identical recipe, five reference routers and exhaustive one- and two-step oracles.
The zero-initialised readout prescribed by the proposal does not learn because of a scale mismatch; a scale-aware non-zero initialisation does. Sparse edits then lower a frozen backbone's held-out loss by 2.8% and raise compositional exact match by about two points, but the global-recurrence adapter matches or exceeds this at every budget, and where to write does not matter for fixed schedules. An oracle that reads the target finds large opportunity (−15.4% of loss), but 25% of the variance of intervention gains is state-level rather than action-level, the opportunity is concentrated in high-uncertainty states where a wrong action costs more than a right one gains, and the learned controller ranks actions within a state near chance; it is worse than no routing on held-out data.
On the locked test regimes the pre-registered primary effect is +0.9 points against a three-point threshold. At 103M parameters on 98M tokens of web text, neither the sparse editor nor the global adapter moves the frozen backbone by more than 0.1% of loss and a parameter-matched plain decoder does better than both. Measured on Apple Silicon, per-action dispatch overhead erases most of the arithmetic savings of sparsity. The paper reports the complete negative result on learned spatial routing, its mechanistic explanation, and the conditions under which the hypothesis could still be rescued.
Key findings
+0.9 pts
pre-registered primary effect vs a 3-point threshold
−2.8%
held-out loss from sparse edits on a frozen backbone
−15.4%
loss reduction found by a target-reading oracle
25%
of intervention-gain variance is state-level, not action-level
≤ 0.1%
loss change of either adapter at 103M on 98M tokens
3×
measured latency of eight sparse edits vs analytic 1.6×
Scientific contributions
- A complete, tested reference implementation
- PRISME is implemented with exact parameter and operation accounting for 103M, 459M and 1.18B configurations, a fifteen-invariant test suite, an incremental inference engine and a bitwise-exact zero-budget bypass.
- Seven pre-registered hypotheses, several refuted
- On the locked test regimes the primary effect is +0.9 points against a three-point threshold; the editor-usefulness hypothesis holds only weakly and the spatial-selection, intervention-supervision and adaptivity hypotheses are rejected.
- Global recurrence matches sparse editing
- Over five adapter seeds, sparse edits lower held-out loss by 2.8% and raise exact match by about two points, but a parameter-matched global-recurrence adapter trained with the identical recipe does at least as well at every budget and dominates the quality-versus-cost frontier.
- Why the controller cannot capture the oracle's gain
- A one-step oracle lowers loss by 15.4% on the synthetic suite and 3.6% on web text, yet 25% of gain variance is state-level, opportunity sits in high-uncertainty states where wrong actions cost more than right ones gain, and the controller's within-state ranking is at chance.
- The prescribed zero-initialised readout does not learn
- A scale mismatch prevents the zero-initialised differential readout from training; a scale-aware non-zero bootstrap is required, a correction to the original proposal.
- Dispatch overhead erases sparse savings at small scale
- On an Apple M5 Max, eight sparse edits cost three times the backbone's decode latency at 103M against an analytic 1.6 times, and dense sixteen-slot writes are only 17% slower than sparse ones.
Methodology
Methodology (5)
Architecture with provable invariants
Source bank, ephemeral 16-slot workspace, shared two-block sparse editor (24 actions plus STOP) and differential readout; causality, bitwise identity at budget zero, exact locality, bounded displacement and one-step selection regret are proved and tested.
Counterfactual supervision of the controller
Intervention gains g(s,a;y) are measured by executing paired edit/no-edit branches at training time; the controller is trained with calibration-based early stopping and evaluated against five reference routers and exhaustive oracles.
Matched-cost controlled baselines
A parameter-matched plain decoder, a dense sixteen-slot editor and a global-recurrence adapter trained with the identical recipe; comparisons use five seeds with t-intervals and per-seed paired item bootstraps.
Pre-registration and locked test manifest
Seven hypotheses, a three-point primary threshold and a decision rule were registered before evaluation; deviations are logged, and every manuscript number is checked against run files by a claim-verification script.
Latency and operation accounting
Analytic operation tables for 103M, 459M and 1.18B configurations are compared with measured per-token latency on Apple Silicon (PyTorch MPS), isolating per-action dispatch overhead.
Research data
- Synthetic compositional suite
- A seeded, deterministically regenerated suite of templated compositional tasks with locked dev and test splits, used for the frozen-backbone experiments with five adapter seeds and for the exhaustive one- and two-step oracles.
- FineWeb-Edu subset (103M pilot)
- 98.3M training tokens streamed in canonical shard order from FineWeb-Edu sample-10BT (ODC-By 1.0), exactly deduplicated and hashed, with a locked 327,680-token test split; the backbone was pretrained in 2.3 hours on one Apple M3 Ultra (perplexity 45.72).
- Counterfactual intervention labels
- About 100k executed paired edit/no-edit branches with logged propensities, collected from an editor-unseen stream, used to train the value controller under four feature sets and three loss variants.
Reproducibility
- Pipeline in prisme/, training/ (stages A–D), baselines/ and evaluation/, with 23 passing pytest invariants and cache-equivalence tests
- Pre-registered protocol, locked test manifest and deviation log in protocol/; every run has an immutable ID, config hash, commit, device and wall-clock record
- All tables and figures regenerated by scripts/make_tables.py and scripts/reproduce_figures.py; scripts/verify_manuscript_claims.py checks every number in the manuscript
- scripts/reproduce_pilot.sh runs a miniature end-to-end pipeline on CPU in about one minute; full LaTeX source in paper/
Research artifacts
Version history
| Version | Date | Description | File |
|---|---|---|---|
| v1.0current | October 2026 | Current manuscript as published on www.spboucher.ai (revised edition after the 2026-10-17 scientific review for papers 1–17). | PDF · 49 p. |