Edition README
Arithmetic J-Space Training Lab
README.md
1 # Arithmetic J-Space Training Lab v0.4.0
2
3 ## Event-sampler hotfix
4
5 This build replaces rare-event rejection sampling with a constructive
6 exactly-one-carry or exactly-one-borrow generator. It fixes the deterministic
7 `carry_discovery` failure at the first 10-digit OOD evaluation. An original
8 v0.4.0 discovery run is accepted only when its resolved configuration differs
9 solely by the allowlisted package-manifest update. The migration is recorded in
10 `package_hotfix_migration.json`, and training resumes from the latest valid
11 rolling checkpoint.
12
13 This package trains standard decoder-only transformers from scratch on
14 answer-only addition and subtraction, then tests a **candidate balanced causal
15 arithmetic subspace**. It does not insert a workspace, supervise a hidden
16 space, use answer labels during subspace discovery, or modify a trained model
17 to create the candidate.
18
19 The shipped task uses positive integer operands and nonnegative results;
20 subtraction operands are ordered to avoid negative answers. It does not test
21 signed arithmetic, zero identities, multiplication, or division.
22
23 Version 0.4 is deliberately candidate-only. It can establish behavioral
24 competence and run target-blind geometric and causal diagnostics, but it cannot
25 establish a J-space equivalent. Formal causal mediation, native shared
26 multi-query access, and component-specific upstream-writer rescue are not
27 implemented. Every profile therefore reports:
28
29 ```text
30 JSPACE_CLAIM=NOT_ESTABLISHED
31 ```
32
33 ## Recommended Windows run
34
35 Use the known `komorebi-full` Python explicitly:
36
37 ```powershell
38 $d="$env:USERPROFILE\Downloads\Arithmetic_JSpace_Training_Lab_v0.4.0"; & "$d\Run_Arithmetic_Workspace_Experiment.ps1" -Profile carry_discovery -Python "$env:USERPROFILE\miniconda3\envs\komorebi-full\python.exe" -Device cuda -SkipInstall
39 ```
40
41 The command runs all three discovery seeds. It is resumable: rerun the
42 identical command after interruption. Progress is printed and flushed every
43 100 optimizer steps. Light evaluation runs every 500 steps and full evaluation
44 every 2,000 steps in the serious profiles.
45
46 ## Profiles
47
48 | Profile | Purpose | Model | Seeds × arms × steps |
49 |---|---|---:|---:|
50 | `workspace_smoke` | Fast engineering and resume check | 3 layers, width 128 | 1 × 4 × 20 |
51 | `carry_discovery` | Exploratory layer/rank discovery | 12 layers, width 512, ~38M parameters | 3 × 4 × 60,000 |
52 | `carry_confirmatory` | Locked replication of candidate assays | Same ~38M model | 8 × 4 × 80,000 |
53
54 `carry_confirmatory` requires the `workspace_lock.json` written by a completed
55 discovery run. It is a locked candidate replication—not a profile capable of
56 establishing a J-space claim.
57
58 On an 8 GB RTX 3070 Ti Laptop GPU, budget minutes for smoke, roughly 3–9 days
59 for discovery, and roughly 12–35 days for the full confirmation. These are
60 planning ranges. Use the measured ETA after the first 500 steps of each arm.
61 Keep roughly 30 GB free for the default discovery checkpoints and 80 GB for
62 the full locked replication. Exact use depends on checkpoint serialization and
63 the number of completed arms; outputs are not deleted automatically.
64
65 ## Scratch model and objective
66
67 - fixed 260-token byte vocabulary (`0..255`, PAD, BOS, SEP, EOS)
68 - prompt-masked causal loss on answer bytes and EOS only
69 - pre-normalized causal decoder with rotary positions
70 - no pretrained weights, scratchpad, carry trace, auxiliary representation
71 loss, bottleneck, or probe loss
72 - AdamW with warmup-stable-decay learning-rate schedule
73 - BF16 when supported, otherwise scaled FP16 on CUDA
74 - rolling atomic resume checkpoints and immutable research checkpoints
75
76 The primary model is not Qwen or another downloaded model.
77
78 ## Four arms
79
80 | Arm | Training target | Role |
81 |---|---|---|
82 | `correct` | Exact addition/subtraction | Arithmetic candidate |
83 | `no_carry` | Exact arithmetic restricted to zero-carry/zero-borrow training rows | Restricted-computation behavioral control |
84 | `pseudo_fst` | Seeded finite-state digit values with arithmetic-answer-matched length | Pseudo-valued sequential control, arithmetic-answer-length matched |
85 | `deranged` | Audited random value derangement | Random-label/format control |
86
87 `correct`, `pseudo_fst`, and `deranged` use the same input-expression order and
88 paired initial weights. `no_carry` necessarily uses a different restricted
89 training stream; it is not globally prompt-, token-, or answer-width-matched.
90 The exact shared comparison with it is the sealed `no_transfer` panel.
91
92 Each derangement window first attempts a seeded random value permutation within
93 operator/output-width nuisance strata, never a lexical sort-and-rotate. An
94 underdetermined stratum is deterministically coarsened to output-width-only
95 pooling when needed. Exact empirical target-multiset preservation and zero
96 source/assigned value fixed points remain mandatory for every nonsynthetic
97 stratum, including a successfully coarsened stratum. Within a seeded candidate,
98 synthetic labels are not considered until exact operator/output-width and exact
99 output-width-only constructions are mathematically impossible. Additional
100 seeded candidates serve the residual-correlation and string-agreement gates.
101
102 Only when an output-width-only stratum of at most 31 rows makes such a value
103 derangement mathematically impossible does the final fallback synthesize
104 canonical same-width decimal labels, deterministically and with no
105 source/assigned value fixed points. Synthetic rows therefore do **not**
106 preserve the empirical target multiset. Their count and total-window fraction
107 are explicitly reported in the control audit. The 31-row limit applies to each
108 impossible width stratum separately. The strict fraction cap is
109 `0.001` (0.1%); the audit records the cap and pass/fail result, and the run fails
110 closed if the cap is exceeded. A constructed window that uses the fallback
111 emits one `DERANGEMENT_SYNTHETIC_FALLBACK` telemetry line. This fallback weakens
112 the deranged control, but prevents a rare finite training window from becoming
113 impossible to construct. The
114 residual-correlation gate uses the absolute row-weighted Pearson correlation
115 over predeclared eligible strata. Tiny or degenerate strata are still reported
116 but excluded from that gate using the fixed eligibility rule in
117 `workspace_data.py`.
118
119 ## Target-blind balanced-space discovery
120
121 Discovery receives tokenized prefixes ending at SEP. It rejects answer tokens
122 and EOS. Upstream residual perturbations estimate empirical controllability;
123 gradients of fixed, answer-independent output summaries estimate local
124 observability. The summaries are exactly:
125
126 - digit log-probability mass
127 - EOS log probability
128 - minus-sign log probability
129 - output entropy
130 - top log probability
131 - four sealed random projections of the vocabulary log-probability vector
132
133 The leading directions of
134
135 \[
136 G_c^{1/2}G_oG_c^{1/2}
137 \]
138
139 form the candidate. Rank is selected by heldout prediction of changes in those
140 same target-free summaries. Rank-matched controls are PCA, one seeded random
141 subspace, and a complementary balanced-spectrum subspace. There is no
142 bootstrap-stability assay or distribution of many random subspaces in v0.4.
143
144 Geometry runs only after target-task competence and is fitted independently in
145 the competent `correct` and `pseudo_fst` arms. Their causal effects are compared
146 as a limited specificity diagnostic. The pseudo-FST controls digit values but
147 deliberately uses the true answer width to match output length. It therefore
148 retains a coarse arithmetic-derived schema signal and is not an entirely
149 arithmetic-free control.
150
151 ## Implemented causal assays
152
153 On sealed, semantically matched carry/borrow counterfactual pairs, every basis
154 (candidate, PCA, random, complement) receives the same free-generation tests at
155 the final prefix SEP position:
156
157 1. candidate-coordinate ablation
158 2. clean-to-corrupt coordinate patching with directional digit audit
159 3. independent transplant of a corrupt donor residual at an earlier layer
160 4. restoration of only the registered source-layer coordinates after that
161 upstream transplant
162
163 Control interventions are rank-matched and scaled to the candidate intervention
164 energy for the same state. Candidate fitting and selection never receive
165 answers. After the candidate is frozen, sealed answers and their widths are
166 used to construct/match counterfactual pairs, define eligibility, and score
167 free generation.
168
169 Primary rates are computed only on qualified pairs. Ablation requires a valid,
170 EOS-terminated natural clean answer. Patching requires valid and correct natural
171 clean and corrupt answers with distinct targets. Rescue additionally requires
172 the unrescued upstream corrupt-donor transplant to generate the corrupt target
173 exactly. Discovery requires at least 128 baseline-qualified pairs and 32 such
174 directional rescue pairs; locked confirmation requires 512 and 128. A missing
175 denominator fails closed rather than being treated as a null effect.
176
177 The upstream transplant is a residual-state intervention, not a head/MLP writer
178 intervention. Coordinate rescue is not a formal mediation analysis. No native
179 shared multi-query assay or unrelated-language preservation assay is bundled.
180
181 ## Statistics and evidence ceiling
182
183 Confirmation retains item-level outputs. Seed is the replication unit for
184 candidate-minus-strongest-control ablation, patch, and rescue effects. The
185 registered analysis uses paired seed tests, Holm familywise correction, and a
186 family-clustered hierarchical bootstrap of directional patch effects.
187
188 These statistics can support a replicated **candidate effect**. They cannot
189 override the v0.4 claim ceiling. Missing formal mediation, shared multi-query
190 access, and component-specific writer rescue keep the claim state
191 `NOT_ESTABLISHED` even if every implemented comparison passes.
192
193 ## Output and resume
194
195 The result directory contains:
196
197 ```text
198 resolved_experiment.json
199 runtime.json
200 progress.jsonl
201 training/seed_<n>/<arm>/
202 behavior/seed_<n>/
203 behavior/seed_<n>/control_audits.json
204 geometry/seed_<n>/
205 causal/seed_<n>/
206 causal/confirmation_statistics.json
207 workspace_lock.json # discovery only, if a selection passes
208 experiment_summary.json
209 ```
210
211 Rerunning the same command resumes. A resolved-config mismatch fails closed;
212 use a new `-Output` path for a changed run. `-NoResume` disables resumption.
213
214 For exact preregistered comparisons and limitations, read
215 `EXPERIMENT_DESIGN.md` and `WORKSPACE_PROTOCOL.md`.