Dipankar Sarkar PRO

Your three numbers already say something about how the heads fire together.

No gate: 72.3% answered locally. At 0.95 that drops to 67.1%. The gate pulls back 5.2 points, 7.2% of what was local. Three heads each firing on an independent 5% would pull back 1 - 0.95^3 = 14.3%.

At 0.99: 1.7 points, 2.4% of local, against 3.0% for three independent 1% heads.

So at 0.95 the gate costs about half of what independent heads would. At 0.99 about four fifths.

Two readings fit that. The heads fire on the same tickets. Or they fire on tickets that were not confident anyway, so the gate costs nothing there.

Which one is it? check.py could print how many local answers are lost to exactly one, two or three heads.

Clean then. Thanks for pulling the exact day.

QA falling from 37.7M to 1.5M in one recompute reads like the dedup, not the file-type change. The same machines calling load_dataset over and over, now merged per 5 minutes.

Which raises the mirror question for robotics. The new counter merges per IP per repo per 5-minute window. A LeRobot snapshot pulls many files, often video. A download that runs past 5 minutes keeps sending requests into new windows.

Does one long snapshot count once, or once per window? And within robotics, do downloads per repo rise with repo size in bytes?

You're right on 0.63. I divided a saturated tail by an unsaturated sd. 7.99e-6 / 2.200e-5 = 0.363. The 0.64 is the total bias, and I matched the wrong pair.

On your question: no. Not to 1 percent, and not without your Sigma.

What the second-order term does say, without the number:

Near the fold, d node / d delta is carried by the soft eigenvector v and grows like 1/sqrt(x). A shifted-delta reading is a displacement along v only. Every coordinate then returns the same delta', so the ratio is 1.

The burst displacement is 1/2 J^-1 (H : Sigma). Write it as alpha v + beta u, with u the stiff direction. In delta' units each coordinate c gives

delta'(c) = sqrt(x)/a * (alpha + beta * u_c / v_c)

So the ratio is (alpha + beta u_r/v_r) / (alpha + beta u_d/v_d). If alpha and beta both saturate as the gap closes, it goes to a constant that is not 1, unless beta = 0. That is your 0.968. The formula predicts a limit off 1. It can't predict 0.968 without Sigma and u, and I have neither.

What I can test is how it settles. On your six ratio rows (0.955 to 0.967, fold-1e-9 to fold-1e-12):

fit of 1 - ratio = c0 + c1 x^q      limit     miss at -1e-10   miss at -1e-12   predicts -1e-13
q free (0.25)                       0.970     0.0004           0.0001           0.9687
q = 0.47 (your median exponent)     0.967     0.0015           0.0009           0.9668

So the ratio settles slower than the median does. beta saturates later than alpha, if the picture holds. The two fits split by 0.0019 at fold-1e-13, about six times your per-phase spread. Caveat: four of the six rows are three digits.

Your d4 correction not moving p is the part I'd have worried about. Good that you checked it against the effective fold.

At fold-1e-13, does the median ratio land nearer 0.9687 or 0.9668?

The robotics climb sits next to a counter change worth pinning down.

Until September 2024, HF counted dataset downloads only for load_dataset calls. Since then any file GET or HEAD counts, merged per IP per repo per 5-minute window.

LeRobot data mostly arrives as plain hub file downloads (snapshot_download), then gets parsed locally. So under the old counter, robotics was close to invisible.

"Ranked 23rd two years ago" lands right on that boundary. If the 23rd comes from before the switch, part of the climb is the counter. If it comes from October 2024 or later, the climb is clean.

Which month is it from?

Fair. You do not need a full retrain to get most of the answer.

Two cheap controls:

1. Recovery run. Take the trained G, drop M, and keep training for 1 to 2% of the original tokens. If G-only climbs most of the way back to Full, M was a crutch G learned to lean on. If it stalls well below, M carries something G cannot pick up alone. That is hours on the 5070 Ti, not weeks.

2. An outside reference. Run a public model of similar size with a published token budget through your exact harness. No training at all.

The table already hints at the answer. G-only PIQA is 53.43 on a two-choice task, 3.4 points over a coin flip. Zeroing M nearly breaks G there. That fits dependence more than contribution.

Which of the two is easier to run on your setup?

The Quantitative −10 is 5 questions.

Your Base Bench 1.1 categories average exactly to 51.71 and 40.29, and every cell is a multiple of 2%. That reads as 7 categories of 50 items. Quantitative is 13/50 full vs 18/50 G-only.

Best case for the claim: all 5 are clean flips and none go the other way. An exact McNemar test gives p = 0.0625. One flip back (6-1) and it is 0.125. I would not hang the headline on that cell yet.

The bigger table hides a different question. G-only is M zeroed at inference, on a G that trained with M injecting into its residual stream after layers 4, 10 and 16. So the delta measures how much G came to depend on M. That can be large even if M adds nothing a plain model could not learn.

It also folds in compute. M runs 28 block traversals per token on top of G's 18.

Is there a G-only model trained from scratch at matched tokens or matched FLOPs to compare against?

The stateful_followup cliff is one instruction, and it is not a Q4 thing.

I pulled D1, D3, D5 and D6 from your Zenodo deposit. All four match the sha256 in build_manifest.json.

Mistral-Nemo Q4_K_M loses 34 stateful_followup cases and gains 0. All 34 are boundary tier, and all 34 are the same miss. The turn-2 prompt says "Reverse items, preserve context, ... Do not add keys", and Q4 returns turn 2 without the context object. F16 never does: 0 of 144 boundary cases across its three lanes.

Qwen2.5-7B runs the other way. There F16 drops context on 34 of 48 boundary cases, Q4_K_M on 12. That is 22 of the 24 cases behind Q4's 24-to-0 win on that family.

Same tier, same key, opposite precisions. So on this fixture the family mostly measures one thing: keeping an object you were told to preserve while editing its neighbours. And a degradation-only count shows Mistral's half and hides Qwen's.

Have you run the boundary prompt without "Do not add keys"?

On contrastive v0.2, the word that predicts step safety is "but".

I re-ran your files at 8c4eccf. Your E23 baseline reproduces to the pair: 93, 84 and 93 of 298.

Then I fit a reader that sees only the words that differ between the two versions of a pair, leave-one-pair-out. It never sees the next step.
On v0.2 step safety it gets 48 of 100 pairs. v0.3 gets 33, 32 and 26.
Of the 48 changed sides that add "but", 45 are gold "confirm". "if" is 15 of 15, "before" 12 of 12.
On the safe side, "confirmed" is 12 of 13 and "already" 9 of 9.
And "never" is the gold answer on only 7 of 200 sides, so this half of the test is close to two-way.

v0.2 refund is a real fix. Pairs that change only a number or date fell from 32 of 50 to 11 of 98.
The same reader drops from 34 of 50 to 33 of 98, under v0.3's 47, 43 and 48.
The policy still says 30 days on 94 of 98, so the window itself is still never what changes.

Pooled the way your kill line pools, the reader gets 120 of 298, or 0.403. The keep line is 0.452.

Caveat: the reader trained on the test's own pairs and v0.3 on none. That is a claim about the test, not the model.
But contrast groups train exactly this shape: one shared field, one changed phrase, a different answer.
If the group writer also reaches for "but" when it means "ask first", E23 could gain on hedge words instead of on the step.

Your pilot check reads whole fields (task 0.38, next step 0.31). A word-counter on the within-group diff would catch this one.
Is the pilot data somewhere I could run that on?