PrimeIntellect/inkling-jlens

Inkling J-space lens (n=250, all-layer)

A Jacobian lens (J-space) fit of thinkingmachines/Inkling (~950B-param MoE: 66 layers, d_model 6144, 256 routed experts/layer + 2 shared, hybrid SWA-512/global attention, sconv), fitted from the official NVFP4 checkpoint (thinkingmachines/Inkling-NVFP4).

J_ℓ = E[∂h_target,t′ / ∂h_ℓ,t] — the expected Jacobian of the final residual stream with respect to layer ℓ's residual stream, averaged over token positions and prompts.

Fit configuration

source layers	all 65 (0–64), target = layer 65
prompts	250 wikitext-103 (train, max_chars 2000), seq_len 128, 2-way prompt-sharded
weights	NVFP4 (block-16 e4m3 scales) resident, bit-exact fused dequant-autograd (bf16 compute, fp32 accumulation)
dim_batch	64 (96 backward passes/prompt)
rate	~526 s/prompt/node on 8×H200, 18.6 h/shard, 2 nodes
merge	exact n-weighted: J = (n₀·J₀ + n₁·J₁)/(n₀+n₁), verified ≤3.1e-5

Files

• inkling_jacobian_lens_n250.pt — merged lens (jlens JacobianLens payload: per-layer fp32 J[6144,6144] means for layers 0–64, target 65, n=250)
• shards/shard0of2.pt, shards/shard1of2.pt — the two disjoint-prompt half-lenses (125 prompts each; used for the split-half robustness checks — same-layer CKA median 0.997)
• merge_weights.json, convergence_shard{0,1}.csv — provenance
• cka/ — the recombined CKA space: inkling_cka.npz (65×65 CKA-vs-ρ matrix), inkling_cka_blocks.json (KTS-DP segmentation), inkling_cka_heatmap.png, summary.json