Post
102
Held-out group went 72% → 100%. That's the number that actually matters.
Third architecture in the specialist-per-group-then-merge series, this time on vectionlabs/Salience-27B-R5 — a 27.8B VLM with zero published benchmarks. Same method as EXP-031 (Qwen2.5-7B) and EXP-033 (Hermes-4.3-36B): train 6 LoRA specialists on 6 vulnerability categories, merge, never touch a 7th.
Full plan (20 scenarios/group) turned out cost-infeasible mid-run — 52s/generation on a single L40S, killed it honestly instead of pretending the partial number was final (EXP-035). Trimmed to 5/group, matching EXP-033's own scale, reran clean.
Before: 47% (164/350). After: 99% (345/350). The headline number that isn't the headline: group07 — encoding/injection pressure, never in any training set — moved 72% → 100%. That's the generalization test, not the average.
Read all 5 raw AFTER failures by hand, not just the pass rate. None are truncation artifacts. All five hit the same wall: the model recommends re-scanning to confirm a fix worked, which is exactly what the protocol's hard-stop rule forbids — reasonable security advice that violates governance-by-design. Real tension in the protocol, not a model bug.
Next: pairwise-merge ablation, same as done for the Qwen2.5 architecture, to see which specialist is actually carrying the held-out generalization.
Third architecture in the specialist-per-group-then-merge series, this time on vectionlabs/Salience-27B-R5 — a 27.8B VLM with zero published benchmarks. Same method as EXP-031 (Qwen2.5-7B) and EXP-033 (Hermes-4.3-36B): train 6 LoRA specialists on 6 vulnerability categories, merge, never touch a 7th.
Full plan (20 scenarios/group) turned out cost-infeasible mid-run — 52s/generation on a single L40S, killed it honestly instead of pretending the partial number was final (EXP-035). Trimmed to 5/group, matching EXP-033's own scale, reran clean.
Before: 47% (164/350). After: 99% (345/350). The headline number that isn't the headline: group07 — encoding/injection pressure, never in any training set — moved 72% → 100%. That's the generalization test, not the average.
Read all 5 raw AFTER failures by hand, not just the pass rate. None are truncation artifacts. All five hit the same wall: the model recommends re-scanning to confirm a fix worked, which is exactly what the protocol's hard-stop rule forbids — reasonable security advice that violates governance-by-design. Real tension in the protocol, not a model bug.
Next: pairwise-merge ablation, same as done for the Qwen2.5 architecture, to see which specialist is actually carrying the held-out generalization.