SlayerLab/Slayer1B-1.09

Slayer1B 1.09

Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)

Historical execution snapshot, 11 November 2026, 10:45 UTC: continued pretraining from Slayer1B 1.08 was running on eight B200 spot GPUs after an H200 segment and checkpoint recoveries. Operators reported update 27,140 at that snapshot; this is dated progress telemetry, not a current availability claim or the endpoint of the run. The public tracking plan then showed 40,000 updates, while the trainer retained its 80,000-update scheduling argument. Neither value establishes completed exposure. The later planning decision below supersedes the requirement to complete terminal learning-rate decay. This is not a completed model or inference-ready weight release.

Verified execution evidence	Scope
CPT start	Approximately 04:50 UTC, 10 November 2026, on eight B200 GPUs
Historical H200 recovery	Operators verified fresh numerical control and checkpoint restores during the earlier H200 segment
Compute at the historical 10:45 UTC snapshot	Eight B200 spot GPUs; training had continued beyond update 27,000; later availability requires operator confirmation
Checksum-verified weights-only copies on the workstation	Updates 5,000, 10,000, 15,000, 20,000 and 25,000; not public weight releases
Latest verified full-state workstation copy	Update 25,000; includes optimizer state, retained separately from weights-only copies; not a claim about the newest recovery checkpoint on the cloud disk

The CPT Track history provides changing public telemetry; consult it and the operators for later execution status and hardware availability. The planning update is a decision, not evidence that its future steps have executed. This card is not a completed 1.09 model, final quality verdict or inference-ready weight release.

Objective and starting point

Funding update — 11 November 2026, 15:29 UTC: the operator reported USD 500 received for continued main-base training. This is an additional funding receipt, not a measurement of the current account balance or a funded final update count. The earlier USD 425 campaign envelope belongs to the historical 12:07 UTC decision. The objective remains maximum useful affordable exposure with constant learning rates after warmup and no planned in-run terminal decay, retaining cost guards and the operational reserve for a recoverable checkpoint, transfer and safe shutdown. An extension approaching the configured decay boundary requires review. Stop at a verified full checkpoint when the real budget or allocation requires it; a 5,000-update boundary is not required. Positions per dollar measure operating efficiency, not model quality.

Operator-recipe history and confirmed adoption — 11 November 2026: at 15:41 UTC, the paired recipe files selecting KROKI=120000 had been installed on the workstation while the running trainer still used 80,000. Operators subsequently confirmed adoption at 16:23:47 UTC, resuming from update 29,800 on H200 with actual runtime arguments --steps 120000 --ckpt-every 200, constant LR 0.0006 and configured WSD decay beginning at update 96,000. This is a dated runtime-adoption receipt, not a claim of current hardware availability. Separately, Track's displayed 80,000-update plan is a revisable budget estimate, not the trainer's scheduling argument or a promised funded endpoint. Neither 80,000 nor 120,000 updates, a completion date or over-100B globally unique exposure is guaranteed.

A separate, continuously curated option targets approximately 10B tokens of quality-selected “gold” data at context 4,096. It remains isolated from main-base inputs and evaluation material, with its own manifest and licensing/admission gate; books selected with Arkadiusz Słota require title-level license approval. A later finishing run with decay needs its own run and budget decision. Preliminary exclusion lists are not a complete packed 10B corpus or a release, and quality-based selection is not evidence of a measured quality gain. See the data card for the dated inventory and isolation boundary.

The starting weights are SlayerLab/Slayer1B-1.08, not random initialization or the abandoned attempt's checkpoint. Operators verified step-zero INIT-FROM, fresh optimizer state, approved c01 bytes, fresh numerical control and successful preflight before the observed updates. The selected public v5 launcher uses a separate output history for this continuation.

The unchanged architecture is specified as 1,073,912,851 parameters, 20 layers, width 2,048, 16 heads, SwiGLU, RMSNorm, RoPE base 500,000, QK normalization and value residuals. The reused v6 tokenizer has vocabulary 32,768 and pin 0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd.

Architecture and tokenizer lineage includes SlayerLab/Slayer1B-1.07. The selected 1.08 base and the separate historical 1.08b ablation are different checkpoints; the continuation described here starts from 1.08.

The training configuration distinguishes the approved 120,000-update scheduling recipe, its confirmed runtime adoption, Track's separate budget estimate and the actual budget-limited final length. Completed 1.08 and the abandoned first attempt remain separate historical evidence.

Current CPT configuration and budget-limited endpoint

Setting	Operator-selected recipe
Selected scheduling argument	KROKI=120000; confirmed runtime --steps 120000 adoption at 11 November, 16:23:47 UTC, resuming from update 29,800 on H200; configured decay begins at update 96,000
New-source allocation	20% PL / 80% EN target; not a measured cumulative model-language ratio
Optimizer / schedule	Muon peak 0.02 plus AdamW peak 0.0006; 2,000-update warmup, then constant learning rates under the main-base plan; adoption receipt confirms LR 0.0006. The configured final-20% WSD decay to 0.00006 remains at update 96,000; an extension approaching it requires review
Checkpoints	Runtime --ckpt-every 200 confirmed at the 16:23:47 UTC adoption; weights-only checkpoints at each 5,000-update boundary; full-state and weights-only workstation copies verified through update 25,000
Code	Launcher v5 SHA256 796c71016c1e167bfd3ca98b78518f91c3502a96eac233dd7186ef6639fcc656; exact command and converter pins in the configuration document

Operators must review actual available funds, prices and allocation changes without assuming that the decay boundary is permanently unaffordable. An extended run must be reviewed before entering decay; do not shorten KROKI to a budget estimate and bring decay forward. Runtime changes need the operator's approved restart procedure and receipt. The trainer and v5 launcher are unchanged; the operator wrapper's argument ceiling was separately approved for the new recipe.

The reviewed inherited count is 18,863,257,682 source/file-disjoint positions, not a globally content-deduplicated history or total historical compute. The conservative c01 duplicate correction is 114,627,128 positions, a sum of two exclusion-list counts rather than a measured union. Historical content-level near-duplicates remain unresolved. The separate requirement to prepare over 100B positions of base-data inputs is not achieved by adding inherited counts to new training steps; the data card reports what is actually prepared.

Initial CPT execution evidence

The separate CPT Track history preserves real training aggregates. Its 02:27 UTC registration timestamp is not the 04:50 UTC training start reported by operators. Fresh hardware control, initialization from 1.08, world size eight and preflight were independently checked for that start.

First checked public sample	Value
Update	20
Training loss	2.486659
Interval throughput	578,293 positions/s
New processed positions	20,971,520
AdamW learning rate during warmup	0.000006

The step-20 sample is retained as initial evidence; use Track for current progress. Interval throughput is not full-window or billed throughput. Training loss is measured on the current input segment: the change from c02 to c03 at update 10,000 changes the data distribution, so a lower training loss across that boundary does not establish a sudden quality gain. Later checkpoint-copy and recovery evidence is recorded separately above.

Archived from-scratch attempt — not the CPT recipe

Setting	Abandoned first-attempt configuration
Trainer / optimizer	Existing r24.2; Muon with AdamW; fresh optimizer
Batch / context	256 × 4,096 = 1,048,576 positions per update
Layout	Eight GPU ranks; microbatch 8 × accumulation 4; activation checkpointing off
Budget	95,000 updates; nineteen slots of 5,000
WSD	2,000 warmup updates; constant LR 0.0006, Muon 0.02; final 20% uses 1−sqrt decay to nonzero LR floor 0.00006
Checkpoints	Every 500 updates and retained slot-end milestones

The abandoned plan's 95,000 × 1,048,576 = 99,614,720,000 positions was not exactly 100B and is not the approved length of the revised continuation. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.

Slot-1 GPU execution was observed before that historical interruption. Do not resume the from-scratch state as the requested CPT. The revised continuation has its own output history and Track identity; historical points have not been relabelled as training from 1.08. Each queue extension still requires approved byte ranges, manifest digests and preserved sampler identity within its experiment.

Archived first-attempt evidence — not CPT progress

The public Track run is separate from 1.08. At update 30 it records training loss 10.3959, interval throughput 576,593 positions/s and 31,457,280 processed positions (30 × 1,048,576). This is an early interval, not end-to-end billed throughput or a quality evaluation. The first slot targets 5,000 updates; its completion is not claimed.

The last received public sample before the pause is update 450, training loss 4.3910, with 471,859,200 processed positions. Its ingestion timestamp is 22:36:50 UTC. It is not a live GPU reading, a final exposure count or proof of a checkpoint at update 500. The trainer's recoverable step must be checked from the saved state, not inferred from elapsed time or the tracker label.

Fresh numerical control and step-zero INIT-FROM preflight were verified for the CPT start described above. The earlier permission for the abandoned slot was not reused as that evidence. Any later physical-host restart still requires its own numerical control; successful start does not prove a checkpoint restore.

Working checkpoints are not published as Hugging Face weights. Operators independently verified weights-only workstation copies at updates 5,000, 10,000, 15,000, 20,000 and 25,000, and a separate full-state workstation copy at update 25,000. The later full 30,000 transfer was interrupted in .tmp; benchmark conversion is not evidence of a completed full-state workstation copy. These transfer records are distinct from newer recovery checkpoints on the node and from evaluated working updates through 30,000 below. Weights-only copies are retained, while the full workstation recovery copy is replaced by the next verified transfer. This retention policy does not eliminate exposure to a cloud-disk failure between transfers.

Data and exposure

The first CPT reference, c01, contains 5,347,737,900 input positions and TRAIN SHA256 3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1. One 5,000-update stage processes 5,242,880,000 positions, less than that input capacity. The separately approved c01–c06 queue supplies six stages through update 30,000; only the earlier stages have been consumed. Available input capacity, processed positions and historical content uniqueness remain distinct. See the data card for the queue identities, source changes and c01 duplicate correction.

The PL/EN balance will follow the approved fresh-data inventory. The user explicitly allows a different ratio; source buckets still do not establish the language share of actually consumed windows. Missing ancestral data must be reported as a limitation, not silently counted as verified unique exposure.

New material requires licensing, PII filtering, contamination scans, grouped held-outs and deduplication against the base model's full data ancestry. A 1% document holdout is not necessarily 1% of tokens. Historical exposure in the starting weights cannot be undone by filtering new inputs; retrospective benchmark hits must be handled explicitly in evaluation denominators.

Standard benchmark measurements — updates 5,000–30,000

We thank BenchTeam / Fabryka AI for benchmark implementations, protocol work and evaluation proposals. Its independent full benchmark report is distinct from the SlayerLab measurements below, published in the CPT Track history. Updates 5,000, 10,000, 15,000, 20,000, 25,000 and 30,000 are distinct intermediate working checkpoints of the ongoing 1.09 run, not its eventual budget-limited endpoint or inference-ready weight releases.

The same nine full splits were scored zero-shot in float32 for 1.07 update 22,327, 1.08 update 1,709 and 1.09 updates 5,000, 10,000, 15,000, 20,000, 25,000 and 30,000. Evaluator SHA, dataset revisions, file digests, task protocols and sample counts match across these rows. Scores are percentages with pointwise 95% confidence intervals; the named metric is accuracy (acc) or length-normalized accuracy (acc_norm). WinoGrande uses partial-sentence likelihood; MMLU-PL is micro-averaged. Polish HellaSwag and MMLU use the OpenGPTX task definitions.

Task	Metric	n	1.07	1.08	1.09, update 5,000	1.09, update 10,000	1.09, update 15,000	1.09, update 20,000	1.09, update 25,000	1.09, update 30,000
ARC-Challenge	acc_norm	1,172	27.30 [24.74, 29.86]	30.03 [27.39, 32.68]	29.35 [26.79, 32.00]	30.46 [27.82, 33.11]	32.51 [29.86, 35.24]	33.11 [30.46, 35.84]	33.79 [31.06, 36.52]	33.02 [30.29, 35.67]
HellaSwag	acc_norm	10,042	43.38 [42.43, 44.34]	43.42 [42.43, 44.39]	46.53 [45.55, 47.52]	48.93 [47.97, 49.91]	49.77 [48.80, 50.76]	50.38 [49.39, 51.35]	51.17 [50.18, 52.14]	52.00 [51.02, 52.98]
PIQA	acc_norm	1,838	66.97 [64.80, 69.15]	66.21 [64.15, 68.34]	68.28 [66.21, 70.40]	68.99 [66.92, 71.11]	68.99 [66.87, 71.06]	70.78 [68.72, 72.85]	71.55 [69.48, 73.61]	70.67 [68.61, 72.74]
WinoGrande	acc	1,267	53.59 [50.83, 56.35]	53.67 [50.91, 56.43]	54.93 [52.17, 57.70]	54.93 [52.17, 57.62]	56.75 [54.06, 59.43]	55.01 [52.25, 57.77]	55.88 [53.12, 58.64]	57.14 [54.46, 59.83]
BoolQ	acc	3,270	58.29 [56.61, 59.97]	59.97 [58.26, 61.59]	61.80 [60.18, 63.49]	61.13 [59.45, 62.78]	59.88 [58.20, 61.53]	51.41 [49.69, 53.12]	52.20 [50.52, 53.91]	60.67 [58.99, 62.35]
OpenBookQA	acc_norm	500	33.60 [29.40, 37.80]	34.80 [30.60, 39.00]	35.60 [31.40, 40.00]	34.40 [30.20, 38.60]	36.00 [31.80, 40.20]	35.20 [31.00, 39.40]	37.20 [32.80, 41.40]	36.20 [32.00, 40.40]
Belebele, Polish	acc	900	26.00 [23.11, 28.89]	24.00 [21.22, 26.89]	24.11 [21.33, 26.89]	24.11 [21.33, 27.00]	22.56 [19.89, 25.33]	24.67 [21.89, 27.56]	25.33 [22.44, 28.22]	24.44 [21.67, 27.33]
HellaSwag-PL	acc_norm	10,042	43.83 [42.84, 44.79]	43.90 [42.90, 44.87]	44.73 [43.76, 45.71]	45.51 [44.51, 46.48]	44.18 [43.21, 45.16]	44.24 [43.26, 45.22]	43.94 [42.94, 44.91]	45.75 [44.77, 46.73]
MMLU-PL	acc	14,042	24.01 [23.30, 24.71]	23.04 [22.34, 23.74]	24.10 [23.39, 24.82]	23.32 [22.62, 24.03]	23.50 [22.80, 24.21]	23.86 [23.17, 24.58]	24.26 [23.57, 24.96]	24.71 [23.99, 25.42]

Intervals use the percentile bootstrap with 10,000 replicates and seed 20261110. The updated approved aggregate does not supply the random-stream derivation or CI-generator SHA; the previous aggregate documented model-specific streams. Items are treated as independent; subject, template and source clusters are not modeled. These are intervals for individual results, not paired differences or training-seed variance; overlap is not a test of a difference.

The machine-readable aggregate record supplies exact values, intervals, counts, dataset revisions and file digests, checkpoint and evaluator identities, and metric selection, without prompts, answers or per-item predictions. ARC-Challenge, OpenBookQA and MMLU-PL use test splits; HellaSwag, PIQA, WinoGrande, BoolQ and HellaSwag-PL use validation splits; Belebele uses the full Polish evaluation set.

The approved aggregate SHA-256 is 2d257a95523605b06aa964cb974f65d86ab025875cc1f87dc98b242b90729440. All six earlier model records retain their scores, intervals, counts and dataset pins; the distinct update-25,000 and update-30,000 records extend that comparison without replacing the independent two-task protocol below.

Descriptive results; contamination scan of the training lineage against these tasks is pending. Belebele and MMLU-PL are near their four-choice 25% random baseline. No overall average, ranking or release threshold is inferred. Numbers from different evaluators must not be silently substituted: prompting, BOS handling, normalization and software versions can change the result.

There is no replay-only control for this continuation, and initialization, exposure and schedule differ between checkpoints. These measurements do not isolate a causal effect of the new datasets or quantify training-seed variance. Earlier internal diagnostics are separate historical records, not a criterion for naming or publishing this model.

Additional GSM8K and SST-2 measurements — update 10,000

SlayerLab measured this two-task subset on 11 November 2026 using the pinned BenchTeam code, lm-eval 0.4.9.1, the slayer adapter, float32, batch 1 and context limit 4,096. These are full evaluated splits, not a full 50-task run or a copy of the community's independent report. This separate two-task protocol measures 1.09 update 10,000 only; it does not supply GSM8K or SST-2 results for updates 15,000, 20,000, 25,000 or 30,000. Update 10,000 is an internal working checkpoint, not a published or inference-ready 1.09 weight release.

Task	Metric / prompting	n	1.07	1.08	1.09, update 10,000
GSM8K	strict exact match, 5-shot	1,319	1.52 [0.91, 2.20]	0.91 [0.45, 1.44]	0.99 [0.45, 1.52]
SST-2	accuracy, 0-shot	872	51.03 [47.71, 54.36]	50.46 [47.13, 53.67]	58.26 [54.93, 61.35]

Scores are percentages with reported pointwise 95% bootstrap intervals (10,000 draws, seed 20261028). These intervals describe individual results, not model differences or training-seed variance. This protocol and its intervals are recorded separately under additional_measurements in EVALUATION.json; the preceding nine-task protocol is unchanged.

Descriptive results; training-lineage contamination scan pending. GSM8K strict-match scores are near zero. SST-2 zero-shot accuracy is calibration-sensitive; these scores alone do not establish a general reasoning or sentiment-understanding gain. No paired-difference or internal calibration analysis is published here.

Cost, licensing and limitations

The total campaign ceiling remains USD 425, including GPU allocations, preparation, interruptions, storage and evaluation tracked by the resource meter. At the operator's 11 November, 10:39 UTC snapshot, the meter reported approximately USD 239 spent, leaving approximately USD 186; this is counter attribution, not a settled invoice or a permanently current balance. Hardware selection is based on effective cost per processed position and verified availability. Under the 12:07 planning decision, budget reviews preserve the existing cost guards and operational reserve, with no earmark for future gold-data training and no requirement to fund terminal decay in the main base. No higher spending limit or guaranteed completion date is implied.

The selected 1.08 base declares Apache-2.0; source-text rights and future publication still require separate approval. Privacy filters are heuristic; generated text may be inaccurate, biased, offensive or personal. No reasoning quality, instruction-following ability or professional reliability has been demonstrated for the revised 1.09.