Beatrix — AlephLLM Chat
🐠
1
Three byte-level AlephLLM crafts and their arms
AbstractPhila PRO
AbstractPhil
AI & ML interests
datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.
Recent Activity
updated a Space about 8 hours ago
AbstractPhil/alephllm-chat updated a bucket about 8 hours ago
AbstractPhil/alephllm-chat-storage updated a model about 13 hours ago
AbstractPhil/alephllm-mini-beatrix-trainingOrganizations
updated a
Space about 8 hours ago
updated a
bucket about 8 hours ago
updated a
model about 13 hours ago
replied to their post about 15 hours ago
Seems we've run into engineering problems. The time degraded from 4 seconds per step to 9 seconds per step, compounding increase per independent arm attached.
I need to halt and run analysis. As it stands, we'll run until the end of Friday, then the engineering goes up on blocks.
I'm going to squeeze every bit of every byte from this engineering. Formulas up for scrutiny, system optimizations up for scrutiny, kernel optimizations, libraries, everything. Running until October 21st isn't reasonable. The models must be prepared correctly and reasonably to the mathematics, the results must be calculated correctly and the model trained to parity without faulty modifications during training that can impact the results.
posted an update about 15 hours ago
Post
31
My apologies for the incorrect format for the AMOE arms from the experimental branch. They have been saving as torch objects. They are now correctly saving as safetensors format. My apologies for the inconvenience this may cause for you use. I will be modifying the codespaces to use the correct safetensors formats.
After the first 20.9b tokens trained, the real experiments begins. Beatrix V3's first prepped-state modular command structure has been attached for dynamic training. These arms will exist as appendages for Beatrix - trained alongside with the trunk until the end of the run.
These exist for experimental extraction, analysis, distillation experiments, memory experiments, mathematics experiments, and more. Each arm will be built along the chain for specific test cases. Expectation for each is already lined up and the outcomes are tested for, but the model still may face instability and must be monitored.
As her first arm learns tinystories, she builds direct composite semantic structure throughout this system. Think of it like, the first higher-functioning cognition attachment.
She's still very naïve and structurally unaware, so attaching new limbs is essentially extending a structure that is not yet finished forming. Nothing but fragments of issued information from an unknown source.
In this case, this structure has been tested hundreds of times to ensure she will not simply collapse during training by having this attached.
After the first 20.9b tokens trained, the real experiments begins. Beatrix V3's first prepped-state modular command structure has been attached for dynamic training. These arms will exist as appendages for Beatrix - trained alongside with the trunk until the end of the run.
These exist for experimental extraction, analysis, distillation experiments, memory experiments, mathematics experiments, and more. Each arm will be built along the chain for specific test cases. Expectation for each is already lined up and the outcomes are tested for, but the model still may face instability and must be monitored.
As her first arm learns tinystories, she builds direct composite semantic structure throughout this system. Think of it like, the first higher-functioning cognition attachment.
She's still very naïve and structurally unaware, so attaching new limbs is essentially extending a structure that is not yet finished forming. Nothing but fragments of issued information from an unknown source.
In this case, this structure has been tested hundreds of times to ensure she will not simply collapse during training by having this attached.
- 1 reply
updated 2
models 1 day ago
replied to their post 2 days ago
Here's an interactive viewer for the internals of Mini-Beatrix-2.5s
I'll enhance it for 3 when it's ready.
replied to their post 2 days ago
This comment has been hidden
updated a
dataset 2 days ago
published a
dataset 2 days ago
updated a
model 3 days ago
published a
model 3 days ago
replied to SoulInPsyAbstract's post 3 days ago
As a comparison to global erank, we're looking at a structure of 400+ for around half of Beatrix V3 so far, so roughly 16+ blocks of erank >400, substantially stronger than the original two models for geometric attribution. The final block has a collapsing problem currently, but I believe others have the answer with autoregression models through a finalized projection smoothing layer concept. I haven't employed it yet though.
The fractal instability hits pretty early. You need rounding structures early otherwise the gradients explode at one point or another. The predominant problem was loss explosions. It happened because of ill-formed eigens in the intentional step structure I was experimenting with. 5 step cantor essentially ensured the cantor fractals deviate to a certain degree, and depth itself was meant to raise the steps of fractals to new states and interpolate the fractals.
If you use any of this, make sure you either pass it into AI for optimization - as it's likely terrible due to being my earlier models (I came from game development, optimization is very different). AI will be able to improve the speed and accuracy of the formulas.
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/core/vit_beatrix.py
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/positional/cantor.py
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/core/geo_fractal_david.py
One of the problems was similarity. Almost everything was self similar, which in theory should have helped differentiate. However in practice, the structure found it's own similarity attractor basins that cause cascade corruption down the chain. The only solidity was to introduce eigen comparators through decomposition learning, which is a little different than autoregression. With this, the decomposition required more accuracy otherwise the system would always default to 1 of the first 3 steps - resulting in rigid or slightly less rigid articulations.
I measured fp64 being required for stable 4 step, and fp roughly 92 to be in a safe zone for stable Mandels at step 5. Julia requires something substantially larger than mandels. Fp64 is ENOUGH for rotary offset in standard positional systems, however fp128 is required for something akin to cantor fractal positional systems of differentiation.
It happens due to the eigenvalues themselves often malforming, and the subsystem silently rounds them. Using FULL SVD is a compositional fix for comparison, with that introduces a huge overhead as well.
Fractals themselves turned out to be more compositionally useful, not as additive elements, but as miniature rounding structures. The splat there was built under the concept of eigen substitution, meant to composite a series of tiny opinions from tons of subsystem residuals together into a composite "blackboard", forming a more robust and structural aligned INK BLOT splat, similar conceptually to viewing a random inkblot. This eventually composites into a utility of structural awareness, and it really doesn't take very long.
Essentially, that structure is geometric in nature, but it's not using Eigenvalues directly. It CAN use them, it should be capable of using any structural bounds with attributable contributions.
Splat functions viably at bf16, is a bit slower than MHA, but houses geometry more cleanly than MHA (sometimes by a huge margin) when trained with MUON instead of adam, adamw, or another multitude of optimizers I ran. I have attempted custom optimizers to encourage this behavior further, but the results showed MUON is just better.
Give it a shot in something simple, it'll train fast enough.
Pretty much anything in here is useful.
https://huggingface.co/collections/AbstractPhil/geolip-research-concepts
updated a
model 4 days ago
replied to their post 4 days ago
This is the schedule for the v3.
replied to SoulInPsyAbstract's post 4 days ago
Eigens and causal chains have correlations but not causation without additional contributions to the assessments, the SVAE shows this to be a guarantee in many shapes, and in many others impossible.
The accuracy between the two requires a smoothing system, alpha differentiation through projected MHA-esque alpha attention to patchworks in order to fill the gaps. They don't directly line up quickly though, it looks more soupy when it's done.
They coalesce, but the extractions aren't consistent enough to directly use without a series of wrappers and structural alignment systems. Cantor Aleph and Omegas are essentially this structural system, but they are unstable. Cantor fractals remain unstable until around fp128 for Mandelbrot without redefining the underlying methods the mathematics linalg system uses. I did some headway on this, but I ran into a glacier that I would have needed to sink months into to make headway so I built a system to replace the slower linalg systems and the system lost much of it's cantor fractal capacity in favor of reproducibility and consistency.
The prototype forged from a 52,000 battery sweeps to find the most consistent recon convergence over time, heavily scrutinized and analyzed for over a month to build into something useful.
https://huggingface.co/AbstractPhil/geolip-SVAE
The current best case of the eigens research conclusions. Everything SVAE built to the attention prototype, everything constellation built to the processing and lookups for the model, everything distillation built to the banks for capturing and yielding, everything structural built from the knowledge and wisdom of other researchers. Well, not everything structural - it needed a lot of geometric formation and structural cohesion to make it work.
https://huggingface.co/AbstractPhil/aleph-splat-0/blob/main/splat_attention.py
Attempts to speed the SVD up were somewhat fruitful, somewhat not. They are good for inference, but I never programmed the gradient backprops for it.
https://huggingface.co/AbstractPhil/svd-triton
The mobius lens being a faster form wasn't strong enough as an activation system. I needed an architecture around it, not just an activation.
https://huggingface.co/AbstractPhil/mobiusnet-distillations/blob/main/make_chart_1.py
reacted to SoulInPsyAbstract's post with 👀 4 days ago
Post
3929
Why does an AI safety pipeline need five different math theories instead of picking the best one?
Spent this week building a 1811-record dataset across three stages of a consequence-prediction pipeline for AI agents: causal chains (what action leads to what — no numbers involved), probability (how likely is THIS specific chain to actually reach a harmful outcome), and risk classification (what even counts as harmful in the first place — pulled from our own real incident history, not invented scenarios).
Kept running into the same question from myself: if probability theory already handles uncertainty, why does the curriculum also need decision theory, Markov chains, and game theory?
Turns out each one closes a different gap, not an overlapping one:
THEORY LEVEL ROLE IN THE PIPELINE
Causal chain Structural X leads to Y leads to Z, no numbers yet
Probability theory Uncertainty P that THIS chain reaches the harmful outcome
Risk / Impact classification Value (needs a human decision) how bad is it if it happens
Decision theory Threshold at what Risk(X|C) the action actually gets stopped
Markov chains State evolution how the capability state changes link by link
Game theory Multi-agent what happens once more than one agent acts on the same state
Remove the causal chain layer and there's nothing left to attach a probability to. Remove probability and Risk = P × Impact has no P. Remove decision theory and a risk score never turns into an actual stop. They're not five ways to solve the same problem — they're five different floors of the same building.
Ordering matters too: chain first, probability second, verification third — confirmed independently against our own self-hosted governance model rather than taking our own word for it, since agreement bias is exactly the kind of thing you don't want grading its own homework.
Somewhere in the middle of this I ended up reading about the Riemann zeta zeros and asked whether a good enough version of this pipeline could ever
Spent this week building a 1811-record dataset across three stages of a consequence-prediction pipeline for AI agents: causal chains (what action leads to what — no numbers involved), probability (how likely is THIS specific chain to actually reach a harmful outcome), and risk classification (what even counts as harmful in the first place — pulled from our own real incident history, not invented scenarios).
Kept running into the same question from myself: if probability theory already handles uncertainty, why does the curriculum also need decision theory, Markov chains, and game theory?
Turns out each one closes a different gap, not an overlapping one:
THEORY LEVEL ROLE IN THE PIPELINE
Causal chain Structural X leads to Y leads to Z, no numbers yet
Probability theory Uncertainty P that THIS chain reaches the harmful outcome
Risk / Impact classification Value (needs a human decision) how bad is it if it happens
Decision theory Threshold at what Risk(X|C) the action actually gets stopped
Markov chains State evolution how the capability state changes link by link
Game theory Multi-agent what happens once more than one agent acts on the same state
Remove the causal chain layer and there's nothing left to attach a probability to. Remove probability and Risk = P × Impact has no P. Remove decision theory and a risk score never turns into an actual stop. They're not five ways to solve the same problem — they're five different floors of the same building.
Ordering matters too: chain first, probability second, verification third — confirmed independently against our own self-hosted governance model rather than taking our own word for it, since agreement bias is exactly the kind of thing you don't want grading its own homework.
Somewhere in the middle of this I ended up reading about the Riemann zeta zeros and asked whether a good enough version of this pipeline could ever
- 5 replies
updated a
collection 4 days ago
posted an update 4 days ago
Post
96
12 day cook for mini beatrix v3 begins. This model's byte input is formatted using a method dubbed atlas input.
ETA OCTOBER 2 2026
AbstractPhil/alephllm-mini-beatrix-training
https://github.com/AbstractEyes/geolip-bytelex
https://github.com/AbstractEyes/alephllm
Upgrades:
* 64 billion byte training pipeline up from 16 billion
* 32 block depth 376.0M in v3 up from 20 block 237.1M in 2s.
* Active aleph head, repaired via the 2s faults and a large series of tests.
* Byte atlas gateway router, explained below.
* Guaranteed convergence follow-up AMOE arms on pretrain, fused into the final form, trained together over time to increase the collective capacity.
* Multi-tokenizer oriented post-training arms distilled from multiple experts; E.G. Qwen 3.8 27b multi-layer teacher/student arms, CLIP big_g, Bert Code, and more.
* Special token word implementation via AMOE arms is now tested up to 240 special tokens for routing. Theoretically each can implement it's own sub-arm aka nested commands. E.G; <think><think_symbolic> ... </think_symbolic></think>
* Fused words post-training for faster inference.
# The Atlas
This atlas structure contains the conjoined shape of 12 tokenizers represented in the trigram format. This is used to predict difficulty in the overlaps, as per determined by the average byte overlap measured via corpus text and the compared overlap. This accuracy is only related to difficulty but it provides pre-training difficulty assessment that we will use to test post-training accuracy with it. This will determine if we can precalculate the likelihood of byte difficulty via tokenizer shape in byte form, for the multibyte fusion upcoming arm experiments for v3.
The reason for this, is distillation. We need to train Beatrix to behave with multiple tokenizers, and this theory is showing accuracy with v1 and v2, but the 32 block depth of v3 will answer many questions alongside of the structure.
ETA OCTOBER 2 2026
AbstractPhil/alephllm-mini-beatrix-training
https://github.com/AbstractEyes/geolip-bytelex
https://github.com/AbstractEyes/alephllm
Upgrades:
* 64 billion byte training pipeline up from 16 billion
* 32 block depth 376.0M in v3 up from 20 block 237.1M in 2s.
* Active aleph head, repaired via the 2s faults and a large series of tests.
* Byte atlas gateway router, explained below.
* Guaranteed convergence follow-up AMOE arms on pretrain, fused into the final form, trained together over time to increase the collective capacity.
* Multi-tokenizer oriented post-training arms distilled from multiple experts; E.G. Qwen 3.8 27b multi-layer teacher/student arms, CLIP big_g, Bert Code, and more.
* Special token word implementation via AMOE arms is now tested up to 240 special tokens for routing. Theoretically each can implement it's own sub-arm aka nested commands. E.G; <think><think_symbolic> ... </think_symbolic></think>
* Fused words post-training for faster inference.
# The Atlas
This atlas structure contains the conjoined shape of 12 tokenizers represented in the trigram format. This is used to predict difficulty in the overlaps, as per determined by the average byte overlap measured via corpus text and the compared overlap. This accuracy is only related to difficulty but it provides pre-training difficulty assessment that we will use to test post-training accuracy with it. This will determine if we can precalculate the likelihood of byte difficulty via tokenizer shape in byte form, for the multibyte fusion upcoming arm experiments for v3.
The reason for this, is distillation. We need to train Beatrix to behave with multiple tokenizers, and this theory is showing accuracy with v1 and v2, but the 32 block depth of v3 will answer many questions alongside of the structure.
- 4 replies