🤝 Open to Collab
Andrei Ovcharenko
Kreout
AI & ML interests
Java backend development, Spring Boot, Kafka, infrastructure and automation. Interested in local AI tools and practical developer workflows. Open-source developer tools: https://github.com/krotname.
Recent Activity
repliedto SeaWolf-AI's post about 13 hours ago
We pretrained a 325M language model from zero on an inference-only NPU
The accepted view is that inference-only NPUs cannot train, let alone pretrain a language model from scratch. We put that view to a direct test on the Rebellions ATOM-Max, a commercially available Korean NPU released in 2025 and designed for inference. We built our own training loop that runs backpropagation on the chip itself. Starting from random initialization, we pretrained VKORE-325M-NPU-Base, a 325.6M parameter model, on 5 billion byte tokens of FineWeb-Edu. The run took 76,292 steps over about four days and finished without a single divergence. No pretrained weights were used.
To make the comparison fair, we trained the same model on a B200 GPU with identical initialization, data and learning-rate schedule. On 1,024 windows of text that neither model ever saw, the NPU reached a loss of 0.7993 and the GPU 0.7994. The window-by-window difference is -0.0001, well inside the noise. The two models are indistinguishable.
This is seven times larger than our first NPU model, VKORE-46M, and the first time this hardware has carried a real pretraining run to GPU quality. Please note that the hardware alone does not make this possible: the NPU and its standard software are built for inference, and training on it depends on our own training stack and stability technique, which are developed in house and under patent application. Buying the chip does not by itself enable training. Both sets of weights and the evaluation script are public, so anyone can verify the result in minutes.
📄 Technical report: https://huggingface.co/blog/FINAL-Bench/pretraining-a-325m-language-model-from-zero-on-an
🤗 Models: https://huggingface.co/FINAL-Bench/VKORE-325M-NPU-Base
#NPU #Rebellions #Pretraining #VKORE repliedto ProCreations's post 1 day ago
Nanbeige4.3-33B DFlash2 speculative drafter ⚡️
1.76× faster decoding on BF16, 1.65× on FP8, 1.51× on NVFP4. Agentic coding ~2.6×. Same outputs: exact rejection sampling. This is an early result, an improved version with higher acceptance is in training.
SGLang 0.5.21, drop-in:
https://huggingface.co/ProCreations/Nanbeige4.3-33B-DFlash2 reacted to juiceb0xc0de's post with ❤️ 2 days ago
You built a model. Wanna know what lights up inside it?💡
I put a little juice in the box so you can find out!🧃
I built Cartographics, a HF ZeroGPU Space for building neural activation atlases of language models.
I wanted to make the tools I use to explore model internals accessible to people building their own LLMs. Punch in a Hugging Face model ID, run the compatibility check, and start a capture. Prompt formatting and the full corpus are handled automatically.
The atlases help you investigate questions like:
- Which layers and channels respond most strongly to different kinds of prompts?
- How do activation patterns vary across code, math, reasoning, and other topics?
- Which features activate together, and where do unusually strong activations appear?
The census contains 8,965 prompts, giving you a shared basis for exploring different models.
GPU capture runs through ZeroGPU using your account’s allowance. I handle the CPU analysis separately. This release supports compatible architectures and models that fit the available GPU memory.
I’m building this independently, and I’d love to see what models builders explore with it.
What would you want to investigate in your own model?
Try Cartographics Beta:
https://huggingface.co/spaces/juiceb0xc0de/cartographics
https://huggingface.co/datasets/juiceb0xc0de/bella-bartender-gemma-e4b-cgs-atlas
If you make an atlas with us, leave the Space a ❤️. You're helping other people discover mechanistic interpretability the way I wish I did.
Prerequisites:
Models under 10B parameters (20GB of weights in bf16)
No trust_remote_code models, for now.
Preflight catches edge cases. Don't be scared to test it out. It's free to check.
Sign in with Hugging Face. The capture runs on your ZeroGPU quota, the atlas build runs on ours.Organizations
