Red (HF)

redaihf

·

AI & ML interests

(SO)MPOA decensoring, residual refusals, contextual ethical realignment

Recent Activity

reacted to AC-723's post with 🚀 about 2 hours ago

MixtureKit 1.1 is out 🚀 Turn the fine-tuned models you already have on the Hub into one Mixture-of-Experts model. pip install mixturekit Give it a base model and its fine-tunes (math, code, a language...). MixtureKit merges them, trains the routers that pick an expert for every token, and shows you what they learned: 🧩 3 composition methods behind one YAML: Branch-Train-MiX, Mixtral-style MoE, Branch-Train-Stitch 💻 Merges stream the experts tensor by tensor: they run on a laptop 🎯 Router training with a load balancing loss, optional LoRA 🔍 A routing visualizer: in the image, two Nile 4B experts split a mixed Egyptian sentence, Arabic script to one expert, Arabizi to the other 📦 Export to Mixtral / Qwen3-MoE, then GGUF for llama.cpp 🤗 mixturekit push uploads the result with a generated model card ✅ Llama 3, Qwen 2.5/3/3.5, Gemma 3/4 experts; tested on transformers 4.52 to 5.19 In the paper, a BTX model built with MixtureKit outperforms the dense baselines on multiple benchmarks. ▶️ Build, train and visualize an MoE in 15 min on a free Colab GPU: https://colab.research.google.com/github/MBZUAI-Paris/MixtureKit/blob/main/examples/build_your_own_moe.ipynb 📄 Paper: https://huggingface.co/papers/2512.12121 💻 Code: https://github.com/MBZUAI-Paris/MixtureKit 📚 Docs: https://mbzuai-paris.github.io/MixtureKit/ Work with @guokan-shang and @oe2015 at @MBZUAI-Paris 🙌

new activity about 2 hours ago

Vortex5/Fenrir-X-26B-A4B:Reasoning models

updated a collection about 5 hours ago

View all activity

Organizations

redaihf 's datasets 1

redaihf/goetia13_frankenmerge_sample

Viewer • Updated Aug 7 • 34 • 7 • 1