VIDRAFT_LAB

MRI(Model Region Imaging)
MRI stands for Model Region Imaging, a framework for scanning a black-box LLM to detect and visualize functional regions such as language, mathematics, reasoning, and coding.

In response to community requests, I created Darwin-4B-Opus, a lightweight Gemma-4-4B-based Darwin hybrid/evolution model with reasoning enhanced through Claude Opus distillation, and released it on Hugging Face.

Model: https://huggingface.co/FINAL-Bench/Darwin-4B-Opus
Demo: https://huggingface.co/spaces/FINAL-Bench/Darwin-4B-Opus

After small models make nemotron-120

https://huggingface.co/blobbybob/Nemotron-3-Super-120B-A12B-BF16-Claude-4.6-Opus-Reasoning-Distilled

Yes, that is exactly the plan.

I will first work on the smaller Darwin models, and after that I also want to move toward a Nemotron-120 class model. Thank you for the suggestion and support.

Thank you, that is very helpful feedback.
A 1B–4B Darwin is definitely an interesting direction, especially for lightweight and local use cases.
I will seriously consider it.

We already have a smaller model here: https://huggingface.co/FINAL-Bench/Darwin-9B-Opus

Do you think an even smaller one is needed?

Good stuff. That actually sounds kind of interesting.

Just the least you could do is write the model card yourselves such that you catch these things. I only continued reading because I was like "wait I think I might see the vision" but nothing came of it. That's why I got so frustrated. The least we can do for each other is read the things we post. I'd be pretty disappointed if the model card sold my work short so hard.

I acknowledge I went a little far, it's just tiring to be in this space sometimes.

Looking forward to the detailed report.

Appreciate the honesty, and the critique was deserved. The model card has been rewritten with proper technical definitions and methodology. MMLU-Pro results are in progress and will be published soon. Thanks for pushing us to do better.

Thanks for the detailed critique. Genuinely appreciate someone reading the model card this closely.

Some points are fair, some are misunderstandings. Let me address them.

Evolutionary merging isn't new:
Correct. mergekit already has an evolve feature, and we never claimed to have invented evolutionary merging. What Darwin adds is layer-level profiling of parent models before merging to reduce the search space. Whether this warrants a separate name is debatable, and we should have been clearer about what's new vs. existing methodology.

Architecture mismatch:
This is a misunderstanding. Both parents are Qwen3.5-35B-A3B — identical architecture. The Mother (Jackrong's Claude distill) is a LoRA SFT on the same base, not a different architecture. 'Text-only' refers to the training data (text reasoning chains only), not the model structure. The model card was unclear on this.

No technical definitions for medical terms:
Fair point. Here are the actual definitions:

• Model MRI = Layer-level profiling. We run a 1K-sample calibration dataset across 40 layers x 256 experts and measure expert activation frequency.
• Dead Expert = An expert with activation frequency below 5% on the calibration set. The Mother model showed 50-65% dead experts in middle layers, consistent with LoRA SFT updating only a subset of parameters.
• Routing Entropy = Shannon entropy of the router's softmax distribution. Healthy range for top-8-of-256 is 3.0-4.5 bits. Low entropy means the router is collapsing to a few experts.
• MRI-guided merging = Layers with high dead-expert ratios get higher Father (base model) weight. Healthy layers keep more Mother weight. This is the per-block ratio constraint.

We should have led with these definitions instead of metaphors.

'Impossible by human intuition' is overstated:
Agreed. Communities like Allura already do layerwise ratio tuning. What we meant was that automated search across 40 layers x multiple parameters is faster than manual tuning — not that humans can't get there.

Model card quality:
Fair. We'll rewrite it with formal definitions, reproducible methodology, and ablation results.

On using mergekit:
Darwin uses mergekit as its merge backend. Intentionally. mergekit is excellent infrastructure, and Darwin adds the evolutionary loop and diagnostic layer on top. Using good tools is not a contradiction.

We're currently running MMLU-Pro benchmarks and will publish full results when complete. Based on this feedback, we'll overhaul the model card to be technically grounded.

Appreciate the sharp critique. It helps make the next version better.