intellectlabs/Kepler-8B-Instruct-v2

🪐 Kepler-8B-Instruct — GGUF

Quantized GGUF builds of Kepler-8B-Instruct — ready to run anywhere llama.cpp does.

License: Apache 2.0 · Format: GGUF · Base: Kepler-8B-Instruct

Original Model · llama.cpp · Discussions

---

✨ What is this?

Kepler-8B-Instruct is an 8-billion-parameter assistant built by merging Qwen/Qwen2.5-7B-Instruct (85%) with deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (15%) using mergekit. The blend keeps Qwen2.5-Instruct's clean, reliable instruction-following as the foundation while folding in a touch of DeepSeek's reasoning-distilled behavior.

This repository packages that model as GGUF — the universal format for llama.cpp and everything built on it. One file, no Python environment, no CUDA setup. It runs on a gaming laptop, a Raspberry Pi-class board, or a headless server with equal ease.

Highlights:

• 🧩 A reliable instruction-following foundation with a light touch of reasoning-model flavor
• ⚡ Seven precision levels included, from a ~3 GB ultra-light build to full-precision F16
• 🔌 Drop-in compatible with llama.cpp, LM Studio, Ollama, koboldcpp, GPT4All, Jan, and text-generation-webui
• 🗂️ Native ChatML support, with a built-in default identity — ask "who are you" and it answers as Kepler by intellectlabs, in whatever language you ask in

🚀 Quick Start

llama.cpp

./llama-cli -hf intellectlabs/Kepler-8B-Instruct-GGUF:Q4_K_M -p "Explain quantum entanglement simply."

Or download a specific file manually:

huggingface-cli download intellectlabs/Kepler-8B-Instruct-GGUF Kepler-8B-Instruct-Q4_K_M.gguf --local-dir .
./llama-cli -m Kepler-8B-Instruct-Q4_K_M.gguf -p "Hello, who are you?" -cnv

LM Studio

Search intellectlabs/Kepler-8B-Instruct-GGUF directly in the LM Studio model search bar and download your preferred quant.

Ollama

ollama run hf.co/intellectlabs/Kepler-8B-Instruct-GGUF:Q4_K_M

📦 Available Quantizations

File	Quant	Size (approx.)	Quality	Recommended For
Kepler-8B-Instruct-Q2_K.gguf	Q2_K	~3.1 GB	⭐️	Extreme low-RAM devices only
Kepler-8B-Instruct-Q3_K_M.gguf	Q3_K_M	~4.0 GB	⭐️⭐️	Low-RAM systems, testing
Kepler-8B-Instruct-Q4_0.gguf	Q4_0	~4.7 GB	⭐️⭐️⭐️	Legacy-hardware compatibility
Kepler-8B-Instruct-Q4_K_M.gguf	Q4_K_M	~4.9 GB	⭐️⭐️⭐️⭐️	Best balance — recommended default
Kepler-8B-Instruct-Q5_K_M.gguf	Q5_K_M	~5.7 GB	⭐️⭐️⭐️⭐️	Higher quality, still efficient
Kepler-8B-Instruct-Q8_0.gguf	Q8_0	~8.5 GB	⭐️⭐️⭐️⭐️⭐️	Near-lossless, if you have the RAM
Kepler-8B-Instruct-F16.gguf	F16	~16 GB	⭐️⭐️⭐️⭐️⭐️	Full precision, GPU/server use

💡 New here? Start with Q4_K_M — it's the sweet spot most people use: small enough to run comfortably, strong enough to feel close to the full model.

🧠 Prompt Format

This model uses the ChatML template (inherited from its Qwen2 base):

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Most tools (llama.cpp -cnv, LM Studio, Ollama) apply this automatically via the embedded chat template — no manual formatting needed.

🛠️ How It Was Made

1. Merge — deepseek-ai/DeepSeek-R1-Distill-Qwen-7B linearly blended into Qwen/Qwen2.5-7B-Instruct (15%/85%) using mergekit, with the tokenizer taken from the base model.
2. Convert — merged safetensors converted to GGUF (F16) using llama.cpp's convert_hf_to_gguf.py.
3. Quantize — F16 GGUF quantized into multiple formats using llama-quantize, covering everything from ultra-compact (Q2_K) to near-lossless (Q8_0).

⚠️ Limitations

• This is a merge, not a model trained from scratch — behavior is a blend of its parent models and may inherit their quirks or biases.
• Lower-bit quantizations (Q2_K, Q3_K_M) trade quality for size; expect more inconsistency at extreme compression.
• Not evaluated on standard benchmarks yet — treat outputs with normal LLM caution, especially for factual or high-stakes use.

🙏 Credits

• Base models: Qwen/Qwen2.5-7B-Instruct, deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
• Merge tooling: mergekit
• Quantization tooling: llama.cpp

---

If this was useful, a ⭐️ on the repo helps others find it.