Instructions to use Vortex5/Violet-Starlight-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Vortex5/Violet-Starlight-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Vortex5/Violet-Starlight-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Vortex5/Violet-Starlight-12B")
model = AutoModelForCausalLM.from_pretrained("Vortex5/Violet-Starlight-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Vortex5/Violet-Starlight-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Vortex5/Violet-Starlight-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vortex5/Violet-Starlight-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Vortex5/Violet-Starlight-12B

SGLang

How to use Vortex5/Violet-Starlight-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Vortex5/Violet-Starlight-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vortex5/Violet-Starlight-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Vortex5/Violet-Starlight-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Vortex5/Violet-Starlight-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Vortex5/Violet-Starlight-12B with Docker Model Runner:
```
docker model run hf.co/Vortex5/Violet-Starlight-12B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Violet-Starlight-12B

Overview

Violet-Starlight-12B was created by merging Strawberry_Smoothie-12B-Model_Stock , Lunar-Twilight-12B, and Dreamstar-12B using a custom merge method.

Merge Configuration

models:
  - model: DreadPoor/Strawberry_Smoothie-12B-Model_Stock
  - model: Vortex5/Lunar-Twilight-12B
  - model: Vortex5/Dreamstar-12B
merge_method: smi_oni
chat_template: auto
parameters:
  k_core: 2
  strength_core: 1.0
  strength_nov: 0.35
  novelty_budget: 0.25
  consensus_core: 0.45
  consensus_nov: 0.30
  conflict_bonus: 0.15
  drop_cos: 0.10
  drop_min: 2
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: Vortex5/Lunar-Twilight-12B

Prose

I tested the model with an LLM using neutral prompts to summarize its narrative style.

Violet-Starlight tends toward clear, steady, emotionally direct storytelling. Its prose is grounded and approachable, delivering scenes in a linear, comprehensible flow with an emphasis on concrete action and plainly stated emotion. The model favors straightforward description over stylistic flourish, producing narratives that feel earnest, purposeful, and focused on keeping the reader oriented at all times. Its voice is consistent and unobtrusive—more practical than poetic—resulting in a dependable, accessible style that prioritizes clarity, momentum, and closure over ambiguity or experimental technique.

Intended Use

Designed for narrative-driven creative tasks.

📜 Storytelling Long-form narrative

🎭 Roleplay Character interaction

🖋️ Creative Writing Scenes & Ideas

Downloads last month: 6

Safetensors

Model size

12B params

Tensor type

BF16

Model tree for Vortex5/Violet-Starlight-12B

DreadPoor/Strawberry_Smoothie-12B-Model_Stock

Vortex5/Dreamstar-12B

Vortex5/Lunar-Twilight-12B

Merge model

this model

Quantizations

4 models