Document Voxtral Q4_K GGUF
Browse files
README.md
CHANGED
|
@@ -114,7 +114,7 @@ for the full matrix and drift notes.
|
|
| 114 |
| `VibeVoice-1.5B-GGUF` | `vibevoice-1.5b-bf16.gguf`, `vibevoice-1.5b-q8_0.gguf` | `vibevoice` | 16-bit pass, Q8 drift | MIT |
|
| 115 |
| `VibeVoice-ASR-GGUF` | `vibevoice-asr-f16.gguf`, `vibevoice-asr-q8_0.gguf` | `vibevoice_asr` | 16-bit + Q8 pass | MIT |
|
| 116 |
| `VoxCPM2-GGUF` | `voxcpm2-orig.gguf`, `voxcpm2-bf16.gguf`, `voxcpm2-q8_0.gguf` | `voxcpm2` | orig pass, 16-bit + Q8 ASR-match drift | Apache-2.0 |
|
| 117 |
-
| `Voxtral-Mini-4B-Realtime-2602-GGUF` | `voxtral-mini-4b-realtime-2602-bf16.gguf`, `voxtral-mini-4b-realtime-2602-q8_0.gguf` | `voxtral_realtime` | 16-bit + Q8 pass | Apache-2.0 |
|
| 118 |
|
| 119 |
## Q8 Notes
|
| 120 |
|
|
@@ -125,6 +125,8 @@ for the full matrix and drift notes.
|
|
| 125 |
tensors in Q8 in addition to the default converter selection. `conditioner.embed`,
|
| 126 |
`cond_embed`, and Mimi conv tensors are not forced to Q8 because tested outputs
|
| 127 |
drifted or the current conv path casts quantized conv weights back to F32.
|
|
|
|
|
|
|
| 128 |
|
| 129 |
## Usage
|
| 130 |
|
|
|
|
| 114 |
| `VibeVoice-1.5B-GGUF` | `vibevoice-1.5b-bf16.gguf`, `vibevoice-1.5b-q8_0.gguf` | `vibevoice` | 16-bit pass, Q8 drift | MIT |
|
| 115 |
| `VibeVoice-ASR-GGUF` | `vibevoice-asr-f16.gguf`, `vibevoice-asr-q8_0.gguf` | `vibevoice_asr` | 16-bit + Q8 pass | MIT |
|
| 116 |
| `VoxCPM2-GGUF` | `voxcpm2-orig.gguf`, `voxcpm2-bf16.gguf`, `voxcpm2-q8_0.gguf` | `voxcpm2` | orig pass, 16-bit + Q8 ASR-match drift | Apache-2.0 |
|
| 117 |
+
| `Voxtral-Mini-4B-Realtime-2602-GGUF` | `voxtral-mini-4b-realtime-2602-bf16.gguf`, `voxtral-mini-4b-realtime-2602-q8_0.gguf`, `voxtral-mini-4b-realtime-2602-q4_k.gguf` | `voxtral_realtime` | 16-bit + Q8 pass; Q4_K quick check passed | Apache-2.0 |
|
| 118 |
|
| 119 |
## Q8 Notes
|
| 120 |
|
|
|
|
| 125 |
tensors in Q8 in addition to the default converter selection. `conditioner.embed`,
|
| 126 |
`cond_embed`, and Mimi conv tensors are not forced to Q8 because tested outputs
|
| 127 |
drifted or the current conv path casts quantized conv weights back to F32.
|
| 128 |
+
- Voxtral Q4_K is smaller than Q8_0 and was faster in a quick CUDA path check,
|
| 129 |
+
with transcripts matching Q8_0 except for one capitalization-only difference.
|
| 130 |
|
| 131 |
## Usage
|
| 132 |
|