audio-cpp commited on
Commit
8d34c7f
·
verified ·
1 Parent(s): b0a1dfe

Document Voxtral Q4_K GGUF

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -114,7 +114,7 @@ for the full matrix and drift notes.
114
  | `VibeVoice-1.5B-GGUF` | `vibevoice-1.5b-bf16.gguf`, `vibevoice-1.5b-q8_0.gguf` | `vibevoice` | 16-bit pass, Q8 drift | MIT |
115
  | `VibeVoice-ASR-GGUF` | `vibevoice-asr-f16.gguf`, `vibevoice-asr-q8_0.gguf` | `vibevoice_asr` | 16-bit + Q8 pass | MIT |
116
  | `VoxCPM2-GGUF` | `voxcpm2-orig.gguf`, `voxcpm2-bf16.gguf`, `voxcpm2-q8_0.gguf` | `voxcpm2` | orig pass, 16-bit + Q8 ASR-match drift | Apache-2.0 |
117
- | `Voxtral-Mini-4B-Realtime-2602-GGUF` | `voxtral-mini-4b-realtime-2602-bf16.gguf`, `voxtral-mini-4b-realtime-2602-q8_0.gguf` | `voxtral_realtime` | 16-bit + Q8 pass | Apache-2.0 |
118
 
119
  ## Q8 Notes
120
 
@@ -125,6 +125,8 @@ for the full matrix and drift notes.
125
  tensors in Q8 in addition to the default converter selection. `conditioner.embed`,
126
  `cond_embed`, and Mimi conv tensors are not forced to Q8 because tested outputs
127
  drifted or the current conv path casts quantized conv weights back to F32.
 
 
128
 
129
  ## Usage
130
 
 
114
  | `VibeVoice-1.5B-GGUF` | `vibevoice-1.5b-bf16.gguf`, `vibevoice-1.5b-q8_0.gguf` | `vibevoice` | 16-bit pass, Q8 drift | MIT |
115
  | `VibeVoice-ASR-GGUF` | `vibevoice-asr-f16.gguf`, `vibevoice-asr-q8_0.gguf` | `vibevoice_asr` | 16-bit + Q8 pass | MIT |
116
  | `VoxCPM2-GGUF` | `voxcpm2-orig.gguf`, `voxcpm2-bf16.gguf`, `voxcpm2-q8_0.gguf` | `voxcpm2` | orig pass, 16-bit + Q8 ASR-match drift | Apache-2.0 |
117
+ | `Voxtral-Mini-4B-Realtime-2602-GGUF` | `voxtral-mini-4b-realtime-2602-bf16.gguf`, `voxtral-mini-4b-realtime-2602-q8_0.gguf`, `voxtral-mini-4b-realtime-2602-q4_k.gguf` | `voxtral_realtime` | 16-bit + Q8 pass; Q4_K quick check passed | Apache-2.0 |
118
 
119
  ## Q8 Notes
120
 
 
125
  tensors in Q8 in addition to the default converter selection. `conditioner.embed`,
126
  `cond_embed`, and Mimi conv tensors are not forced to Q8 because tested outputs
127
  drifted or the current conv path casts quantized conv weights back to F32.
128
+ - Voxtral Q4_K is smaller than Q8_0 and was faster in a quick CUDA path check,
129
+ with transcripts matching Q8_0 except for one capitalization-only difference.
130
 
131
  ## Usage
132