Existing methods — GPTQ, AWQ, llama.cpp's k-quants — minimize empirical loss heuristically. None of them prove they are optimal in any information-theoretic sense. ICRB-Q builds a quantization scheme that is provably optimal via the Cramér-Rao lower bound (CRB): no unbiased estimator of a weight can have lower variance than [F(θ)]⁻¹, where F is the Fisher information matrix.
sad grug news: I have burnt a lot of GPU credits already just making grug and it’s variants (I do not own a workstation yet) and grug 35b has a pretty bad issue but if I continue burning GPU credits to fix it then it will take even longer to get the workstation I am working for… so the fix may take a few days or never happen.
The error: I tested it in opencode after benchmarks were good and after release (my mistake) and it had pretty bad repetition failure and simply didn’t work. so.. yea.