savoji/qwen3.5-0.8b-tablevqa-lora
Updated ā¢ 33
Sajjad Pakdamansavoji
savoji
AI & ML interests
VLM, VLA, LLM, Robotics, Computer Vision
Recent Activity
posted an update about 9 hours ago
APEX removes DMD's separate fake-score network for streaming video distillation: 4.4% less peak memory, 9.2% shorter iterations, and +30.0% net human preference on instruction following.
DMD needs an auxiliary network that tracks the generator's output distribution. APEX uses the generator itself. The original text condition generates, and the negated projected text (c_shift = -c) selects fake-velocity estimation with the same weights. The frozen teacher, Self-Replay Forcing and perceptual loss from S2-Avatar stay. The price is weaker motion.
š What changes
ā¢ One set of weights and, by default, shared AdamW states
ā¢ Inference uses only the generation path, then a one-step 2x Refiner
ā¢ The fake estimator is fitted to replay outputs; its clean-latent gap to the teacher gives a detached generator correction
š The numbers
ā¢ Vs DMD at matched settings: 102.7 to 98.2 GiB per GPU, 35.9 to 32.6 s per iteration
ā¢ Vs S2-Avatar, 70 paired 180 s cases at 3 NFE: instruction 22 wins, 47 ties, 1 loss; motion -8.6%
ā¢ Progressive VIF 3.856 vs 3.831, aesthetics 0.642 vs 0.633, pooled AVSync error 1.319 to 1.341 s
ā ļø Caveats
ā¢ One cost measurement with no variance reported; 70-case rounds with no significance tests
ā¢ Only 180 s avatar-style video at 1344x768, and the baseline is the authors' own recipe
Paper:
https://huggingface.co/papers/2611.14033
Would you accept a few percent in training savings and better prompt following in exchange for weaker motion?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/fd4facc13c3f
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/apex-drops-dmds-fake-score-network
ā¢ The thread on X: https://x.com/saavoji/status/2120949703953289381
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#apex-drops-dmds repliedto their post 1 day ago
A 284B-parameter DeepSeek-V4-Flash now runs on a 64 GB desktop with 0 GPUs: llama-modchip shrinks the checkpoint to 56.56 GiB (1.71 bpw) and decodes at 9.1 to 14.7 tok/s.
The standard UD-IQ1_S build is 76.87 GiB, above the ~62.5 GiB usable on a 64 GB Linux box. The repo mixes precisions (routed experts mostly IQ1_S, MLA projections in a custom Fractal-64 type) and patches the CPU kernels. The model plus a 256K KV cache then sits at 57.61 to 58.9 GiB RSS with zero swap. The repo was created on 2026-10-29 and already has 744 stars and 470 points on Hacker News.
š The numbers
ā¢ Size: 56.56 GiB, 20.31 GiB smaller than UD-IQ1_S
ā¢ Prefill: 25.0 to 44.36 tok/s (+77%) on a 16-core Ryzen 9 9955HX
ā¢ Decode: 5.91 tok/s unpatched, 9.06 to 9.95 tok/s at Top-6, 11.5 to 14.7 tok/s with the n-gram drafter
ā ļø Caveats
ā¢ "Lossless" means bit-exact against the same re-quantized checkpoint. Versus UD-IQ1_S the author reports 0.9964 top-logit cosine similarity
ā¢ The 98.5/100 quality score comes from the author's own gate, and the headline speeds come from one machine
ā¢ The custom tensor type 40 is rejected by stock llama.cpp, Ollama and LM Studio
š ļø Try it
ā¢ Run scripts/check_cpu_readiness.py from the repo: it needs no model download
ā¢ Run ./modchip bench for speed, then ./modchip eval to check the quality claims yourself
Repo:
https://github.com/synchopate/llama-modchip
HN thread:
https://news.ycombinator.com/item?id=50411961
Would you accept a custom runtime for 9 to 15 tok/s from a 284B model on a 64 GB box?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/e47ab3581f58
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/llama-modchip-runs-a-284b-deepseek
ā¢ The thread on X: https://x.com/saavoji/status/2119858773280141526
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#llama-modchip repliedto their post 1 day ago
A 284B-parameter DeepSeek-V4-Flash now runs on a 64 GB desktop with 0 GPUs: llama-modchip shrinks the checkpoint to 56.56 GiB (1.71 bpw) and decodes at 9.1 to 14.7 tok/s.
The standard UD-IQ1_S build is 76.87 GiB, above the ~62.5 GiB usable on a 64 GB Linux box. The repo mixes precisions (routed experts mostly IQ1_S, MLA projections in a custom Fractal-64 type) and patches the CPU kernels. The model plus a 256K KV cache then sits at 57.61 to 58.9 GiB RSS with zero swap. The repo was created on 2026-10-29 and already has 744 stars and 470 points on Hacker News.
š The numbers
ā¢ Size: 56.56 GiB, 20.31 GiB smaller than UD-IQ1_S
ā¢ Prefill: 25.0 to 44.36 tok/s (+77%) on a 16-core Ryzen 9 9955HX
ā¢ Decode: 5.91 tok/s unpatched, 9.06 to 9.95 tok/s at Top-6, 11.5 to 14.7 tok/s with the n-gram drafter
ā ļø Caveats
ā¢ "Lossless" means bit-exact against the same re-quantized checkpoint. Versus UD-IQ1_S the author reports 0.9964 top-logit cosine similarity
ā¢ The 98.5/100 quality score comes from the author's own gate, and the headline speeds come from one machine
ā¢ The custom tensor type 40 is rejected by stock llama.cpp, Ollama and LM Studio
š ļø Try it
ā¢ Run scripts/check_cpu_readiness.py from the repo: it needs no model download
ā¢ Run ./modchip bench for speed, then ./modchip eval to check the quality claims yourself
Repo:
https://github.com/synchopate/llama-modchip
HN thread:
https://news.ycombinator.com/item?id=50411961
Would you accept a custom runtime for 9 to 15 tok/s from a 284B model on a 64 GB box?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/e47ab3581f58
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/llama-modchip-runs-a-284b-deepseek
ā¢ The thread on X: https://x.com/saavoji/status/2119858773280141526
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#llama-modchipOrganizations
None yet