Post
30
APEX removes DMD's separate fake-score network for streaming video distillation: 4.4% less peak memory, 9.2% shorter iterations, and +30.0% net human preference on instruction following.
DMD needs an auxiliary network that tracks the generator's output distribution. APEX uses the generator itself. The original text condition generates, and the negated projected text (c_shift = -c) selects fake-velocity estimation with the same weights. The frozen teacher, Self-Replay Forcing and perceptual loss from S2-Avatar stay. The price is weaker motion.
š What changes
ā¢ One set of weights and, by default, shared AdamW states
ā¢ Inference uses only the generation path, then a one-step 2x Refiner
ā¢ The fake estimator is fitted to replay outputs; its clean-latent gap to the teacher gives a detached generator correction
š The numbers
ā¢ Vs DMD at matched settings: 102.7 to 98.2 GiB per GPU, 35.9 to 32.6 s per iteration
ā¢ Vs S2-Avatar, 70 paired 180 s cases at 3 NFE: instruction 22 wins, 47 ties, 1 loss; motion -8.6%
ā¢ Progressive VIF 3.856 vs 3.831, aesthetics 0.642 vs 0.633, pooled AVSync error 1.319 to 1.341 s
ā ļø Caveats
ā¢ One cost measurement with no variance reported; 70-case rounds with no significance tests
ā¢ Only 180 s avatar-style video at 1344x768, and the baseline is the authors' own recipe
Paper:
Few-Step Streaming Video Generation via Condition Shifting (2611.14033)
Would you accept a few percent in training savings and better prompt following in exchange for weaker motion?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/fd4facc13c3f
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/apex-drops-dmds-fake-score-network
ā¢ The thread on X: https://x.com/saavoji/status/2120949703953289381
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#apex-drops-dmds
DMD needs an auxiliary network that tracks the generator's output distribution. APEX uses the generator itself. The original text condition generates, and the negated projected text (c_shift = -c) selects fake-velocity estimation with the same weights. The frozen teacher, Self-Replay Forcing and perceptual loss from S2-Avatar stay. The price is weaker motion.
š What changes
ā¢ One set of weights and, by default, shared AdamW states
ā¢ Inference uses only the generation path, then a one-step 2x Refiner
ā¢ The fake estimator is fitted to replay outputs; its clean-latent gap to the teacher gives a detached generator correction
š The numbers
ā¢ Vs DMD at matched settings: 102.7 to 98.2 GiB per GPU, 35.9 to 32.6 s per iteration
ā¢ Vs S2-Avatar, 70 paired 180 s cases at 3 NFE: instruction 22 wins, 47 ties, 1 loss; motion -8.6%
ā¢ Progressive VIF 3.856 vs 3.831, aesthetics 0.642 vs 0.633, pooled AVSync error 1.319 to 1.341 s
ā ļø Caveats
ā¢ One cost measurement with no variance reported; 70-case rounds with no significance tests
ā¢ Only 180 s avatar-style video at 1344x768, and the baseline is the authors' own recipe
Paper:
Few-Step Streaming Video Generation via Condition Shifting (2611.14033)
Would you accept a few percent in training savings and better prompt following in exchange for weaker motion?
š Read it elsewhere
ā¢ The deep dive on Medium: https://medium.com/p/fd4facc13c3f
ā¢ The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/apex-drops-dmds-fake-score-network
ā¢ The thread on X: https://x.com/saavoji/status/2120949703953289381
ā¢ Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#apex-drops-dmds
