DiffusionART：高效强化扩散模型
用奖励聚合与信任域更新稳定扩散模型在线强化学习。
为什么重要
为图像和视频扩散模型的RL微调提供更稳定高效的可复现方法。
阅读原文arxiv.org
AI 解读（根据原文生成）
论文提出 DiffusionART（Diffusion Reinforcement with Aggregation-First Reward Tilting），用于用黑盒奖励在线强化学习微调扩散模型。方法将每个 prompt 的奖励聚合为共享的奖励改进方向，避免隐藏的前向—反向一致性力，并加入 trust-region-masked 第二次更新以实现无需 EMA 调参的恢复。它不需要可微奖励模型，支持更少样本和 pairwise reward models。代码已在 GitHub 提供。
要点（AI 据原文整理，数字均已与原文核对）
- DiffusionART aggregates each prompt's rewards into a shared reward-improving direction for reverse-process states.
- DiffusionART adds a trust-region-masked second update for drift-adaptive restoration without EMA tuning.
- It requires no differentiable reward model and remains effective with fewer samples per prompt.
- DiffusionART supports pairwise reward models (DiffusionART-Pairwise).
- Across image and video generation, DiffusionART trains efficiently and stably without per-task tuning.
- On HunyuanVideo-13B, DiffusionART reaches late-stage mean reward levels with 12.02 and 11.72 times fewer GPU hours than DiffusionNFT and Flow-GRPO-Fast, respectively.