用 NVIDIA Dynamo-Triton 部署 HSTU 推荐器
展示从 PyTorch 到生产推理的 HSTU 生成式推荐部署流程。
为什么重要
为长历史推荐模型提供了可复用的低延迟部署路径。
阅读原文developer.nvidia.com
AI 解读（根据原文生成）
NVIDIA Technical Blog 介绍了用 NVIDIA Dynamo-Triton 部署 HSTU 生成式推荐模型的端到端流程。该工作流通过 NVIDIA recsys-examples repository 结合 HSTU、PyTorch Ahead-of-Time Inductor 编译、FlexKV 支持的 KV 缓存、原生 C++ 验证、NV embedding cache 和 Dynamo-Triton 部署。文章说明如何导出并提前编译模型，在 Python 与原生 C++ 中验证部署产物，并通过 Dynamo-Triton 服务化；正文未给出代码或权重链接。
要点（AI 据原文整理，数字均已与原文核对）
- NVIDIA Dynamo-Triton now supports an end-to-end HSTU GR inference workflow through the NVIDIA recsys-examples repository.
- The workflow combines HTSUs, PyTorch Ahead-of-Time Inductor compilation, FlexKV-backed KV caching, native C++ validation, NV embedding cache, and Dynamo-Triton deployment.
- At dynamic batch size 8 on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU, Dynamo-Triton with PyTorch AOTI achieved best-case speedups of up to 4.47x for the three-layer HSTU model.
- The speedups are relative to the same AOTI configuration without KV caching.
- The post examines how GPU-backed KV caching reduces repeated computation.
