← Back to feed

ShadowPrefill: Keeping Your Next Model Warm for Model Routing in Agentic Serving

AI · Research · 2026-11-13 19:16

提出ShadowPrefill方法，通过预填充机制保持下一候选模型处于热状态，优化智能体LLM服务中逐步模型路由的延迟与效率。

Source: arxiv.org

Auto-generated summary for reference only. RSS