Google, Red Hat and IBM released llm-d 1.0, an open-source Kubernetes-native stack for distributed LLM inference. It adds multi-tenant flow control, zero-downtime upgrades and agent-aware routing across vLLM, SGLang, TPUs and GPUs
Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets
Google, Red Hat and IBM released llm-d 1.0, an open-source Kubernetes-native stack for distributed LLM inference. It adds multi-tenant flow control, zero-downtime upgrades and agent-aware routing across vLLM, SGLang, TPUs and GPUs
M
Asif Razzaq · via MarkTechPost
Curated & Verified Aggregation
Wednesday, October 21, 2026
2 min
Original reporting by MarkTechPost
Read the full story at MarkTechPostListen to this story
Narrated edition • 2 min read time
researchaitalky dispatch
“Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets”
via MarkTechPostPRIMARY SOURCE
Related Stories
researchaitalky dispatch
“Recursive Releases Flint, an Open-Weight 40B MoE Local Coding Model for Agents with 2.5B Active Params”
via MarkTechPostPRIMARY SOURCE
Recursive Releases Flint, an Open-Weight 40B MoE Local Coding Model for Agents with 2.5B Active Params
researchaitalky dispatch
“Vercel Open-Sources durable-harness for Crash-Proof Coding Agents”
via MarkTechPostPRIMARY SOURCE
Vercel Open-Sources durable-harness for Crash-Proof Coding Agents
