Architecture Library
· Advanced
Kubernetes AI Inference Platform
GPU scheduling, vLLM serving, autoscaling, and observability for production LLM workloads on EKS.
Overview
graph LR
ING[Ingress] --> GW[AI Gateway]
GW --> VLLM[vLLM on GPU nodes]
VLLM --> OTEL[OpenTelemetry]
Use Karpenter for GPU burst capacity — not fixed node pools.
Graphique des connaissances