A Complete Postmortem of an LLM Inference OOM

Q: The vLLM service keeps restarting — how do I confirm it is GPU-memory OOM?

Look for 'CUDA out of memory' or 'No available memory' in the vLLM log. Then run nvidia-smi to check memory usage — if it spiked near the limit just before the restart, it is almost certainly memory OOM. If the log has no OOM but the service still restarts, check the health probe (for example /health returning 503 and the orchestrator killing it) or a watchdog.

Q: Why does GPU memory get blown out — is it a leak?

Two common classes: 1) prefix cache / KV cache growing with the request pattern — long contexts or high concurrency push usage higher and higher past the gpu_memory_utilization ceiling; 2) a genuine leak — long-lived connections, unreleased streaming responses, or a malformed request leaving memory blocks unreclaimed. Sample nvidia-smi over time: a monotonic climb is a leak or cache growth.

Q: Will gpu_memory_utilization set to 0.9 cause OOM?

It can. 0.9 leaves only 10% headroom, and as the KV cache grows with concurrency on top of model weights and activation memory, usage can easily hit 100% and OOM. Leaving 15%-20% headroom is safer, or pair it with max_num_seqs to cap concurrency. The key is to tune gpu_memory_utilization and max_num_seqs together, not one alone.

Q: How do I let it recover on its own after OOM instead of failing to restart forever?

Configure automatic recovery: lower --max-num-seqs, give the health probe a startup grace period (wait for the model to load before probing), and auto-restart the container or process after OOM. The critical part is aligning the probe timeout with the actual startup time — otherwise the probe kills it before the model finishes loading and you get a restart storm.

Q: Which GPU metrics should I monitor to prevent a recurrence?

Memory usage (memory.used / memory.total, alerting near the limit), the OOM counter (GPU or process OOM events), request queue depth (pending requests, which reveals over-subscription), and vLLM's KV-cache usage. Memory usage is the most direct; the OOM counter is the fallback signal that it already happened.

This reconstructs one representative class of "a vLLM inference service kept restarting as GPU memory filled and triggered OOM." Your service might use a different inference engine or CPU deployment, but the diagnostic ladder and the traps are universal: confirm it is memory OOM and not a health-probe kill, check whether concurrency is over-subscribed and the prefix cache is growing, and do not treat "more GPU memory" as the only fix.

Summary

A few hours after going live, the vLLM inference service started restarting repeatedly: the process came up, ran for a while, GPU memory spiked, the process was OOM-killed, then it restarted in a loop. Requests either timed out or streamed partway before cutting off. A single nvidia-smi settled it — memory usage hit 98%+ right before each restart, and the vLLM log explicitly reported CUDA out of memory. This was the classic over-subscribed concurrency + cache growth blowout.

Impact

Inference requests failed or were cut off during every OOM window: timeouts, mid-stream interruptions, 5xx responses.
The service sat in a "half-dead" state: the process was alive but every request risked triggering OOM, so throughput was wildly unstable.
Roughly two hours from the first OOM to lowering concurrency and leaving proper memory headroom.
Because the GPU is a shared resource, other concurrent workloads (like batch inference) were squeezed and slowed too.

Timeline

Time	Event
T+0	First OOM; service restarts
T+20	Short period of normal operation, then OOM again
T+45	Enters a restart loop; user requests start failing heavily
T+70	`nvidia-smi` shows 98% memory; memory OOM confirmed
T+100	`max_num_seqs` lowered, `gpu_memory_utilization` reduced, restart
T+130	Memory stabilizes, throughput recovers; memory/OOM monitoring added
T+150	Postmortem closes; root cause is over-subscribed concurrency + prefix-cache growth

Root cause

The real trigger was over-subscribed concurrency compounding with prefix-cache (KV cache) growth: gpu_memory_utilization was set high (0.9) and max_num_seqs was large, so when long-context requests increased and the prefix cache climbed with the request pattern, memory usage exceeded the reserved ceiling and triggered CUDA OOM. Two amplifiers turned it into a full outage:

No memory monitoring: we monitored "service alive" but not "memory usage." Nobody knew when usage climbed past 90%, until OOM exposed it.
No automatic OOM recovery strategy: after the OOM kill, the orchestrator restarted immediately, but the startup probe (/health) judged it dead before the model finished loading, causing a restart storm — killed again before it was even up.

The decisive clue (and the watershed of the whole postmortem): nvidia-smi showed 98% memory right before the restart, and the vLLM log explicitly said CUDA out of memory — this was a memory blowout, not a probe mis-kill.

Diagnostic traps that actually cost time

What slowed recovery was not "couldn't find it" — it was treating every "repeated restart" as a probe problem. Real lessons:

Read the log before touching the probe. The first reflex when a service restarts repeatedly is "the health check is misconfigured." Run nvidia-smi for memory and journalctl for the vLLM log first — if there is CUDA OOM, do not go tune the probe; that is wasted time.
Sample memory over time. A single nvidia-smi only shows the present. Use watch -n 2 nvidia-smi or spaced samples to see the trend — a steady climb is a leak/cache growth, an immediate spike-to-limit is over-subscription.
A restart storm is probe/startup misalignment. vLLM takes tens of seconds to minutes to load the model, but the probe timeout was short, so it was judged dead before it was up. Give the probe a startup grace period aligned with the real load time.
Do not tune gpu_memory_utilization alone. Lowering the memory ceiling only leaves headroom; if concurrency stays high, the KV cache climbs anyway. Tune max_num_seqs too — capping concurrency is what fixes it.
OOM is not always a leak. OOM can be growth-type (over-subscription / prefix cache) or leak-type (unreleased long connections). Read the trend to tell them apart: a monotonic climb that does not fall back is a leak; a spike-to-OOM is over-subscription.

Fix

Climb the ladder, look at memory first:

# 1) Check memory usage (real-time + spaced samples for the trend)
nvidia-smi
watch -n 2 nvidia-smi          # keep watching for a monotonic climb or a spike

# 2) Check the vLLM log to confirm CUDA OOM
journalctl -u vllm --since "2h ago" | grep -iE "out of memory|no available memory|CUDA"
# OOM present -> memory blowout; absent -> check the probe/watchdog

# 3) Check whether concurrency/memory config is over-subscribed
ps aux | grep vllm | grep -oE "gpu-memory-utilization[= ][0-9.]+|max-num-seqs[= ][0-9]+"

For immediate containment, lower concurrency + reduce the memory ceiling + restart to stabilize:

# Lower concurrency (e.g. 256->128) and memory ceiling (0.9->0.85), then restart
# example launch args:
# --gpu-memory-utilization 0.85 --max-num-seqs 128
systemctl restart vllm
# Watch memory stabilize in the target band
watch -n 5 nvidia-smi

For the permanent fix, align the probe with startup time + rate-limit + auto-recover. Give the probe a startup grace period and auto-restart after OOM instead of repeatedly failing:

# example probe config (k8s readiness/liveness, aligned with startup)
# readinessProbe: /health, initialDelaySeconds=120, periodSeconds=10
# livenessProbe:  /health, initialDelaySeconds=120, periodSeconds=30
# restartPolicy: Always
# key: initialDelaySeconds must be >= the model's real load time, or it dies on startup

Prevention

Three things this incident exposed, each with a guard:

Memory-usage monitoring (first guard): do not monitor "service alive" only — monitor memory usage. Alert on memory.used / memory.total near the limit (e.g. 90%) to leave an intervention window before OOM.
OOM counter monitoring: track the GPU/process OOM event counter as the fallback "it already happened" signal; combined with the memory trend it distinguishes growth-type from leak-type.
Concurrency cap and quota: tune max_num_seqs and gpu_memory_utilization together, leaving 15%-20% headroom for the KV cache; rate-limit or queue when the pending request depth exceeds a threshold, so a burst cannot blow the memory ceiling.
Graceful restart and probe alignment: set the probe's initialDelaySeconds to at least the model load time; auto-restart the container/process after OOM with a startup grace period to prevent a restart storm.
Failure drill: periodically simulate "high concurrency + long context" and verify the memory monitor alerts before OOM and the rate limiter actually kicks in.

Lesson

When an inference service restarts repeatedly, check memory first — nvidia-smi confirms whether it is memory OOM; do not start tuning the probe. The diagnostic order is always: look at the memory trend → read the vLLM log to confirm OOM → check the concurrency/memory config → align the probe with load time. The thing that is easiest to miss is the restart storm: the probe timeout was shorter than the model load, so the service was judged dead before it was up.

The most valuable takeaway is simple: if you monitor the wrong object, you always react half a beat late. We used to "monitor service alive," which only signaled once OOM had already happened. Now we "monitor memory usage and the OOM counter," which surfaces the problem when memory climbs past 90%. Moving the monitor one layer up shortens MTTR more reliably than writing another alert rule — that, more than any single command, is what this incident taught me.

It also reinforced that the cheap diagnostic is usually the right one: one nvidia-smi told us more about where the fault lived than any dashboard. When an inference service keeps restarting, read the log and watch the memory trend first, tune concurrency and memory together second, and only then declare the incident resolved.

Related reading: [vLLM Production Tuning../../ai/vllm-production-tuning.md) · [vLLM Serving../../ai/vllm-serving.md) · [LLM Monitoring../../ai/llm-monitoring.md)