Partner & customer examples

Four insights that show AI Infrastructure Intelligence at work

Each example correlates usage, cost, and ownership across systems, then delivers a plain-English finding leaders can act on. Read-only. Evidence-backed. No cluster mutation.

Insight 01

Idle GPUs vs waiting jobs

$1.84M recoverable, not a chip shortage.

Insight 02

K8s over-request tax

3.2× requested vs used, scheduler sees “full.”

Insight 03

Cost per useful model

Same team: 7.4× cost for tiny eval lift.

Insight 04

“Need H100s” — wrong

I/O saturated; GPUs idle. Stop the bad buy.

NexDiscovery · AI Infrastructure Intelligence · Illustrative customer scenarios

Insight 01 · Queue vs silicon · usage + cost + contracts

14% of reserved A100s sit idle for 14+ days while eligible jobs wait in the queue

$1.84M / year recoverableConfidence 84%Constraint: RI locks & freezeOwners: Platform + FinOps

In plain English: You are paying for GPUs that barely work, while other teams wait. This is a placement and request problem, not a shortage of chips. Do not buy more hardware yet.

DCGM util→K8s placement→Slurm pending→Billing / RIs→Namespace owners

idle reserved GPUs

11.4h

median job wait

namespaces drive 81% of $

96%

GPU↔billing join

The tension

Platform escalates for more A100s. FinOps sees RI util missing target. Same week, opposite stories.

One evidence pack ends the argument: idle silicon + waiting queue + $ impact + who owns it.

Spend by namespace

ml-foundation · $620k

genai-batch · $510k

risk-sim · $360k

cv-training · $180k

nlp · $95k

other · $75k

Recommended actions

→ Reclaim unlocked namespaces first
→ Defer purchase while idle ≥10%
→ Prep freeze-bound pool for day-one

Insight 02 · Request vs reality · Kubernetes + usage + cost

Training jobs request 8 GPUs but use ~2.5: the cluster looks full while cards sit quiet

$960k / year stranded headroomConfidence 88%K8s remediableAvg over-request 3.2×

In plain English: Copy-paste YAML is taxing the whole company. Kubernetes believes nodes are full. Silicon disagrees. Other teams go Pending for no good reason.

K8s requests/limits→DCGM SM util→Pending pods→Amortised node cost→Team / namespace

3.2×

request vs DCGM use

214

jobs over-requesting

37%

Pending due to “no GPU”

$960k

annualised headroom

The tension

ML engineers say “Insufficient nvidia.com/gpu.” Platform thinks capacity is exhausted. Nobody checks actual util vs request.

Worst offenders

· risk-sim Jobs: 3.8× over-request
· ml-foundation overnight trains: 3.4×
· Sticky Jupyter pods holding 1 GPU idle 18h/day

Recommended actions

→ Rightsizing PRs to p75 DCGM use
→ ResourceQuota per namespace
→ TTL on interactive GPU pods

Partner talking point: NexDiscovery does not patch the cluster. It hands SRE/MLOps a ranked list with pod names, ratios, owners, and $ — ready for GitOps tickets.

Insight 03 · Unit economics · model usage + GPU-hours + billing

One experiment costs $186k for +0.4% eval lift; a peer run hits target for $25k

7.4× cost gapConfidence 79%Model-layer inefficiencySame team, same GPU class

In plain English: Not all GPU-hours are equal. Linking experiments to jobs and bills shows which training patterns burn money for almost no model gain, so research leaders can stop the expensive loops.

MLflow runs→Job / Ray / KFP→GPU-hours (DCGM)→Amortised cost→Eval metric delta

$186k

expensive run (Q1)

$25k

efficient peer run

+0.4%

eval lift, expensive

+2.1%

eval lift, efficient

The tension

Research velocity looks high. GPU bill climbs. Nobody can show cost per useful model improvement by team or experiment.

Connects FinOps language ($/GPU-hour) to ML language (eval lift), the conversation executives actually need.

Recommended actions

→ Cap GPU budget on low-lift recipes
→ Promote efficient configs as defaults
→ Review top 10 runs monthly by $/Δmetric

Insight 04 · I/O false scarcity · usage + network/storage + purchase risk

Multi-node jobs look “GPU hungry” but cards idle while storage and network hit 90%+

$2.1M purchase challengedConfidence 81%Not a GPU shortageFix storage / fabric first

In plain English: The team asked for a new H100 pod because training is slow. Correlation shows GPUs waiting on I/O. Buying more accelerators would waste capital. Fix the pipe, not the chip count.

Job timelines→DCGM SM (low/spiky)→Prometheus disk/net→Node placement→Purchase request

22%

median SM during “slow” jobs

91%

disk util, same windows

multi-node jobs affected

$2.1M

H100 buy on hold

The tension

Training SLAs slip. Vendors pitch more GPUs. CapEx request is in flight. Root cause is invisible in a GPU-only dashboard.

Stops a seven-figure mistake. Proves “need more GPUs” is false scarcity when I/O is the ceiling.

Recommended actions

→ Pause H100 purchase pending I/O fix
→ Raise FS / NIC throughput or locality
→ Re-test GPU util after remediation

How to use these with customers

Four stories. One platform pattern.

Every example follows the same NexDiscovery loop: map systems → correlate usage and cost → challenge the finding → deliver value, confidence, constraints, and human-led actions.

What partners should emphasise

1. Idle vs queue ends “buy more” fights.
2. K8s over-request is remediable by engineering.
3. Model unit economics links research to FinOps.
4. I/O false scarcity protects CapEx.

Always disclose

Confidence drivers (join coverage, metric completeness), data limits, and constraints (RI locks, freezes). Read-only — NexDiscovery recommends; the customer remediates.

AI Infrastructure Intelligence turns fragmented util, Kubernetes, jobs, and bills into verified stories — so organisations recover capacity and spend with evidence, not opinions.