Local GPU vs Cloud API: Calculate the Break-Even Request Volume

A local GPU becomes cheaper than a cloud API only when monthly request volume covers its fixed cost through lower cost per request. Calculate that threshold using the API’s effective price after caching, then check whether the local system can actually serve that volume at the required quality and latency. A cheaper theoretical threshold is useless if the workload cannot meet those conditions.

Scope: an illustrative monthly operating decision, with API rates checked on November 13, 2026 KST. Local hardware costs, request volumes and capacity below are assumptions, not measured performance or a hardware quotation.

Terms and route through the calculation

• 1. Qualified request: an output that meets the task and service target
• 2. Effective API price: the average bill per request, including cache writes
• 3. Fixed cost and variable cost: monthly commitment versus additional work
• 4. Break-even volume: where the two monthly cost lines cross
• 5. Feasible capacity: requests deliverable within the latency target
• 6. Decision checks and the next measurement

1. Qualify the work before comparing dollars

Start with the same task set: representative prompts, document lengths, output requirements and an acceptance rubric. Fix time-to-first-token and end-to-end latency targets, concurrency, and the hours when service must be available. A fast answer that fails validation does not count as a successful business result.

The hosted model used for the price example is not a claim that an open model on one GPU offers equivalent quality. Select a locally runnable candidate, verify that its weights and working memory fit, and evaluate both routes on the same tasks. Token counts can differ between tokenizers; collect each route’s actual usage instead of assuming equal text means equal tokens.

For a real comparison, divide all attributable costs, including retries and rejected outputs, by accepted results. The simplified arithmetic below assumes one successful attempt per request. If either route fails the quality or service test, resolve that first; a cost crossover alone does not qualify a replacement.

2. Compute the API price with a cache-write allowance

The current direct Claude API price table lists Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. Five-minute cache writes cost $2.50 per million tokens, and cache hits cost $0.10 per million. These are USD rate inputs for this example, not a model recommendation. Source: Anthropic pricing.

Assume a request has 8,000 input tokens and 500 billed output tokens. Without caching, its cost is (8,000 × $2 + 500 × $10) / 1,000,000 = $0.021. Use billed output usage, including any billable reasoning, rather than counting only the text displayed to the reader.

Now assume 6,000 input tokens form a reusable prefix and 2,000 remain uncached. The first request writes that prefix: (6,000 × $2.50 + 2,000 × $2 + 500 × $10) / 1,000,000 = $0.024. Each subsequent hit costs $0.0096. A hypothetical group of 100 requests with one write and 99 hits therefore averages $0.009744 per request.

That reuse rate is an assumption. The prefix must match and remain valid under the cache lifetime; the default five-minute lifetime refreshes when used. Inspect cache creation, cache read and uncached input usage separately. Do not bill cached tokens again as ordinary input. Changed prefixes, expiry and unsuccessful reuse raise the average. Source: prompt-caching behavior and usage fields.

3. Separate the local monthly commitment from marginal work

Assume a complete local setup costs $5,400 and allocate it across 36 months: $150 per month. Add an assumed $90 of fixed monthly operations and idle costs. The resulting fixed cost, F, is $240 per month. This is a cost-allocation illustration, not a forecast of resale value, accounting treatment or cash payback.

Assume each additional qualified request adds v = $0.003 in local operating cost. With N monthly requests, local cost is $240 + $0.003 × N. The assumed variable cost is not derived from a particular GPU’s throughput or power consumption.

Replace these inputs with your own hardware, host memory, storage, support, software licensing and operating costs. Keep idle electricity in the fixed bucket and additional load electricity in the variable bucket if using this split; counting total electricity in both would inflate cost. NVIDIA’s GPU power reading covers the board, so it should not silently stand in for whole-system wall power. Source: NVIDIA-SMI power definitions.

4. Compare three volumes, then solve the crossover

The table uses the same hypothetical request shape throughout. The cached API column includes one write per 100 requests; all three monthly volumes contain complete groups. Amounts are rounded only for display. Taxes, special service tiers, regional premiums and optional tool charges are outside this simplified scope.

Original calculation: monthly cost in USD, not a measured deployment

Requests / month	Monthly costs
4,000	Local: $252.00 API: $84.00 Cached API: $38.98
8,000	Local: $264.00 API: $168.00 Cached API: $77.95
16,000	Local: $288.00 API: $336.00 Cached API: $155.90

Break-even requests = F / (c − v)
Here c is effective API cost per request. If c is less than or equal to v and F is positive, there is no finite volume at which this local cost model becomes cheaper.

Against the uncached API, $240 / ($0.021 − $0.003) gives about 13,333 requests: the local option first becomes cheaper at 13,334 whole requests per month. Against the cached average, the threshold rises to approximately 35,600 requests per month. This second figure is a smooth average-rate estimate; actual cache-write counts determine the exact bill.

At 16,000 requests, the local scenario saves $48 against the uncached API, yet costs $132.10 more than the cached API. The hardware did not change. The comparison changed because the cloud request became cheaper through assumed prefix reuse.

5. Reject a crossover that exceeds usable capacity

Suppose the local setup could deliver 100 qualified requests per hour for 160 service hours per month. That is an assumed ceiling of 16,000 requests. The uncached crossover consumes about 83% of this ceiling; the cached crossover exceeds it. Under these assumptions, the single local setup cannot reach the volume needed to beat the cached API.

This fraction is demand divided by qualified capacity, not the GPU utilization percentage shown by a monitoring tool. Bursty arrivals, maintenance and headroom can reduce what is practical. Adding another machine changes fixed cost, so recalculate the crossover instead of extending the original cost line indefinitely.

Benchmark the intended model and workload under latency constraints before accepting any capacity figure. vLLM’s serving benchmark supports goodput constraints for time to first token, time per output token and end-to-end latency. That provides a useful measurement framework; no vLLM or GPU benchmark was run for this article. Source: vLLM serving benchmark.

6. Turn the calculation into a purchasing or operating check

• New purchase: include the full relevant commitment and test a low-demand month. A crossover near the capacity limit leaves little room for a weaker forecast.
• Already-owned hardware: make a separate incremental-cost comparison. Do not mix a sunk-cost view on the local side with a full-cost view elsewhere without stating that choice.
• Work can wait: compare eligible asynchronous API pricing too. Anthropic lists a 50% Batch API token discount, but an asynchronous workflow must satisfy the actual delivery deadline. Pricing reference.
• Inputs still unknown: collect representative token usage, cache-write frequency, accepted-output rate and measured service capacity. Keep the purchase decision conditional until those values replace the assumptions.

A usable result is a range of monthly demand where both quality and latency pass and one route has lower attributable cost. If that range disappears after a realistic cache rate or capacity check, the calculation has still answered the question: the proposed local configuration has not established a cost advantage for that workload.

Appendix: what was executed

The arithmetic and crossover boundary checks were executed offline with Node.js v24.14.1 on Windows. 18 assertions passed, including the no-crossover case and both sides of the uncached whole-request threshold. No model inference, paid API request, GPU power measurement or capacity benchmark was executed. The local $5,400 setup, $90 monthly fixed allowance, $0.003 variable cost and 100-request hourly capacity remain hypothetical.

For the next measurement, use the request-log guide to p50, p95 and SLO goodput and the guide to aligning GPU power and latency on one timeline. Both are existing English Deoks articles; they supply measurement steps, not evidence that this hypothetical setup achieved the stated capacity.

Primary sources

• Anthropic — Pricing; rate table checked November 13, 2026 KST.
• Anthropic — Prompt caching; lifetime, usage categories and cache accounting.
• vLLM — bench serve; current stable documentation, not an installed-version test.
• NVIDIA — NVIDIA-SMI documentation; board power measurement boundary.