H100 SXM vs. H200 SXM: Which GPU Should You Choose?

H200 SXM can change memory fit and bandwidth-bound serving relative to H100 SXM; compute-bound work needs its own benchmark.

Updated 2026-10-07 · refreshed every 12 hours

Choose H200 SXM when its larger memory keeps your workload on a usable configuration. When the job fits either GPU, compare measured runtime and the current rental rate; extra memory alone does not prove faster compute.

Run a bounded test with the same model, precision, batch size and sequence length on each feasible option. Record peak memory, billed runtime and completed work; use the actual checkout rate to compare compute cost per completed job.

Does peak runtime memory exceed H100 SXM capacity but fit H200 SXM? If so, test H200 SXM as a single-device option. If the workload fits either, benchmark the intended decode, prefill or training path separately. If it exceeds both, verify an actual distributed configuration.

Which option meets your deadline and job budget in a representative test? Measure completion time first. The H100 SXM-to-H200 SXM modeled cost-per-job ratio is 0.8; it uses specification-based assumptions and does not establish real runtime or a deadline.

Can the exact configuration launch under the rental terms your job needs? Confirm the GPU count, topology and availability at checkout. Use spot only after testing checkpoint recovery; if the required configuration cannot launch, reassess fit before substituting another GPU.

Today's prices

USD/hr · 1x GPU · prices grouped by provider and purchase model

Provider	H100 SXM	H200 SXM
Hyperstack	$3.20 · Not reported	$3.99 · Not reported
RunPod	$3.49	$4.59
DataCrunch	$3.85	$5.02
Vast.ai	$3.87	Not offered
Crusoe	$3.90 · Not reported	$4.29 · Not reported
Together AI	$3.99 · Not reported	$5.99 · Not reported
Paperspace	$5.95 · Not reported	Not offered
Lambda	Unavailable as of 2026-10-07	Not offered

Configuration prices used by workload rows

USD/hr · currently eligible on-demand configurations at the exact GPU count shown in the workload table

Provider	GPU	Configuration	Price	Price basis	Availability
CoreWeave	H100 SXM	8 GPUs	$49.24/hr	Published list rate	Not reported
Lambda	H100 SXM	2 GPUs	$8.38/hr	Published list rate	In stock

see all H100 prices →

Where the two parts differ

Specification	H100 SXM	H200 SXM
Memory	80 GB HBM3	141 GB HBM3e
Memory bandwidth	3350 GB/s	4800 GB/s
Dense BF16	989 TFLOPS	989.5 TFLOPS
Dense FP8	1979 TFLOPS	1979 TFLOPS
NVLink	900 GB/s	900 GB/s
TDP	700 W	700 W

Prices are refreshed every 12 hours. Marlin matches your requirements across supported providers. Try Marlin →

0.80×

H100 SXM-to-H200 SXM on-demand hourly price ratio. Below one favors the first GPU on hourly rate; above one favors the second.

0.80×

H100 SXM-to-H200 SXM modeled cost-per-job ratio. Below one favors the first under the stated model; validate with measured runtime.

$2336/mo

Continuous-use monthly baseline for one H100 SXM at its current on-demand minimum; scale the budget to your billed hours.

What your workload needs

Find your parameter count on the model's Hugging Face card. Serving maps to the inference rows, training to the fine-tuning rows.

Workload	VRAM needed	H100 SXM	H200 SXM	Cheapest today
7B Q4 inference	4 GB	1-GPU configuration: $3.20/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	1-GPU configuration: $3.99/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	H100 SXM: 1 GPU at $3.2/hr
13B Q4 inference	8 GB	1-GPU configuration: $3.20/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	1-GPU configuration: $3.99/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	H100 SXM: 1 GPU at $3.2/hr
70B QLoRA	46 GB	1-GPU configuration: $3.20/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	1-GPU configuration: $3.99/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	H100 SXM: 1 GPU at $3.2/hr
70B FP8 training	154 GB	2-GPU configuration: $8.38/hr (In stock; observed 2026-10-07; price basis: Published list rate)	No eligible 2-GPU configuration	H100 SXM: 2 GPUs at $8.38/hr
70B FP16 inference	168 GB	8-GPU configuration: $49.24/hr (Not reported; observed 2026-10-07; price basis: Published list rate)	No eligible 2-GPU configuration	H100 SXM: 8 GPUs at $49.24/hr

What the numbers say

H200 SXM offers more memory capacity and bandwidth than H100 SXM, while their recorded tensor compute specifications are comparable. The capacity difference matters when model state, KV cache or training memory crosses the smaller device boundary. Each workload row has its own assumptions; it is not one universal memory allowance.

H100 SXM has an on-demand minimum of $3.20/hr and H200 SXM has an on-demand minimum of $3.99/hr. The H100 SXM-to-H200 SXM on-demand price ratio is 0.8. When both configurations meet the same requirements, compare the current rates rather than assuming that either GPU is always cheaper. Spot listings have different interruption terms and are not the basis of that ratio.

Benchmark decode at the intended context length and concurrency when memory traffic is the bottleneck. For compute-bound training or prefill, measure separately instead of assuming that H200 memory capacity implies a proportional speedup. Verify the exact sold GPU count whenever the workload needs more than one device.

How to test the comparison

Hold the workload constant
Compare the same model, precision, batch size, sequence length and output-quality target. Changing these between GPUs changes the question being tested.
Measure the result you need
For serving, record latency and throughput at the intended concurrency. For training, record time for the same completed work. Check peak memory and failures in both cases.
Price the complete run
Multiply the full configuration's checkout rate by billed runtime. Compare storage, transfers and restart costs separately. A datasheet-based ratio is a screening model, not this measurement.

Before you rent

Data egress fees can raise the total bill; verify the provider's current per-GB transfer terms for your deployment region before renting.

Crossing a single GPU's VRAM ceiling can force a multi-GPU configuration, multiplying device-hours while communication overhead extends billed runtime.

Lower-priced spot rows are interruptible, so workloads without reliable checkpoint recovery risk repeated compute and missed completion windows.

Billing granularity can make short experiments cost more than runtime alone suggests; each provider's pricing page lists the applicable minimum charge.

Marlin matches your workload requirements to the lowest-priced suitable GPU across supported providers.

Memory fit and cost-per-job comparisons are models. Datasheet peaks do not establish application throughput; verify the listed configuration with your own workload. https://vast.ai/pricing · https://www.coreweave.com/pricing · https://www.hyperstack.cloud/gpu-pricing · https://www.together.ai/pricing · https://datacrunch.io/pricing · https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet · https://www.nvidia.com/en-us/data-center/h200/ · https://www.runpod.io/pricing · https://lambda.ai/service/gpu-cloud · https://crusoe.ai/cloud/pricing

Marlin beta

Stop comparing. Start running.

Marlin matches your workload to the lowest-priced suitable GPU across supported providers.

Get started with Marlin