GPU Server configurations
GPU Servers are provisioned by choosing a server plan, which defines the amount of CPU, memory and number of GPUs. When the server is stopped, you can change the GPU model or plan by selecting another GPU Server plan and booting the server back up.
Choosing the right GPU
- NVIDIA L4 - Image generation, speech-to-text, and basic inference workloads.
- NVIDIA L40S - Video, graphics, 3D rendering, and medium inference for production services.
- NVIDIA RTX Pro 6000 - Video, graphics, 3D rendering, and intensive inference workloads.
- NVIDIA H100 - High-throughput inference, massive batch processing, and large-model training for demanding AI workloads.
- NVIDIA B200 - Trillion-parameter inference, foundation-model training, and real-time AI for the most compute-heavy deployments.
- NVIDIA B300 - Reasoning models, long-context inference, and large-scale training that need more GPU memory than B200.
| GPU | Type | VRAM | Bandwidth | FP8 (PFLOPS) | Interconnect |
|---|---|---|---|---|---|
| L4 | PCIe | 24 GB GDDR6 | 300 GB/s | 0.48* | - |
| L40S | PCIe | 48 GB GDDR6 | 864 GB/s | 1.46* | - |
| RTX Pro 6000 | PCIe | 96 GB GDDR7 | 1.6 TB/s | 2.0 | - |
| H100 | SXM | 80 GB HBM3 | 3.35 TB/s | 3.9* | NVLink 4.0 |
| B200 | SXM | 192 GB HBM3e | 8.0 TB/s | 9.0* | NVLink 5.0 |
| B300 | SXM | 288 GB HBM3e | 8.0 TB/s | 9.0* | NVLink 5.0 |
* With sparsity. Performance without sparsity is half the figure shown.
GPU Servers with H100, B200 and B300 GPUs include NVIDIA NVLink for direct GPU-to-GPU communication. NVLink moves data between GPUs without going through CPU memory, enabling highly efficient multi-GPU workloads, distributed training, and large-scale model inference. NVLink bandwidth between GPUs is 900 GB/s on H100 and 1.8 TB/s on B200 and B300 (bidirectional).
What every plan includes
Plans include
- CPU & Memory & GPUs
- 99.999% SLA
- Billed only when server is started
Add-ons
- Block storage (any tier)
- Public IP addresses
- Backup options
Each server must have at least one block storage device for the operating system. Block storage devices can be of any size (1 GB - 4 TB each) and from any storage tier.
GPU Server plans are only billed when the server is powered on. However, attached block storages and public IPv4 addresses are reserved and thus billed even when the server is shut down.
GPU Servers with NVIDIA L4
CPU model: AMD EPYC 9575F
| CPU cores | RAM | GPU | Identifier |
|---|---|---|---|
| 8 cores | 64 GB | 1 x NVIDIA L4 | GPU-8xCPU-64GB-1xL4 |
| 12 cores | 128 GB | 1 x NVIDIA L4 | GPU-12xCPU-128GB-1xL4 |
| 12 cores | 128 GB | 2 x NVIDIA L4 | GPU-12xCPU-128GB-2xL4 |
| 16 cores | 192 GB | 1 x NVIDIA L4 | GPU-16xCPU-192GB-1xL4 |
| 16 cores | 192 GB | 2 x NVIDIA L4 | GPU-16xCPU-192GB-2xL4 |
| 16 cores | 192 GB | 3 x NVIDIA L4 | GPU-16xCPU-192GB-3xL4 |
| 20 cores | 256 GB | 1 x NVIDIA L4 | GPU-20xCPU-256GB-1xL4 |
| 20 cores | 256 GB | 2 x NVIDIA L4 | GPU-20xCPU-256GB-2xL4 |
| 20 cores | 256 GB | 3 x NVIDIA L4 | GPU-20xCPU-256GB-3xL4 |
| 32 cores | 384 GB | 2 x NVIDIA L4 | GPU-32xCPU-384GB-2xL4 |
| 32 cores | 384 GB | 3 x NVIDIA L4 | GPU-32xCPU-384GB-3xL4 |
GPU Servers with NVIDIA L40S
CPU model: AMD EPYC 9575F
| CPU cores | RAM | GPU | Identifier |
|---|---|---|---|
| 8 cores | 64 GB | 1 x NVIDIA L40S | GPU-8xCPU-64GB-1xL40S |
| 12 cores | 128 GB | 1 x NVIDIA L40S | GPU-12xCPU-128GB-1xL40S |
| 12 cores | 128 GB | 2 x NVIDIA L40S | GPU-12xCPU-128GB-2xL40S |
| 16 cores | 192 GB | 1 x NVIDIA L40S | GPU-16xCPU-192GB-1xL40S |
| 16 cores | 192 GB | 2 x NVIDIA L40S | GPU-16xCPU-192GB-2xL40S |
| 16 cores | 192 GB | 3 x NVIDIA L40S | GPU-16xCPU-192GB-3xL40S |
| 20 cores | 256 GB | 1 x NVIDIA L40S | GPU-20xCPU-256GB-1xL40S |
| 20 cores | 256 GB | 2 x NVIDIA L40S | GPU-20xCPU-256GB-2xL40S |
| 20 cores | 256 GB | 3 x NVIDIA L40S | GPU-20xCPU-256GB-3xL40S |
| 32 cores | 384 GB | 2 x NVIDIA L40S | GPU-32xCPU-384GB-2xL40S |
| 32 cores | 384 GB | 3 x NVIDIA L40S | GPU-32xCPU-384GB-3xL40S |
GPU Servers with NVIDIA RTX Pro 6000
CPU model: AMD EPYC 9555
| CPU cores | RAM | GPU | Identifier |
|---|---|---|---|
| 16 cores | 80 GB | 1 x NVIDIA RTX Pro 6000 | GPU-16xCPU-80GB-1xRTXPRO6000 |
| 32 cores | 160 GB | 2 x NVIDIA RTX Pro 6000 | GPU-32xCPU-160GB-2xRTXPRO6000 |
| 64 cores | 320 GB | 4 x NVIDIA RTX Pro 6000 | GPU-64xCPU-320GB-4xRTXPRO6000 |
| 128 cores | 640 GB | 8 x NVIDIA RTX Pro 6000 | GPU-128xCPU-640GB-8xRTXPRO6000 |
GPU Servers with NVIDIA H100
CPU model: Intel Xeon Platinum 8462Y+
| CPU cores | RAM | GPU | Identifier |
|---|---|---|---|
| 12 cores | 240 GB | 1 x NVIDIA H100 | GPU-12xCPU-240GB-1xH100 |
| 24 cores | 480 GB | 2 x NVIDIA H100 | GPU-24xCPU-480GB-2xH100 |
| 48 cores | 960 GB | 4 x NVIDIA H100 | GPU-48xCPU-960GB-4xH100 |
| 96 cores | 1920 GB | 8 x NVIDIA H100 | GPU-96xCPU-1920GB-8xH100 |
GPU Servers with NVIDIA B200
CPU models: Intel Xeon Platinum 8570, AMD EPYC 9575F
| CPU cores | RAM | GPU | Identifier |
|---|---|---|---|
| 24 cores | 240 GB | 1 x NVIDIA B200 | GPU-24xCPU-240GB-1xB200 |
| 48 cores | 480 GB | 2 x NVIDIA B200 | GPU-48xCPU-480GB-2xB200 |
| 96 cores | 960 GB | 4 x NVIDIA B200 | GPU-96xCPU-960GB-4xB200 |
| 192 cores | 1920 GB | 8 x NVIDIA B200 | GPU-192xCPU-1920GB-8xB200 |
GPU Servers with NVIDIA B300
CPU model: Intel Xeon 6768P
| CPU cores | RAM | GPU | Identifier |
|---|---|---|---|
| 28 cores | 240 GB | 1 x NVIDIA B300 | GPU-28xCPU-240GB-1xB300 |
| 56 cores | 480 GB | 2 x NVIDIA B300 | GPU-56xCPU-480GB-2xB300 |
| 112 cores | 960 GB | 4 x NVIDIA B300 | GPU-112xCPU-960GB-4xB300 |
| 224 cores | 1920 GB | 8 x NVIDIA B300 | GPU-224xCPU-1920GB-8xB300 |
Spot pricing
All GPU Server plans are also available with spot pricing, offering significant discounts compared to on-demand rates for workloads that can tolerate interruptions. Spot plans use the same identifiers with SPOT added after GPU, for example GPU-SPOT-8xCPU-64GB-1xL40S. See Spot instances for how spot capacity is reclaimed.
Free trial
GPU Servers are not included in the free trial. If you have specific requirements to try on-demand GPU Servers, please contact us.
NVIDIA is a registered trademark of NVIDIA Corporation.