Spot instances

Overview

Spot GPU Servers provide a cost-effective way to access powerful GPU computing resources. They are identical to our standard GPU Servers in terms of performance and specifications, but are offered at significantly reduced prices. This pricing model is ideal for workloads that can tolerate interruptions or are not time-critical.

The trade-off for lower costs is that spot instances can be reclaimed when the capacity is needed for on-demand GPU Servers. A graceful shutdown mechanism gives your applications time to save state and shut down cleanly.

Key features

• Significant cost savings: Lower pricing than on-demand pricing.
• Same performance: Identical hardware specifications to standard GPU Servers.
• Data persistence: Storage and IP addresses are preserved even if the instance is stopped.
• Graceful termination: An ACPI shutdown request and 30-second grace period allow for cleanup and checkpointing before a forced stop.

Ideal use cases

Spot instances work best for:

• Batch processing: Jobs that can be interrupted and restarted without significant penalty.
• Data analysis: Non-urgent analytical workloads and data processing pipelines.
• Development and testing: Safe environment for testing GPU-accelerated code.
• Rendering and media processing: Parallel workloads that can be distributed.
• Research computing: Long-running simulations that can be checkpointed.
• Image and video processing: One-time processing jobs without strict deadlines.

When not to use spot instances

Avoid spot instances for:

• Production services: Applications requiring high availability and zero interruptions.
• Real-time systems: Low-latency applications with strict SLA requirements.
• Long-running transactions: Database operations that cannot be easily rolled back.
• Time-sensitive workloads: Tasks with hard deadlines that cannot tolerate delays.

Termination mechanism

Understanding how spot instance termination works is crucial for designing resilient applications.

Graceful shutdown process

When the capacity needs to be reclaimed, your spot instance goes through a controlled stop process:

1. Graceful shutdown request: UpCloud sends an ACPI shutdown request to the guest, similar to pressing the power button on a physical machine.
2. Cleanup window (30 seconds): The guest operating system and your applications have 30 seconds to:
   • Save state and data
   • Gracefully close connections
   • Flush buffers and caches
   • Complete critical operations
3. Forced stop: After 30 seconds, the instance is forcibly powered off if it has not already stopped.
4. Preserved state: The instance enters a stopped state with all permanent storage and IP addresses preserved.

What happens to your data

• Persistent storage: Any attached storage volumes remain intact and can be accessed when the instance restarts.
• IP addresses: Assigned IP addresses are preserved with the instance.
• Temporary data: Data in RAM and any temporary storage is lost.
• Manual cleanup: If you no longer need the instance or its associated resources, you must manually delete them.

Billing

• Hourly billing: Spot instances are billed by the hour while powered on, just like standard GPU Servers.
• Reserved resources: Attached storage volumes and reserved IP addresses continue to be billed while the instance is stopped.
• Refunded credits: When UpCloud stops a spot instance, you receive account credits for the remaining unused time in the current billing hour.
• User-initiated stops: Unused-hour credits are not refunded when you stop an instance yourself.

Best practices

1. Implement checkpointing: Regularly save progress to persistent storage so work can resume from the last checkpoint.
2. Handle guest OS shutdown: Configure services to shut down cleanly when the operating system receives the ACPI shutdown request. On common Linux distributions using systemd, services typically receive SIGTERM during this shutdown sequence.
3. Use distributed workloads: Design jobs that can be distributed across multiple instances for resilience.
4. Monitor billing: Track your cost savings and compare with on-demand instances.
5. Combine with standard instances: Use spot for testing and on-demand instances for critical components.

Limitations and considerations

• Availability: Spot capacity may not always be available. Not all GPU models can be used with spot pricing.
• Interruption patterns: Interruptions may cluster during peak demand periods.

Service level agreement

Spot instances are exempt from the standard Service Level Agreement (SLA).