Using block storage volumes with services¶

Snowflake supports these storage volume types for your containerized applications: Snowflake internal stage, local storage, memory storage volumes, and block storage volumes.

Specifying block storage in service specification¶

To create a service (including job service) that uses block storage, you provide the necessary configuration in the service specification as follows:

Specify the spec.volumes field to define the block storage volumes to create.
```
volumes:
  - name: <name>
    source: block
    size: <size in Gi>
    blockConfig:                             # optional
      initialContents:
        fromSnapshot: <snapshot name>
      iops: <number-of-operations>
      throughput: <MiB per second>
```
Copy
The following fields are required:
- name: Name of the volume.
- source: Type of the volume. For block storage volume, the value is block.
- size: Storage capacity of the block storage volume measured in bytes. The value must always be an integer, specified using the Gi unit suffix. For example, 5Gi means 5*1024*1024*1024 bytes. The size value ranges for cloud providers:
  - 1Gi to 16384Gi for AWS and Azure.
  - 4Gi to 16384Gi for Google Cloud.
The following are optional fields:
- blockConfig.initialContents.fromSnapshot: Specifies the name of a snapshot (explained in the following sections) taken from another volume. The snapshot is used to initialize the block volume.
  
  The snapshot name can be a fully qualified object identifiers, such as TUTORIAL_DB.DATA_SCHEMA.MY_SNAPSHOT. Also, the snapshot name is resolved relative to the database and the schema of the service. For example, if you created your service in TUTORIAL_DB.DATA_SCHEMA, then fromSnapshot: MY_SNAPSHOT is equivalent to fromSnapshot: TUTORIAL_DB.DATA_SCHEMA.MY_SNAPSHOT.
- blockConfig.iops: Specifies the supported peak number of input/output operations per second. Note that the data size per operation is capped at 256 KiB.
  - For AWS: The supported range is 3000-16000, with a default of 3000.
  - For Azure: The supported range is 3000-80000, with a default of 3000.
  - For Google Cloud:
    - Google Cloud CPU instances: The supported range 2000-160000 with the following defaults:
      - 2000 IOPS for a 4 Gi disk size
      - 2500 IOPS for a 5 Gi disk size
      - 3000 IOPS for all other disk sizes
    - Google Cloud GPU instances: Snowflake recommends specifying only throughput. blockConfig.iops must be 16 * blockConfig.throughput for GPU instances in Google Cloud.
- blockConfig.throughput: Specifies the peak throughput, in MiB/second, to provision for the volume.
  - For AWS: The supported range is 125 - 1000, with a default of 125.
  - For Azure: The supported range is 125 - 1200, with a default of 125.
  - For Google Cloud:
    - Google Cloud CPU instances: The supported range is 140 - 2400, with the default of 140.
    - Google Cloud GPU instances: The supported range is 400 - 1,200,000, with the default of 400, but not less than 0.12 per GB of volume size.
```
volumes:
  - name: vol-1
    source: block
    size: 200Gi
    blockConfig:
      initialContents:
        fromSnapshot: snapshot1
      iops: 3000
      throughput: 125
```
Copy
Specify the spec.containers.volumeMount field to describe where in your application containers to mount the block storage volumes. The information you provide in this field is the same for all supported storage volumes.

Access control requirements¶

If you want to use an existing snapshot (fromSnapshot is in the specification) to initialize the volume, the service’s owner role must have the USAGE privilege on the snapshot.

The service’s owner role must also have the USAGE privilege on the database and schema that contain the snapshot.

Managing snapshots¶

You can take snapshots of your block storage volume and later use the backup as follows:

Use the snapshot backup to restore an existing block storage volume.
Use the snapshot backup as seed data to initialize a new block storage volume when creating a new service.

You should ensure all your updates are flushed to the disk before you take the snapshot.

Snowflake provides the following commands to create and manage snapshots:

In addition, to restore a snapshot on an existing block storage volume, you can execute the ALTER SERVICE … RESTORE VOLUME command. Note that you need to suspend the service before you can restore a snapshot. After restoring a volume, service is automatically resumed.

Block storage costs¶

For more information, see the Snowflake Service Consumption Table.

When a block storage volume is used with a job service, Snowflake stops charging block storage costs after the job service is either dropped by the user or cleaned up by Snowflake after completion.

Example¶

For an example, see Tutorial. The tutorial provides step-by-step instructions to create a service with a block storage volume mounted.

Guidelines and limitations¶

The following restrictions apply on services that use block storage volumes:

General limitations. If you encounter any issues with these limitations, contact your account representative.

The block storage volume size ranges for cloud providers:
- 1Gi to 16384Gi for AWS and Azure
- 4Gi to 16384Gi for Google Cloud
The maximum number of block storage volumes per service is 3.
The maximum number of block storage volumes per Snowflake account is 100.

The following table lists the maximum number of block storage volumes that can be mounted per compute pool node depending on the instance type of the node. Snowflake ensures that service instances using block storage volumes are placed in accordance with these limits. This might result in services in the PENDING state waiting for additional resources.

Instance family	AWS limit	Azure limit
CPU_X64_XS	22	3
CPU_X64_S	22	8
CPU_X64_M	22	16
CPU_X64_L	22	32
HIGHMEM_X64_S	22	16
HIGHMEM_X64_M	22	32
HIGHMEM_X64_L	22	32
GPU_NV_XS (Azure only)	n/a	8
GPU_NV_S (AWS only)	22	n/a
GPU_NV_SM (Azure only)	n/a	32
GPU_NV_M (AWS only)	21	n/a
GPU_NV_2M (Azure only)	n/a	32
GPU_NV_3M (Azure only)	n/a	16
GPU_NV_SL (Azure only)	n/a	32
GPU_NV_L (AWS only)	14	n/a

The maximum number of snapshots allowed per Snowflake account is 100.

The service using block storage volumes must have the same minimum and maximum number of instances.
After the service is created, the following apply:
- You can’t change the number of service instances using the ALTER SERVICE … SET … command when a service is using block storage volumes.
- You can’t change the size, iops, or throughput fields of block storage volumes.
- No new block storage volumes can be added, and no existing block storage volumes can be removed.
- Block storage volumes are preserved if a service is upgraded, or suspended and resumed. When a service is suspended, the you continues to pay for the volume because it is preserved. After you upgrade or resume a service, Snowflake attaches each block storage volume to the same service instance ID as before.
- Block storage volumes are deleted if service is dropped. To preserve data in the volumes, take snapshots of the volumes. You can use the snapshots later to initialize new volumes.
- Block storage volumes do not support Tri-Secret Secure and Periodic rekeying. This means that if your account has enabled Tri-Secret Secure or periodic rekeying, while all other Snowflake data will continue to have added security, any images stored in your Snowpark Container Services block storage volumes will not benefit from this added security.
  
  To create a block storage volume in an account with Tri-Secret Secure or periodic rekeying, you must first confirm that you understand and agree to continue without the benefit from this additional security for your block storage volumes. To confirm agreement, an account administrator (user with the ACCOUNTADMIN role) will need to set the account-level parameter ENABLE_TRI_SECRET_AND_REKEY_OPT_OUT_FOR_SPCS_BLOCK_STORAGE to TRUE.
IOPS and throughput related guidelines
- If your service IO performance is not meeting your expectations and the service is affected by block volume IO or throughput, you might consider increasing IOPS or throughput. In the current implementation, any such changes require you to recreate the service.
  
  You can review these available platform metrics to identify if your service is bottlenecked on block storage:
  - container.cpu.usage
  - volume.read.iops
  - volume.write.iops
  - volume.read.throughput
  - volume.write.throughput
- For AWS:
  - The maximum IOPS that can be configured is 500 IOPS per GiB of volume size, to a maximum of 16,000 IOPS. For example, the maximum IOPS of a 10 GiB volume can be 500 * 10 = 5000. Accordingly, note that the maximum IOPS of 16,000 can only be configured if your volume is 32 GiB or larger.
  - The maximum throughput that can be configured is 1 MiB/second for every 4 IOPS, to a maximum of 1000 MiBs/second. For example, with the default 3000 IOPS you can configure throughput up to 750 MiB/second (3000/4=750).
- For Azure:
  - After a volume size of 6 GB, the supported number of IOPS increase by 500 for each GB beyond 6 GB (disks-types). The maximum IOPS of a 10GB volume can be 500 * 4 + 3000 = 5000. Accordingly, note that the maximum IOPS of 80,000 can only be configured if your volume is 160 GiB or larger.
  - After 6 GB, the maximum throughput that can be configured is 0.25 MiB/second for every IOPS, to a maximum of 1200 MiBs/second. For example, with the default 3000 IOPS you can configure throughput up to 750 MiB/second (3000*0.25=750).
- For Google Cloud:
  - For CPU instances:
    - IOPS are configurable up to 500 IOPS per Gi of volume size, with a maximum of 160,000 IOPS. For example, a 10 Gi volume can achieve a maximum of 5,000 IOPS (500 IOPS * 10 Gi). To reach the maximum of 160,000 IOPS, a volume size of 320 Gi or larger is required.
    - A maximum throughput of 2400 MiB/second can be configured, with a rate of 1 MiB/second for every 4 IOPS. For example, 3000 IOPS enables up to 750 MiB/second throughput (3000 / 4 = 750).
  - For GPU instances:
    - IOPS cannot be set independent of throughput; IOPS is calculated as 16 multiplied by the throughput value. Therefore, specifying throughput automatically determines the IOPS. Configuring IOPS is not advised for disks used with GPU instances.
    - You must configure a minimum throughput. It must be at least 400 MiB/s, or 0.12 MiB/s for every GiB of volume size, whichever is higher.
    - The configurable throughput rate is 1600 MiB/s per GiB of volume size, subject to a maximum of 1,200,000 MiB/s. As an example, a 10 GiB volume can achieve a maximum throughput of 16,000 MiB/s (1600 * 10). Note that the upper limit of 1,200,000 MiB/s is only attainable with volumes of 750 GiB or greater.