Dell PowerFlex - powerflex
Description: Dell PowerFlex is a software-defined storage solution from Dell Technologies. Among other things it offers the consumption of redundant block storage across the network. LXD offers access to PowerF...

Hide navigation sidebar
Hide table of contents sidebar
Skip to content
Back to top
View this page
Toggle table of contents sidebar

# Dell PowerFlex - `powerflex`

Dell PowerFlex is a software-defined storage solution from Dell Technologies. Among other things it offers the consumption of redundant block storage across the network.

LXD offers access to PowerFlex storage clusters using either NVMe/TCP or Dell’s Storage Data Client (SDC). In addition, PowerFlex offers copy-on-write snapshots, thin provisioning and other features.

To use PowerFlex with NVMe/TCP, make sure you have the required kernel modules installed on your host system. On Ubuntu these are `nvme_fabrics` and `nvme_tcp`, which come bundled in the `linux-modules-extra-$(uname -r)` package. LXD takes care of connecting to the respective subsystem.

When using the SDC, LXD requires it to already be connected to the Dell Metadata Manager (MDM) beforehand. As LXD doesn’t set up the SDC, follow the official guides from Dell for configuration details.

## Terminology

PowerFlex groups various so-called SDS under logical groups within a protection domain. Those SDS are the hosts that contribute storage capacity to the PowerFlex cluster. A protection domain contains storage pools, which represent a set of physical storage devices from different SDS. LXD creates its volumes in those storage pools.

You can take a snapshot of any volume in PowerFlex, which will create an independent copy of the parent volume. PowerFlex volumes get added as a drive to the respective LXD host the volume got mapped to. In case of NVMe/TCP, the LXD host connects to one or multiple NVMe SDT provided by PowerFlex. Those SDT run as components on the PowerFlex storage layer. In case of SDC, the LXD hosts don’t set up any connection by themselves. Instead they depend on the SDC to make the volumes available on the system for consumption.

## `powerflex` driver in LXD

The `powerflex` driver in LXD uses PowerFlex volumes for custom storage volumes, instances and snapshots. For storage volumes with content type `filesystem` (containers and custom file-system volumes), the `powerflex` driver uses volumes with a file system on top (see `block.filesystem`). By default, LXD creates thin-provisioned PowerFlex volumes.

LXD expects the PowerFlex protection domain and storage pool already to be set up. Furthermore, LXD assumes that it has full control over the storage pool. Therefore, you should never maintain any volumes that are not owned by LXD in a PowerFlex storage pool, because LXD might delete them.

This driver behaves differently than some of the other drivers in that it provides remote storage. As a result and depending on the internal network, storage access might be a bit slower than for local storage. On the other hand, using remote storage has big advantages in a cluster setup, because all cluster members have access to the same storage pools with the exact same contents, without the need to synchronize storage pools.

When creating a new storage pool using the `powerflex` driver in `nvme` mode, LXD tries to discover one of the SDT from the given storage pool. Alternatively, you can specify which SDT to use with `powerflex.sdt`. LXD instructs the NVMe initiator to connect to all the other SDT when first connecting to the subsystem.

Due to the way copy-on-write works in PowerFlex, snapshots of any volume don’t rely on its parent. As a result, volume snapshots are fully functional volumes themselves, and it’s possible to take additional snapshots from such volume snapshots. This tree of dependencies is called the PowerFlex vTree. Both volumes and their snapshots get added as standalone disks to the LXD host.

### Volume names

Due to a limitation in PowerFlex, volume names cannot exceed 31 characters. Therefore the driver is using the volume’s `volatile.uuid` to generate a fixed length volume name. A UUID of `5a2504b0-6a6c-4849-8ee7-ddb0b674fd14` will render to the base64-encoded string `WiUEsGpsSEmO592wtnT9FA==`.

To be able to identify the volume types and snapshots, special identifiers are prepended to the volume names:

Type | Identifier | Example
--- | --- | ---
Container | `c_` | `c_WiUEsGpsSEmO592wtnT9FA==`
Virtual machine | `v_` | `v_WiUEsGpsSEmO592wtnT9FA==.b`
Image (ISO) | `i_` | `i_WiUEsGpsSEmO592wtnT9FA==.i`
Custom volume | `u_` | `u_WiUEsGpsSEmO592wtnT9FA==`

### Limitations

The `powerflex` driver has the following limitations:

Limit of snapshots in a single vTree

  An internal limitation in the PowerFlex vTree does not allow to take more than 126 snapshots of any volume in PowerFlex. This limit also applies to any child of any of the parent volume’s snapshots. A single vTree can only have 126 branches.

Non-optimized image storage

  Due to the limit of 126 snapshots in the vTree, the PowerFlex driver doesn’t come with support for optimized image storage. This would limit LXD to create only 126 instances from an image. Instead, when launching a new instance, the image’s contents get copied to the instance’s root volume.

Copying volumes

  PowerFlex does not support creating a copy of the volume so that it gets its own vTree. Therefore, LXD falls back to copying the volume on the local system. This implicates an increased use of bandwidth due to the volume’s contents being transferred over the network twice.

Volume size constraints

  In PowerFlex, the size (quota) of a volume must be in multiples of 8 GiB. This results in the smallest possible volume size of 8 GiB. However, if not specified otherwise, volumes are getting thin-provisioned by LXD. PowerFlex volumes can only be increased in size.

Sharing custom volumes between instances

  The PowerFlex driver “simulates” volumes with content type `filesystem` by putting a file system on top of a PowerFlex volume. Therefore, custom storage volumes can only be assigned to a single instance at a time.

Sharing the PowerFlex storage pool between installations

  Sharing the same PowerFlex storage pool between multiple LXD installations is not supported.

Recovering PowerFlex storage pools

  Recovery of PowerFlex storage pools using `lxd recover` is not supported.

## Configuration options

The following configuration options are available for storage pools that use the `powerflex` driver and for storage volumes in these pools.

### Storage pool configuration

`powerflex.clone_copy`

Whether to use non-sparse copies for snapshots

Key: | `powerflex.clone_copy`
Type: | bool
Default: | `true`
Scope: | global

If this option is set to `true`, PowerFlex makes a non-sparse copy when creating a snapshot of an instance or custom volume. See Limitations for more information.

`powerflex.domain`

Name of the PowerFlex protection domain

Key: | `powerflex.domain`
Type: | string
Scope: | global

This option is required only if `powerflex.pool` is specified using its name.

`powerflex.gateway`

Address of the PowerFlex Gateway

Key: | `powerflex.gateway`
Type: | string
Scope: | global

`powerflex.gateway.verify`

Whether to verify the PowerFlex Gateway’s certificate

Key: | `powerflex.gateway.verify`
Type: | bool
Default: | `true`
Scope: | global

`powerflex.mode`

How volumes are mapped to the local server

Key: | `powerflex.mode`
Type: | string
Default: | the discovered mode
Scope: | global

The mode gets discovered automatically if the system provides the necessary kernel modules. This can be either `nvme` or `sdc`.

`powerflex.pool`

ID of the PowerFlex storage pool

Key: | `powerflex.pool`
Type: | string
Scope: | global

If you want to specify the storage pool via its name, also set `powerflex.domain`.

`powerflex.sdt`

Comma separated list of PowerFlex NVMe/TCP SDTs

Key: | `powerflex.sdt`
Type: | string
Scope: | global

`powerflex.user.name`

User for PowerFlex Gateway authentication

Key: | `powerflex.user.name`
Type: | string
Default: | `admin`
Scope: | global

Must have at least SystemAdmin role to give LXD full control over managed storage pools.

`powerflex.user.password`

Password for PowerFlex Gateway authentication

Key: | `powerflex.user.password`
Type: | string
Scope: | global

`rsync.bwlimit`

Upper limit on the socket I/O for `rsync`

Key: | `rsync.bwlimit`
Type: | string
Default: | `0` (no limit)
Scope: | global

When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O.

`rsync.compression`

Whether to use compression while migrating storage pools

Key: | `rsync.compression`
Type: | bool
Default: | `true`
Scope: | global

`volume.size`

Size/quota of the storage volume

Key: | `volume.size`
Type: | string
Default: | `8GiB`
Scope: | global

The size must be in multiples of 8 GiB. See Limitations for more information.

Tip

In addition to these configurations, you can also set default values for the storage volume configurations. See Configure default values for storage volumes.

### Storage volume configuration

`block.filesystem`

File system of the storage volume

Key: | `block.filesystem`
Type: | string
Default: | same as `volume.block.filesystem`
Condition: | block-based volume with content type `filesystem`
Scope: | global

Valid options are: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed.

`block.mount_options`

Mount options for block-backed file system volumes

Key: | `block.mount_options`
Type: | string
Default: | same as `volume.block.mount_options`
Condition: | block-based volume with content type `filesystem`
Scope: | global

`block.type`

Whether to create a `thin` or `thick` provisioned volume

Key: | `block.type`
Type: | string
Default: | same as `volume.block.type` or `thick`
Scope: | global

`security.shared`

Enable volume sharing

Key: | `security.shared`
Type: | bool
Default: | same as `volume.security.shared` or `false`
Condition: | virtual-machine or custom block volume
Scope: | global

Enabling this option allows sharing the volume across multiple instances despite the possibility of data loss.

`security.shifted`

Enable ID shifting overlay

Key: | `security.shifted`
Type: | bool
Default: | same as `volume.security.shifted` or `false`
Condition: | custom volume
Scope: | global

Enabling this option allows attaching the volume to multiple isolated instances.

`security.unmapped`

Disable ID mapping for the volume

Key: | `security.unmapped`
Type: | bool
Default: | same as `volume.security.unmappped` or `false`
Condition: | custom volume
Scope: | global

`size`

Size/quota of the storage volume

Key: | `size`
Type: | string
Default: | same as `volume.size`
Scope: | global

The size must be in multiples of 8 GiB. See Limitations for more information.

`snapshots.expiry`

When snapshots are to be deleted

Key: | `snapshots.expiry`
Type: | string
Default: | same as `volume.snapshots.expiry`
Condition: | custom volume
Scope: | global

Specify an expression like `1M 2H 3d 4w 5m 6y`.

`snapshots.pattern`

Template for the snapshot name

Key: | `snapshots.pattern`
Type: | string
Default: | same as `volume.snapshots.pattern` or `snap%d`
Condition: | custom volume
Scope: | global

You can specify a naming template that is used for scheduled snapshots and unnamed snapshots.

The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name.

To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second.

Another way to avoid name collisions is to use the placeholder `%d` in the pattern. For the first snapshot, the placeholder is replaced with `0`. For subsequent snapshots, the existing snapshot names are taken into account to find the highest number at the placeholder’s position. This number is then incremented by one for the new name.

`snapshots.schedule`

Schedule for automatic volume snapshots

Key: | `snapshots.schedule`
Type: | string
Default: | same as `snapshots.schedule`
Condition: | custom volume
Scope: | global

Specify either a cron expression (`<minute> <hour> <dom> <month> <dow>`), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default).

`volatile.idmap.last`

JSON-serialized UID/GID map that has been applied to the volume

Key: | `volatile.idmap.last`
Type: | string
Condition: | filesystem

`volatile.idmap.next`

JSON-serialized UID/GID map that has been applied to the volume

Key: | `volatile.idmap.next`
Type: | string
Condition: | filesystem

`volatile.uuid`

The volume’s UUID

Key: | `volatile.uuid`
Type: | string
Default: | random UUID
Scope: | global