This feature is under development. Documentation may not be up-to-date and features may not work as expected.
Integrating Container Storage Interface (CSI) snapshot support into Velero enables Velero to backup and restore CSI-backed volumes using the Kubernetes CSI Snapshot Beta APIs.
By supporting CSI snapshot APIs, Velero can support any volume provider that has a CSI driver, without requiring a Velero-specific plugin to be available.
The following are the prerequisites for using Velero to take Container Storage Interface (CSI) snapshots:
Ensure that the Velero server is running with the
EnableCSI feature flag. See Enabling Features for more information.
Also, the Velero CSI plugin (Docker Hub) is necessary to integrate with the CSI volume snapshot APIs.
Both of these can be added with the
velero install command.
velero install \ --features=EnableCSI \ --plugins=<object storage plugin>,velero/velero-plugin-for-csi:v0.1.0 \ ...
To include the status of CSI objects associated with a Velero backup or restore in
velero backup describe or
velero restore describe output, run
velero client config set features=EnableCSI.
See Enabling Features for more information about managing client-side feature flags.
This section documents some of the choices made during implementation of the Velero CSI plugin:
DeletionPolicyon the volumesnapshotclass is set to
Retain. To accomplish this, during deletion of the backup the prior to deleting the volumesnapshot, volumesnapshotcontent object will be patched to set its
Delete. Thus deleting volumesnapshot object will result in cascade delete of the volumesnapshotcontent and the snapshot in the storage provider.
Velero’s support level for CSI volume snapshotting will follow upstream Kubernetes support for the feature, and will reach general availability sometime after volume snapshotting is GA in upstream Kubernetes. Beta support is expected to launch in Velero v1.4.
Velero’s CSI support does not rely on the Velero VolumeSnapshotter plugin interface.
Instead, Velero uses a collection of BackupItemAction plugins that act first against PersistentVolumeClaims.
When this BackupItemAction sees PersistentVolumeClaims pointing to a PersistentVolume backed by a CSI driver, it will create a CSI VolumeSnapshot object with the PersistentVolumeClaim as a source. This VolumeSnapshot object resides in the same namespace as the PersistentVolumeClaim that was used as a source.
From there, the CSI external-snapshotter controller will see the VolumeSnapshot and create a VolumeSnapshotContent object, a cluster-scoped resource that will point to the actual, disk-based snapshot in the storage system.
The external-snapshotter plugin will call the CSI driver’s snapshot method, and the driver will call the storage system’s APIs to generate the snapshot.
Once an ID is generated and the storage system marks the snapshot as usable for restore, the VolumeSnapshotContent object will be updated with a
status.snapshotHandle and the
status.readyToUse field will be set.
Velero will include the generated VolumeSnapshot and VolumeSnapshotContent objects in the backup tarball, as well as upload all VolumeSnapshots and VolumeSnapshotContents objects in a JSON file to the object storage system. When Velero synchronizes backups into a new cluster, VolumeSnapshotContent objects will be synced into the cluster as well, so that Velero can manage backup expiration appropriately.
VolumeSnapshotContents will have a
Retain for the life of the Velero backup in order to prevent them from being deleted on the storage system.
When the Velero backup expires, the VolumeSnapshot objects will be deleted and the VolumeSnapshotContent objects will be updated to have a
Delete, in order to free space on the storage system.
For more details on how each plugin works, see the CSI plugin repo’s documentation.