This is the documentation for the latest development version of Velero. Both code and docs may be unstable, and these docs are not guaranteed to be up to date or correct. See the latest version.

Node-agent Prepare Queue Length

During CSI Snapshot Data Movement, Velero built-in data mover launches data mover pods to run the data transfer.
During fs-backup, Velero also launches data mover pods to run the data transfer.
Other intermediate resources may also be created along with the data mover pods, i.e., PVCs, VolumeSnapshots, VolumeSnapshotContents, etc.

Velero uses node-agent Concurrency Configuration to control the number of concurrent data transfer activities across the nodes, by default, the concurrency is 1 per node.

when the parallelism across the available nodes are much lower than the total number of volumes to be backed up/restored, the intermediate objects may exist for much longer time unnecessarily, which takes unnecessary resources from the cluster.
The available nodes are decided by various factors, e.g., node OS type (linux or Windows), Node Selection (for CSI Snapshot Data Movement only), etc.

Velero allows you to configure the prepareQueueLength in node-agent Configuration, which defines the maximum number of DataUpload/DataDownload/PodVolumeBackup/PodVolumeRestore CRs under the preparation statuses but are not yet processed by any node (e.g., in phases of Accepted, Prepared). In this way, the number of intermediate objects are constrained.

Sample

Here is a sample of the configMap with prepareQueueLength:

{
    "prepareQueueLength": 10
}

To create the configMap, save something like the above sample to a json file and then run below command:

kubectl create cm node-agent-config -n velero --from-file=<json file name>

To provide the configMap to node-agent, edit the node-agent daemonset and add the - --node-agent-config argument to the spec:

Open the node-agent daemonset spec

kubectl edit ds node-agent -n velero

Add - --node-agent-config to spec.template.spec.containers

spec:
  template:
    spec:
      containers:
      - args:
        - --node-agent-config=<configMap name>

Documentation

Node-agent Prepare Queue Length

Sample

Getting Started