Getting Started

This page walks you through installing Site Recovery and creating your first protected VM. You will deploy the quorum control plane, install the workload control plane on your primary and DR clusters, and submit your first Custom Resource to bring a VM under DR protection. By the end, you will have a working protection configuration that you can validate with a test failover.

Before you begin, ensure the following are in place:

Clusters

• Three Kubernetes clusters (quorum, primary, DR) for centralized storage deployments, or two clusters (primary, DR) for DRBD Operator deployments
• Kubernetes ≥ 1.28 on all clusters, or OpenShift ≥ 4.14
• KubeVirt ≥ 1.0 installed on the primary and DR clusters

Storage

• DRBD kernel module ≥ 9.0 on all worker nodes
• DRBD Operator installed on the primary and DR clusters (DRBD Operator deployment model only)
• LVM thin-provisioned storage pools on worker nodes
• VolumeSnapshot support (CSI snapshotter) on the primary and DR clusters if you plan to run test failovers

Tooling

• Helm ≥ 3.0
• kubectl configured for all three clusters
• Ansible (required for Ansible-based deployment playbooks)

Network

• TCP 7000–7999 open between primary and DR worker nodes (DRBD replication)
• TCP 6443 open from the quorum cluster to both the primary and DR Kubernetes API servers
• Round-trip latency < 10 ms between primary and DR clusters if you intend to use Protocol C (synchronous replication); Protocol A (asynchronous) has no latency requirement

Access

• Kubeconfig files with administrative access for each cluster
• The following environment variables set in your shell:

export KUBECONFIG_QUORUM=~/.kube/config-quorum
export KUBECONFIG_CLUSTER1=~/.kube/config-cluster1
export KUBECONFIG_CLUSTER2=~/.kube/config-cluster2

The steps below follow the DRBD Operator deployment model — the fastest path to a working setup. If you are using a centralized storage controller, the sequence is the same but you will use a ProtectionGroup instead of a ProtectionRequest to protect VMs.

1. Deploy the workload control plane on the primary cluster:

helm install dr-deployment ./helm/site-recovery-workload-control-plane \
  --kubeconfig $KUBECONFIG_CLUSTER1 \
  -n trilio-site-recovery-system --create-namespace

2. Deploy the workload control plane on the DR cluster:

helm install dr-deployment ./helm/site-recovery-workload-control-plane \
  --kubeconfig $KUBECONFIG_CLUSTER2 \
  -n trilio-site-recovery-system --create-namespace

3. Deploy the quorum control plane — install the ProtectionZone webhook and CRDs first, then the unified manager:

helm install dr-deployment ./helm/site-recovery-protectionzone-controller \
  --kubeconfig $KUBECONFIG_QUORUM \
  -n dr-deployment --create-namespace

helm install dr-deployment ./helm/site-recovery-quorum-control-plane \
  --kubeconfig $KUBECONFIG_QUORUM \
  -n dr-deployment --create-namespace

4. Create a DRBDReplicationPolicy that maps storage classes between your clusters and sets the replication protocol:

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f drbd-replication-policy.yaml

5. Submit a ProtectionRequest to bring a VM under DR protection:

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f protect-my-vm.yaml

6. Verify that protection has been established:

kubectl --kubeconfig $KUBECONFIG_QUORUM get protectionrequest protect-my-vm -n dr-deployment -o yaml

When the ProtectionRequest status shows the VM is fully protected and its volumes are synchronised, your first VM is under DR protection.

Step 1 — Deploy the workload control plane on the primary cluster

The workload control plane installs the node agent and admission webhooks that the quorum controllers depend on. Install it in the trilio-site-recovery-system namespace, which is reserved for Site Recovery components on workload clusters.

helm install dr-deployment ./helm/site-recovery-workload-control-plane \
  --kubeconfig $KUBECONFIG_CLUSTER1 \
  -n trilio-site-recovery-system --create-namespace

Success: helm list -n trilio-site-recovery-system --kubeconfig $KUBECONFIG_CLUSTER1 shows dr-deployment with status deployed.

Step 2 — Deploy the workload control plane on the DR cluster

Repeat the same installation on the DR cluster. Both clusters must run the workload control plane before the quorum controllers can manage them.

helm install dr-deployment ./helm/site-recovery-workload-control-plane \
  --kubeconfig $KUBECONFIG_CLUSTER2 \
  -n trilio-site-recovery-system --create-namespace

Success: The same check on $KUBECONFIG_CLUSTER2 shows deployed.

Step 3 — Deploy the quorum control plane

The quorum cluster hosts the protection-controller, failover-controller, pg-sync-controller, and the Site Manager UI. The ProtectionZone webhook and CRDs must be installed first because the unified manager references them at startup.

# Install the ProtectionZone webhook and CRDs
helm install dr-deployment ./helm/site-recovery-protectionzone-controller \
  --kubeconfig $KUBECONFIG_QUORUM \
  -n dr-deployment --create-namespace

# Install the unified quorum manager
helm install dr-deployment ./helm/site-recovery-quorum-control-plane \
  --kubeconfig $KUBECONFIG_QUORUM \
  -n dr-deployment --create-namespace

The namespace dr-deployment is the DR deployment namespace. Site Recovery isolates every deployment in its own dr-<name> namespace; all quorum-side resources for this deployment will live here.

Success: helm list -n dr-deployment --kubeconfig $KUBECONFIG_QUORUM shows both releases as deployed.

Step 4 — Define a DRBDReplicationPolicy

A DRBDReplicationPolicy tells the DRBD Operator how to replicate volumes between your clusters. It specifies the replication protocol, the replication endpoints on each cluster, and which storage class on the primary cluster maps to which storage class on the DR cluster.

Create a file named drbd-replication-policy.yaml:

apiVersion: siterecovery.trilio.io/v1
kind: DRBDReplicationPolicy
metadata:
  name: cross-cluster-policy
  namespace: dr-deployment
spec:
  drbdProtocol: C            # Protocol C = synchronous, RPO=0; use A for async over high-latency links
  storageClassMappings:
    - primaryStorageClass: ocs-storagecluster-ceph-rbd
      drStorageClass: ocs-storagecluster-ceph-rbd
  primaryCluster:
    name: cluster1
    replicationEndpoint: "10.0.0.1:7000"
  drCluster:
    name: cluster2
    replicationEndpoint: "10.0.0.2:7000"

Replace the storage class names and IP addresses with values from your environment.

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f drbd-replication-policy.yaml

Success: kubectl --kubeconfig $KUBECONFIG_QUORUM get drbdreplicationpolicy -n dr-deployment shows cross-cluster-policy.

Step 5 — Protect a VM with a ProtectionRequest

A ProtectionRequest is the declaration that you want a specific VM to be DR-protected. When the protection-controller on the quorum cluster picks it up, it validates the VM, creates a DRBDVolume for each of the VM's PVCs, waits for initial synchronisation to complete, and then switches the VM to DRBD-backed frontend PVCs.

Create a file named protect-my-vm.yaml:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-my-vm
  namespace: dr-deployment
spec:
  vmName: my-vm
  vmNamespace: default
  sourceCluster: cluster1

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f protect-my-vm.yaml

The controller progresses through the following lifecycle: validation → DRBDVolume creation → synchronisation → frontend PVC switchover → protected.

Success: Watch the status until it reports the VM is fully protected:

kubectl --kubeconfig $KUBECONFIG_QUORUM get protectionrequest protect-my-vm \
  -n dr-deployment -w

Step 6 — Verify replication health

The replication-monitor agent is deployed automatically as part of the standard Ansible playbooks and writes ReplicationGroupStatus resources that give you an aggregated health view.

# Check aggregated replication health
kubectl --kubeconfig $KUBECONFIG_QUORUM get replicationgroupstatus -n dr-deployment

# Inspect the DRBDVolume created for your VM to see per-volume sync progress
kubectl --kubeconfig $KUBECONFIG_QUORUM get drbdvolume -n dr-deployment

When the ReplicationGroupStatus shows Healthy and your DRBDVolume shows sync progress at 100%, the VM is fully protected and ready for failover.

Step 7 — (Optional) Run a test failover to validate DR readiness

Test failover is supported only in DRBD Operator deployment models. It snapshots your production volumes, boots test VMs on the DR cluster, runs verification checks, and then cleans up — without affecting production workloads.

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f - <<'EOF'
apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: validate-my-vm
  namespace: dr-deployment
spec:
  protectionGroupRef:
    name: my-first-pg
    namespace: default
  cleanupPolicy: Manual
  retentionTime: 2h
EOF

# Watch progress
kubectl --kubeconfig $KUBECONFIG_QUORUM get testfailover validate-my-vm -n dr-deployment -w

# When done, clean up test resources
kubectl --kubeconfig $KUBECONFIG_QUORUM delete testfailover validate-my-vm -n dr-deployment

Example 1 — DRBDReplicationPolicy with asynchronous replication

Use Protocol A when your primary and DR clusters are separated by more than 50 ms round-trip latency. Writes are acknowledged after committing to local disk; data is shipped to the DR cluster in the background. This accepts a small RPO window in exchange for reduced write latency.

apiVersion: siterecovery.trilio.io/v1
kind: DRBDReplicationPolicy
metadata:
  name: async-policy
  namespace: dr-deployment
spec:
  drbdProtocol: A
  storageClassMappings:
    - primaryStorageClass: local-storage
      drStorageClass: local-storage-dr
  primaryCluster:
    name: cluster1
    replicationEndpoint: "10.0.0.1:7000"
  drCluster:
    name: cluster2
    replicationEndpoint: "10.0.0.2:7000"

Example 2 — ProtectionRequest for a VM with multiple disks

The protection-controller creates one DRBDVolume per PVC that the VM owns. You only need to reference the VM — the controller discovers its volumes automatically.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-db-vm
  namespace: dr-deployment
spec:
  vmName: database-primary
  vmNamespace: production
  sourceCluster: cluster1

After applying, check the DRBDVolume resources to see one entry per disk:

kubectl --kubeconfig $KUBECONFIG_QUORUM get drbdvolume -n dr-deployment

Expected output (one row per PVC):

NAME                             SYNC PROGRESS   FRONTEND PVC                  AGE
protect-db-vm-data-disk          100%            database-primary-data-drbd    5m
protect-db-vm-log-disk           100%            database-primary-log-drbd     5m

Example 3 — FailoverRequest for a planned failover

A planned failover gracefully stops VMs on the primary cluster before promoting volumes on the DR cluster, guaranteeing zero data loss. Submit the FailoverRequest on the quorum cluster; the failover-controller orchestrates the entire sequence.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-failover-pg1
  namespace: dr-deployment
spec:
  protectionGroupRef:
    name: my-first-pg
  targetCluster: cluster2
  failoverType: planned

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f planned-failover.yaml

# Watch the operation progress
kubectl --kubeconfig $KUBECONFIG_QUORUM get failoverrequest planned-failover-pg1 \
  -n dr-deployment -w

Example 4 — Listing all protected resources across the deployment

Use these commands to get a quick health snapshot of your deployment:

# ProtectionRequests on the quorum cluster
kubectl --kubeconfig $KUBECONFIG_QUORUM get protectionrequest -n dr-deployment

# DRBDVolumes showing per-volume sync state
kubectl --kubeconfig $KUBECONFIG_QUORUM get drbdvolume -n dr-deployment

# ReplicationGroupStatus for aggregated health
kubectl --kubeconfig $KUBECONFIG_QUORUM get replicationgroupstatus -n dr-deployment

# Any RPO violations recorded by the replication-monitor
kubectl --kubeconfig $KUBECONFIG_QUORUM get rpoevent -n dr-deployment

ProtectionRequest stays in Validating phase

Symptom: The ProtectionRequest does not advance beyond Validating after several minutes.

Likely causes:

• The VM named in spec.vmName does not exist in spec.vmNamespace on the source cluster.
• The protection-controller on the quorum cluster cannot reach the source cluster's Kubernetes API (TCP 6443).
• The workload control plane is not installed on the source cluster.

Fix:

1. Confirm the VM exists: kubectl --kubeconfig $KUBECONFIG_CLUSTER1 get vm <vmName> -n <vmNamespace>
2. Confirm the quorum can reach the source API server and that TCP 6443 is open.
3. Confirm the workload control plane is deployed: helm list -n trilio-site-recovery-system --kubeconfig $KUBECONFIG_CLUSTER1
4. Check the protection-controller logs on the quorum cluster: kubectl --kubeconfig $KUBECONFIG_QUORUM logs -l app=protection-controller -n dr-deployment

DRBDVolume sync progress is stuck below 100%

Symptom: kubectl get drbdvolume -n dr-deployment shows sync progress stalled at a value below 100% for an extended period.

Likely causes:

• TCP 7000–7999 is blocked between primary and DR worker nodes.
• The DRBD kernel module is not loaded on one or more worker nodes.
• Network latency exceeds 50 ms and Protocol C is configured — the volume pair may be experiencing repeated resync.

Fix:

1. Verify port connectivity: from a DR worker node, run nc -zv <primary-worker-ip> 7000.
2. On each worker node, confirm the DRBD module is loaded: lsmod | grep drbd.
3. If latency is high, consider switching the DRBDReplicationPolicy to drbdProtocol: A.
4. Check RPOEvent resources for recorded lag violations: kubectl --kubeconfig $KUBECONFIG_QUORUM get rpoevent -n dr-deployment

Helm install fails for the quorum control plane

Symptom: helm install dr-deployment ./helm/site-recovery-quorum-control-plane fails with a webhook or CRD not found error.

Likely cause: The ProtectionZone webhook and CRDs were not installed before the unified manager. The manager references CRDs that do not yet exist in the cluster.

Fix: Install releases in the correct order:

# Step 1 — webhook and CRDs
helm install dr-deployment ./helm/site-recovery-protectionzone-controller \
  --kubeconfig $KUBECONFIG_QUORUM -n dr-deployment --create-namespace

# Step 2 — unified manager (only after step 1 completes successfully)
helm install dr-deployment ./helm/site-recovery-quorum-control-plane \
  --kubeconfig $KUBECONFIG_QUORUM -n dr-deployment --create-namespace

ReplicationGroupStatus shows Degraded or Critical

Symptom: kubectl get replicationgroupstatus -n dr-deployment reports a health status of Degraded or Critical.

Likely cause: One or more volumes in the Protection Group have fallen behind or lost their replication connection. The replication-monitor records details in RPOEvent resources.

Fix:

1. Inspect the ReplicationGroupStatus for per-volume detail: kubectl --kubeconfig $KUBECONFIG_QUORUM get replicationgroupstatus -n dr-deployment -o yaml
2. Check RPOEvent resources to see which volumes violated their RPO threshold and the recorded severity.
3. If network connectivity between clusters has been interrupted and both sides resumed as primary, you may have a split-brain condition. Do not trigger failover until the split-brain is resolved manually — promoting a diverged volume will cause data loss.
4. After resolving the underlying cause, verify that sync progress returns to 100% on all DRBDVolume resources before proceeding with any failover operations.

Test failover fails at CreatingSnapshots phase

Symptom: A TestFailover resource stalls or moves to Failed during the CreatingSnapshots phase.

Likely cause: VolumeSnapshot support (CSI snapshotter) is not installed on the cluster, or the storage class used by the VM's PVCs does not support snapshots.

Fix:

1. Confirm the CSI snapshotter is installed on the primary cluster: kubectl --kubeconfig $KUBECONFIG_CLUSTER1 get crd volumesnapshots.snapshot.storage.k8s.io
2. Confirm that the storage class used by the VM's PVCs has a matching VolumeSnapshotClass.
3. Check the test-failover-controller logs on the quorum cluster for the specific error: kubectl --kubeconfig $KUBECONFIG_QUORUM logs -l app=test-failover-controller -n dr-deployment

Note: Test failover is only supported in DRBD Operator deployment models.