Creating Your First Resource

This guide walks you through defining your first disaster recovery plan for a virtual machine and running a non-disruptive test recovery to verify it works. By the end, you will have a protected VM with block-level replication running between your primary and DR clusters, and you will have executed a TestFailover that proves your DR posture without touching production workloads. These two steps—protecting a VM and running a test recovery—are the foundation of every DR workflow in Trilio Site Recovery.

Before you begin, confirm the following are in place:

• OpenShift 4.14 or later on all clusters
• OpenShift Virtualization (CNV) 1.0 or later installed on the primary and DR clusters
• DRBD kernel module 9.x or later on every worker node of the primary and DR clusters
• Helm 3.0 or later on your workstation
• kubectl or oc CLI on your workstation
• Kubeconfig files for the quorum cluster, primary cluster, and DR cluster—referenced in this guide as $KUBECONFIG_QUORUM, $KUBECONFIG_CLUSTER1, and $KUBECONFIG_CLUSTER2
• Trilio Site Recovery already deployed: the site-recovery-protectionzone-controller and site-recovery-quorum-control-plane Helm charts installed on the quorum cluster, and the site-recovery-workload-control-plane chart installed on both the primary and DR clusters. If you have not done this yet, complete the deployment steps before continuing.
• DRBD Operator installed on the primary and DR clusters
• TCP ports 7000–7999 open between primary and DR worker nodes for DRBD replication traffic
• TCP port 6443 open from the quorum cluster to the primary and DR API servers
• A VM named my-vm already running in the default namespace on your primary cluster. Substitute your actual VM name and namespace throughout this guide.
• The DR deployment namespace on the quorum cluster (for example, dr-prod) already exists and matches the namespace used during Helm installation.

This guide uses the DRBD Operator deployment model with a three-cluster topology (quorum, primary, DR).

This guide does not re-install the operator itself, but you do need to apply two prerequisites before creating your first recovery plan: a DRBDReplicationPolicy that tells Site Recovery how to replicate storage between clusters, and then the ProtectionRequest that enrolls your VM.

Step 1 — Export kubeconfig paths

Set environment variables for each cluster's kubeconfig. All subsequent commands reference these variables.

export KUBECONFIG_QUORUM=~/.kube/config-quorum
export KUBECONFIG_CLUSTER1=~/.kube/config-cluster1
export KUBECONFIG_CLUSTER2=~/.kube/config-cluster2

Step 2 — Create the DRBDReplicationPolicy on the quorum cluster

The DRBDReplicationPolicy defines how storage replicates between your primary and DR clusters: which storage classes map to each other, where the DRBD endpoints listen, and whether replication is synchronous (Protocol C, RPO=0) or asynchronous (Protocol A). Create this resource in the same DR deployment namespace as your quorum control plane.

Save the following manifest as replication-policy.yaml. Replace the IP addresses and storage class names with values from your environment.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: DRBDReplicationPolicy
metadata:
  name: my-first-policy
  namespace: dr-prod
spec:
  drbdProtocol: C
  clusters:
    - name: cluster1
      role: primary
    - name: cluster2
      role: dr
  diskConfig:
    storageClassMappings:
      - primaryStorageClass: ocs-storagecluster-ceph-rbd
        drStorageClass: ocs-storagecluster-ceph-rbd
  networkConfig:
    primaryEndpoint: "10.0.0.1:7000"
    drEndpoint: "10.0.0.2:7000"
  replicationMode: synchronous
  isDefault: true

Key fields:

• drbdProtocol: C — synchronous replication; every write is committed on both clusters before the application receives an acknowledgment, achieving RPO=0. Use A for asynchronous replication over higher-latency links.
• replicationMode: synchronous — aligns the controller's scheduling behavior with the chosen DRBD protocol.
• isDefault: true — marks this policy as the default for ProtectionRequest resources that do not explicitly reference a policy.
• storageClassMappings — maps each storage class on the primary cluster to its equivalent on the DR cluster. Add an entry for every storage class your VMs use.

Apply the policy to the quorum cluster:

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f replication-policy.yaml

Verify the resource was accepted:

kubectl --kubeconfig $KUBECONFIG_QUORUM get drbdreplicationpolicy my-first-policy -n dr-prod

Step 3 — Protect your VM with a ProtectionRequest

A ProtectionRequest tells the protection-controller to enroll a specific VM in block-level replication. The controller validates the VM, creates a DRBDResource pair (one per cluster side) covering all of the VM's disks, and switches the VM to DRBD-backed frontend PVCs.

Save the following manifest as protect-my-vm.yaml. Replace my-vm and default with your VM's name and namespace.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-my-vm
  namespace: dr-prod
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: my-vm
    namespace: default
  replicationConfig:
    policyRef:
      name: my-first-policy

Apply the ProtectionRequest to the quorum cluster, where the protection-controller runs:

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f protect-my-vm.yaml

Watch protection progress:

kubectl --kubeconfig $KUBECONFIG_QUORUM get protectionrequest protect-my-vm -n dr-prod -w

The protection-controller moves the resource through the following phases: Pending → Validating → ProvisioningStorage → ConfiguringReplication → Protected. When the status shows Protected, your VM's disks are being replicated to the DR cluster in real time.

The two custom resources you created above have additional fields worth understanding as your DR requirements evolve.

DRBDReplicationPolicy

ProtectionRequest

TestFailover

FailoverRequest

Once your VM is in the Protected state, your day-to-day DR workflow involves three activities: checking replication health, running periodic test failovers to validate your DR posture, and—when needed—triggering an actual failover.

Checking replication health

The ReplicationGroupStatus resource aggregates per-volume sync state and last-sync timestamps into a single health indicator.

# View replication health for all groups in the deployment namespace
kubectl --kubeconfig $KUBECONFIG_QUORUM get replicationgroupstatus -n dr-prod

The STATUS column shows Healthy, Degraded, or Critical. A Degraded status means one or more volumes are behind but replication is still running. Critical means replication has stopped and you should investigate immediately.

To see per-volume detail:

kubectl --kubeconfig $KUBECONFIG_QUORUM get replicationgroupstatus <name> -n dr-prod -o yaml

Checking RPO events

When replication lag exceeds the thresholds configured in your DRBDReplicationPolicy, the system emits an RPOEvent.

kubectl --kubeconfig $KUBECONFIG_QUORUM get rpoevent -n dr-prod

Each event records the observed lag in seconds, the severity (warning or critical), and the affected volumes. Use these as an audit trail and to drive alerting.

Running a test failover

A TestFailover validates your DR readiness without affecting production VMs. The test-failover-controller takes snapshots of the replicated volumes, provisions test PVCs from those snapshots in an isolated namespace on the DR cluster, starts test VMs, runs verification checks, and then cleans up. Your production VMs keep running throughout.

Create a manifest named test-failover.yaml:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: test-my-vm
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-first-pg
    namespace: dr-prod
  cleanupPolicy: Manual
  retentionTime: 2h

Apply it to the quorum cluster:

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f test-failover.yaml

Monitor progress:

kubectl --kubeconfig $KUBECONFIG_QUORUM get testfailover test-my-vm -n dr-prod -w

When you are done inspecting the test VMs, delete the resource to trigger cleanup:

kubectl --kubeconfig $KUBECONFIG_QUORUM delete testfailover test-my-vm -n dr-prod

Triggering a planned failover

When you need to move VMs to the DR cluster intentionally—for example, during planned primary cluster maintenance—create a FailoverRequest with failoverType: planned. The failover-controller gracefully shuts down the VMs on the primary cluster, promotes the DRBD volumes on the DR cluster, and starts the VMs there.

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f - <<'EOF'
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-failover
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-first-pg
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: planned
EOF

Watch the operation:

kubectl --kubeconfig $KUBECONFIG_QUORUM get failoverrequest planned-failover -n dr-prod -w

The resource progresses from Pending → InProgress → Completed. If it reaches Failed, inspect the resource's status.conditions for the specific failure reason.

Example 1 — Full protection-to-test-failover walkthrough

This example shows the complete sequence from applying a ProtectionRequest through a successful TestFailover.

# 1. Apply the replication policy (if not already done)
kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f replication-policy.yaml

# 2. Protect the VM
kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f protect-my-vm.yaml

# 3. Wait for the VM to reach Protected status
kubectl --kubeconfig $KUBECONFIG_QUORUM get protectionrequest protect-my-vm -n dr-prod -w

Expected output when protection completes:

NAME             STATUS      AGE
protect-my-vm   Protected   4m12s

# 4. Confirm the DRBDResource pair was created on both clusters
kubectl --kubeconfig $KUBECONFIG_CLUSTER1 get drbdresource -A
kubectl --kubeconfig $KUBECONFIG_CLUSTER2 get drbdresource -A

Expected output (one DRBDResource per cluster side for your VM):

NAMESPACE   NAME             ROLE      SIDE      STATUS
default     my-vm-drbd       Primary   primary   Connected

# 5. Run a test failover
kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f test-failover.yaml

# 6. Watch phase transitions
kubectl --kubeconfig $KUBECONFIG_QUORUM get testfailover test-my-vm -n dr-prod -w

Expected phase progression:

NAME           PHASE               AGE
test-my-vm    Pending             0s
test-my-vm    CreatingSnapshots   8s
test-my-vm    CreatingVolumes     23s
test-my-vm    CreatingVMs         41s
test-my-vm    VerifyingData       1m02s
test-my-vm    Succeeded           1m38s

# 7. Clean up test resources
kubectl --kubeconfig $KUBECONFIG_QUORUM delete testfailover test-my-vm -n dr-prod

After deletion the controller enters the CleaningUp phase and then removes itself.

Example 2 — Checking replication health after protection

kubectl --kubeconfig $KUBECONFIG_QUORUM get replicationgroupstatus -n dr-prod -o wide

Expected output:

NAME                     HEALTH    LAST-SYNC              VOLUMES
my-vm-repgroup           Healthy   2025-01-15T14:23:01Z   1/1

To get full per-volume detail:

kubectl --kubeconfig $KUBECONFIG_QUORUM describe replicationgroupstatus my-vm-repgroup -n dr-prod

Example 3 — Inspecting a test VM during a manual-cleanup test failover

When cleanupPolicy: Manual is set, the test VMs remain running after Succeeded until you delete the TestFailover resource. This lets you SSH into the test VM or run application-level checks.

# List test VMs on the DR cluster (they run in a generated namespace)
kubectl --kubeconfig $KUBECONFIG_CLUSTER2 get vm -A | grep test

# Describe a test VM
kubectl --kubeconfig $KUBECONFIG_CLUSTER2 describe vm my-vm-test -n dr-prod-test

# When done, delete the TestFailover to trigger cleanup
kubectl --kubeconfig $KUBECONFIG_QUORUM delete testfailover test-my-vm -n dr-prod

Example 4 — Planned failover

kubectl --kubeconfig $KUBECONFIG_QUORUM apply -f - <<'EOF'
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-fo-01
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: my-first-pg
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: planned
  drainTimeoutSeconds: 120
  batchBootTimeoutSeconds: 180
EOF

kubectl --kubeconfig $KUBECONFIG_QUORUM get failoverrequest planned-fo-01 -n dr-prod -w

Expected output:

NAME             STATUS       AGE
planned-fo-01   Pending      0s
planned-fo-01   InProgress   5s
planned-fo-01   Completed    3m47s

Use the following format for each issue: Symptom → Likely cause → Fix.

ProtectionRequest stays in Pending

Symptom: The ProtectionRequest remains in Pending for more than two minutes after creation.

Likely cause: The protection-controller on the quorum cluster cannot reach the primary cluster's API server, or the VM name or namespace in the spec.virtualMachine field does not match an existing VM.

Fix:

1. Confirm the VM exists on the primary cluster:
   Copy

   kubectl --kubeconfig $KUBECONFIG_CLUSTER1 get vm my-vm -n default
   

2. Check protection-controller logs on the quorum cluster:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM logs -l app=site-recovery-quorum-control-plane -n dr-prod --tail=100
   

3. Verify port 6443 is reachable from the quorum cluster to the primary cluster's API server.

DRBDResource shows Disconnected

Symptom: After protection succeeds, kubectl get drbdresource shows Disconnected status on one or both cluster sides.

Likely cause: TCP ports 7000–7999 are blocked between primary and DR worker nodes, or the networkConfig.primaryEndpoint / drEndpoint values in your DRBDReplicationPolicy specify an unreachable address.

Fix:

1. Verify the endpoints in your policy:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM get drbdreplicationpolicy my-first-policy -n dr-prod -o jsonpath='{.spec.networkConfig}'
   

2. Test connectivity from a primary worker node to the DR endpoint on port 7000:
   Copy

   # Run on a primary worker node
   nc -zv 10.0.0.2 7000
   

3. Check drbd-node-agent logs on both clusters:
   Copy

   kubectl --kubeconfig $KUBECONFIG_CLUSTER1 logs -l app=drbd-node-agent -n trilio-site-recovery-system --tail=50
   kubectl --kubeconfig $KUBECONFIG_CLUSTER2 logs -l app=drbd-node-agent -n trilio-site-recovery-system --tail=50
   

4. If firewall rules are the cause, open ports 7000–7999 bidirectionally between all primary and DR worker nodes and re-check the DRBDResource status.

ReplicationGroupStatus shows Degraded

Symptom: kubectl get replicationgroupstatus returns Degraded for your protection group.

Likely cause: One or more volumes fell out of sync, usually due to a temporary network interruption or a primary worker node restart. DRBD is still connected but is catching up.

Fix:

1. Describe the status resource to identify which volume is behind:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM describe replicationgroupstatus <name> -n dr-prod
   

2. Check RPOEvent resources for the duration and severity of the lag:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM get rpoevent -n dr-prod
   

3. If the status does not return to Healthy within the expected resync window, check drbd-node-agent logs for sync errors. A Critical status means replication has stopped entirely—treat this as urgent and do not attempt a test failover until replication is restored.

TestFailover stays in CreatingSnapshots

Symptom: The TestFailover resource remains in the CreatingSnapshots phase for longer than five minutes.

Likely cause: The VolumeSnapshot API is unavailable on the DR cluster, or the snapshot class referenced by the test-failover-controller is not installed.

Fix:

1. Check test-failover-controller logs on the quorum cluster:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM logs -l app=site-recovery-quorum-control-plane -n dr-prod --tail=100 | grep -i snapshot
   

2. Verify that VolumeSnapshot CRDs are installed on the DR cluster:
   Copy

   kubectl --kubeconfig $KUBECONFIG_CLUSTER2 get crd volumesnapshots.snapshot.storage.k8s.io
   

3. Confirm a VolumeSnapshotClass is available and set as default on the DR cluster:
   Copy

   kubectl --kubeconfig $KUBECONFIG_CLUSTER2 get volumesnapshotclass
   

FailoverRequest reaches Failed

Symptom: A FailoverRequest transitions to Failed rather than Completed.

Likely cause: For a planned failover, VMs on the primary cluster did not shut down within the drainTimeoutSeconds window. For any failover type, the failover-controller could not promote DRBD volumes on the target cluster.

Fix:

1. Inspect the resource's status conditions for the failure reason:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM describe failoverrequest <name> -n dr-prod
   

2. Check failover-controller logs:
   Copy

   kubectl --kubeconfig $KUBECONFIG_QUORUM logs -l app=site-recovery-quorum-control-plane -n dr-prod --tail=200
   

3. If VMs failed to drain, increase drainTimeoutSeconds in the FailoverRequest spec and retry by creating a new FailoverRequest (do not reuse the failed one).
4. If volume promotion failed, verify that the DRBD volumes on the target cluster are Connected before retrying.

Collecting a full diagnostic bundle

If you cannot resolve an issue with the steps above, collect a support bundle using the tsr-gather tool and share it with Trilio support:

oc adm must-gather --image=<tsr-gather-image>:<tag>

This collects logs, CRD state, and configuration from the quorum cluster and all associated workload clusters into a single bundle. Secret data, kubeconfigs, and DRBD shared secrets are never captured. Start by reading SUMMARY.md in the bundle output for a high-level description of what was found.