Custom Resource Schemas

This page documents the Custom Resource Definition (CRD) schemas for Trilio Site Recovery. Each CRD represents a discrete piece of DR state—protection configuration, replication policy, failover intent, or health observation—and is the primary way you interact with the operator. Understanding the schema of each resource, including its required fields, valid values, and validation rules, lets you author manifests confidently, write GitOps policies, and diagnose status conditions without guesswork. All resources share the API group siterecovery.trilio.io/v1alpha1 unless noted.

Before working with these schemas you need:

• OpenShift 4.14 or later
• OpenShift Virtualization (CNV) 1.0 or later
• Trilio Site Recovery installed: site-recovery-quorum-control-plane Helm chart deployed to the quorum cluster, site-recovery-workload-control-plane chart deployed to the primary and DR clusters
• site-recovery-protectionzone-controller chart deployed to the quorum cluster before the quorum control plane chart
• kubectl or oc CLI with kubeconfig contexts for all clusters
• Helm 3.0 or later (for installation; not required for day-to-day CRD operations)
• DRBD kernel module 9.x on all worker nodes
• The DRBD Operator installed on primary and DR clusters (DRBD Operator deployment model)

The CRD manifests are bundled with the Helm charts and applied automatically during chart installation. You do not need to install CRDs manually in normal deployments. If you need to apply or refresh CRD manifests independently—for example, after a partial installation or during an upgrade—use kubectl apply directly.

Step 1 — Install the ProtectionZone controller (quorum cluster first)

This chart must be installed before the quorum control plane because it registers the ProtectionZone admission webhook and CRDs that the quorum control plane depends on.

helm upgrade --install site-recovery-protectionzone-controller \
  trilio/site-recovery-protectionzone-controller \
  --namespace dr-system \
  --create-namespace \
  --kubeconfig ~/.kube/config-quorum

Step 2 — Install the quorum control plane (quorum cluster)

helm upgrade --install site-recovery-quorum-control-plane \
  trilio/site-recovery-quorum-control-plane \
  --namespace dr-system \
  --kubeconfig ~/.kube/config-quorum

This deploys the failover-controller, protection-controller, pg-sync-controller, and test-failover-controller, and registers the FailoverRequest, ProtectionRequest, TestFailover, RPOEvent, ReplicationGroupStatus, and ProtectionGroup CRDs on the quorum cluster.

Step 3 — Install the workload control plane (primary cluster)

helm upgrade --install site-recovery-workload-control-plane \
  trilio/site-recovery-workload-control-plane \
  --namespace dr-system \
  --create-namespace \
  --kubeconfig ~/.kube/config-primary

Step 4 — Install the workload control plane (DR cluster)

helm upgrade --install site-recovery-workload-control-plane \
  trilio/site-recovery-workload-control-plane \
  --namespace dr-system \
  --create-namespace \
  --kubeconfig ~/.kube/config-dr

Steps 3 and 4 install the drbd-node-agent DaemonSet, admission webhooks, and the DRBDReplicationPolicy and DRBDResource CRDs on the workload clusters.

Step 5 — Verify CRD registration

# Quorum cluster
kubectl get crds --kubeconfig ~/.kube/config-quorum | grep siterecovery

# Primary cluster
kubectl get crds --kubeconfig ~/.kube/config-primary | grep siterecovery

All siterecovery.trilio.io CRDs should appear in the output before you proceed to create resources.

Each CRD has its own spec fields. The tables below list every field defined in the authoritative code surface, its type, whether it is required, and its effect on operator behavior.

ProtectionZone

Defines a named DR deployment that links a primary cluster, a DR cluster, and the quorum cluster under a single management namespace (dr-<name>). Install one ProtectionZone per DR deployment before creating any other resources.

ProtectionGroup

Groups one or more VMs so they fail over together as a coordinated unit. The admission webhook enforces that all member VMs are in the same namespace (see Troubleshooting for the multi-namespace rejection behavior).

Status fields (read-only)

The protection-group-controller writes the following to .status:

• state: Active, Degraded
• replicationHealth: Healthy, Degraded, Critical
• currentState: running, stopped, mixed
• protectedVMs: per-VM replication state array

ProtectionRequest

Requests block-level replication protection for a single VM in the DRBD Operator deployment model. Creating this resource causes the protection-controller to validate the VM, create a DRBDResource pair, and switch the VM to DRBD-backed frontend PVCs.

Status phases: Pending → Validating → ProvisioningDRBDResource → SwitchingPVCs → Protected (or Failed)

DRBDReplicationPolicy

Defines how volumes replicate between the primary and DR clusters: which storage classes to map, the replication endpoints, and whether to use synchronous (Protocol C) or asynchronous (Protocol A) replication. One policy can be marked as the default for a ProtectionZone.

DRBDResource

Represents all disks of a single VM as one replicated DRBD resource. Created automatically by the protection-controller when processing a ProtectionRequest; you do not normally create these manually.

FailoverRequest

Triggers a planned or unplanned failover for a ProtectionGroup. Creating this resource causes the failover-controller to stop VMs on the source cluster, promote DRBD volumes on the target cluster, and start VMs there.

Status phases: Pending → InProgress → Completed (or Failed)

Internal sub-phases (written to .status.phase): StoppingOnSource → WaitingForDRBD → StartingOnTarget → Completed

TestFailover

Triggers a non-disruptive DR validation using volume snapshots. The test-failover-controller creates snapshot-based copies of VMs in an isolated namespace on the DR cluster, runs verification checks, and cleans up—without affecting production workloads.

Status phases: CreatingSnapshots → ProvisioningTestPVCs → StartingTestVMs → VerifyingData → Succeeded (or Failed) → CleaningUp

ReplicationGroupStatus

Aggregates replication health for a ProtectionGroup into a single resource. The replication-monitor reconciler creates and updates this resource automatically; you create the CR to register a group for monitoring.

Status fields (read-only): health (Healthy, Degraded, Critical), lastSyncTimestamp, per-volume sync detail array.

RPOEvent

Records a single replication lag violation for audit and alerting. Created automatically by the replication-monitor when observed lag exceeds the rpoObjectiveSeconds configured in the associated ReplicationGroupStatus.

All day-to-day DR operations are performed by creating or patching Custom Resource manifests with kubectl or oc. The operator controllers continuously reconcile the desired state expressed in these resources against the actual state of VMs, DRBD volumes, and cluster connectivity.

Common patterns

1. Establish a DR deployment

Create a ProtectionZone on the quorum cluster to register the primary and DR cluster pair under a management namespace:

kubectl apply -f protectionzone.yaml --kubeconfig ~/.kube/config-quorum

2. Define a replication policy

Create a DRBDReplicationPolicy on the primary cluster to specify storage class mappings and the replication protocol:

kubectl apply -f drbd-replication-policy.yaml --kubeconfig ~/.kube/config-primary

3. Protect a VM

Create a ProtectionRequest on the quorum cluster. The protection-controller validates the VM, provisions the DRBDResource pair, and switches the VM to DRBD-backed frontend PVCs:

kubectl apply -f protection-request.yaml --kubeconfig ~/.kube/config-quorum
# Watch protection phases
kubectl get protectionrequest <name> -n <namespace> -w --kubeconfig ~/.kube/config-quorum

4. Group VMs for coordinated failover

Create a ProtectionGroup that references the protected VMs. All VMs must be in the same namespace:

kubectl apply -f protection-group.yaml --kubeconfig ~/.kube/config-primary

5. Monitor replication health

Create a ReplicationGroupStatus to register the group for continuous health monitoring:

kubectl apply -f replication-group-status.yaml --kubeconfig ~/.kube/config-quorum
# Check aggregated health
kubectl get replicationgroupstatus <name> -n <namespace> -o jsonpath='{.status.health}' --kubeconfig ~/.kube/config-quorum

6. Run a non-disruptive test failover

Create a TestFailover on the quorum cluster. This does not affect production VMs:

kubectl apply -f test-failover.yaml --kubeconfig ~/.kube/config-quorum
kubectl get testfailover <name> -n <namespace> -w --kubeconfig ~/.kube/config-quorum

7. Execute a planned failover

Create a FailoverRequest on the quorum cluster with failoverType: planned:

kubectl apply -f failover-request.yaml --kubeconfig ~/.kube/config-quorum
kubectl get failoverrequest <name> -n <namespace> -w --kubeconfig ~/.kube/config-quorum

8. Execute an unplanned failover

Create a FailoverRequest with failoverType: unplanned. The failover-controller force-promotes DRBD volumes on the DR cluster without waiting for the primary cluster to respond:

kubectl apply -f unplanned-failover-request.yaml --kubeconfig ~/.kube/config-quorum

9. Audit RPO compliance

List all RPOEvents for a namespace to review historical replication lag violations:

kubectl get rpoevents -n <namespace> --sort-by='.spec.timestamp' --kubeconfig ~/.kube/config-quorum

Each example below is a complete, self-contained manifest. Replace placeholder values (angle brackets) with values from your environment.

Example 1 — ProtectionZone: register a DR deployment

Create this on the quorum cluster before any other resources. The namespace dr-production is created by the ProtectionZone controller.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: production
  namespace: dr-production
spec:
  displayName: "Production DR Zone"
  description: "Primary-to-DR replication for production VMs"
  storageBackendMode: drbd-operator
  clusters:
    - name: primary-cluster
      role: primary
      kubeConfigSecret: primary-cluster-kubeconfig
    - name: dr-cluster
      role: dr
      kubeConfigSecret: dr-cluster-kubeconfig

Apply and verify:

kubectl apply -f protectionzone.yaml --kubeconfig ~/.kube/config-quorum
kubectl get protectionzone production -n dr-production --kubeconfig ~/.kube/config-quorum

Example 2 — DRBDReplicationPolicy: synchronous replication

Create on the primary cluster. This policy uses Protocol C (RPO=0) and is the default for all ProtectionRequests in the zone.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: DRBDReplicationPolicy
metadata:
  name: sync-policy
  namespace: dr-production
spec:
  clusters:
    - name: primary-cluster
      role: source
    - name: dr-cluster
      role: target
  replicationMode: synchronous
  drbdProtocol: C
  isDefault: true
  diskConfig:
    storageClassMappings:
      - sourceStorageClass: standard-rwo
        targetStorageClass: standard-rwo-dr
  networkConfig:
    portRangeStart: 7000
    portRangeEnd: 7999
  rpo:
    objectiveSeconds: 0

Note: drbdProtocol: C requires round-trip latency below 50ms between primary and DR worker nodes on TCP ports 7000–7999.

Example 3 — ProtectionRequest: protect a single VM

Create on the quorum cluster. The protection-controller provisions a DRBDResource pair and switches the VM to DRBD-backed frontend PVCs.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-vm-01
  namespace: dr-production
spec:
  sourceCluster: primary-cluster
  virtualMachine:
    name: web-vm-01
    namespace: production

Watch the protection phases:

kubectl get protectionrequest protect-web-vm-01 -n dr-production -w \
  --kubeconfig ~/.kube/config-quorum

Expected output (phases progress automatically):

NAME                  PHASE                      AGE
protect-web-vm-01     Pending                    2s
protect-web-vm-01     Validating                 5s
protect-web-vm-01     ProvisioningDRBDResource   12s
protect-web-vm-01     SwitchingPVCs              28s
protect-web-vm-01     Protected                  45s

Example 4 — ProtectionGroup: group VMs for coordinated failover

Create on the primary cluster. All VMs must be in the production namespace.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: production-group
  namespace: production
spec:
  sourceCluster: primary-cluster
  virtualMachines:
    - name: web-vm-01
    - name: web-vm-02
    - name: db-vm-01
  sla:
    rpoObjectiveSeconds: 0

Verify replication health:

kubectl get protectiongroup production-group -n production \
  -o jsonpath='{.status.replicationHealth}' \
  --kubeconfig ~/.kube/config-primary

Expected output:

Healthy

Example 5 — TestFailover: non-disruptive DR validation

Create on the quorum cluster. Test VMs are started in an isolated namespace and cleaned up automatically after one hour.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: quarterly-dr-test
  namespace: dr-production
spec:
  protectionGroupRef:
    name: production-group
    namespace: production
  cleanupPolicy: Automatic
  retentionTime: 1h
  timeout: 30m
  batchBootTimeoutSeconds: 300
  verification:
    runHealthProbes: true

Watch phases:

kubectl get testfailover quarterly-dr-test -n dr-production -w \
  --kubeconfig ~/.kube/config-quorum

Expected output:

NAME                 PHASE               AGE
quarterly-dr-test    CreatingSnapshots   5s
quarterly-dr-test    ProvisioningTestPVCs 18s
quarterly-dr-test    StartingTestVMs     35s
quarterly-dr-test    VerifyingData       2m10s
quarterly-dr-test    Succeeded           4m02s
quarterly-dr-test    CleaningUp          4m05s

Example 6 — FailoverRequest: planned failover

Create on the quorum cluster. The failover-controller gracefully shuts down VMs on the primary cluster, promotes DRBD volumes on the DR cluster, and starts VMs there.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-failover-20240315
  namespace: dr-production
spec:
  protectionGroupRef:
    name: production-group
    namespace: production
  targetCluster: dr-cluster
  failoverType: planned
  drainTimeoutSeconds: 120
  batchBootTimeoutSeconds: 300

Watch the failover progress:

kubectl get failoverrequest planned-failover-20240315 -n dr-production -w \
  --kubeconfig ~/.kube/config-quorum

Expected output:

NAME                          PHASE              AGE
planned-failover-20240315     Pending            2s
planned-failover-20240315     StoppingOnSource   8s
planned-failover-20240315     WaitingForDRBD     42s
planned-failover-20240315     StartingOnTarget   55s
planned-failover-20240315     Completed          3m22s

Example 7 — FailoverRequest: unplanned (emergency) failover

Use when the primary cluster is unreachable. DRBD volumes are force-promoted on the DR cluster.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: emergency-failover-20240315
  namespace: dr-production
spec:
  protectionGroupRef:
    name: production-group
    namespace: production
  targetCluster: dr-cluster
  failoverType: unplanned
  batchBootTimeoutSeconds: 300

Warning: With failoverType: unplanned, if the DRBDReplicationPolicy uses Protocol A (asynchronous), VMs on the DR cluster will start from the last replicated state, which may be seconds behind the primary. Protocol C (synchronous) deployments retain RPO=0 even in unplanned failovers.

Example 8 — ReplicationGroupStatus: register a group for health monitoring

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ReplicationGroupStatus
metadata:
  name: production-group-health
  namespace: dr-production
spec:
  protectionGroupRef:
    name: production-group
    namespace: production
  replicationProtocol: C
  rpoObjectiveSeconds: 0
  pollingIntervalSeconds: 30

Query aggregated health:

kubectl get replicationgroupstatus production-group-health -n dr-production \
  -o jsonpath='{.status}' \
  --kubeconfig ~/.kube/config-quorum | jq .

Expected output:

{
  "health": "Healthy",
  "lastSyncTimestamp": "2024-03-15T14:23:01Z",
  "volumes": [
    { "pvcName": "web-vm-01-disk", "syncState": "UpToDate" },
    { "pvcName": "db-vm-01-disk",  "syncState": "UpToDate" }
  ]
}

Use the following patterns to diagnose common failures. For any issue that requires log inspection across clusters, run tsr-gather against the quorum cluster to collect a unified diagnostic bundle before opening a support case.

# Collect a full diagnostic bundle
oc adm must-gather --image=<tsr-gather-image> --kubeconfig ~/.kube/config-quorum

Issue: ProtectionRequest stuck in Validating phase

Symptom: A ProtectionRequest remains in the Validating phase for more than two minutes and does not progress to ProvisioningDRBDResource.

Likely causes:

• The referenced VirtualMachine does not exist or is in a different namespace than stated in spec.virtualMachine.namespace.
• The VM's PVCs use a storage class that is not mapped in any DRBDReplicationPolicy.
• No DRBDReplicationPolicy exists for this ProtectionZone, or none has isDefault: true.

Fix:

# Check protection-controller logs on the quorum cluster
kubectl logs -n dr-system -l app=site-recovery-quorum-control-plane \
  --kubeconfig ~/.kube/config-quorum | grep -i "protect-request-name"

# Verify the VM exists on the primary cluster
kubectl get vm <vm-name> -n <namespace> --kubeconfig ~/.kube/config-primary

# Verify a default DRBDReplicationPolicy exists
kubectl get drbdreplicationpolicy -n dr-production \
  -o jsonpath='{range .items[*]}{.metadata.name}{" isDefault="}{.spec.isDefault}{"\n"}{end}' \
  --kubeconfig ~/.kube/config-primary

Issue: ProtectionGroup admission webhook rejects a create or update with a namespace error

Symptom: kubectl apply returns an error similar to: admission webhook denied: all VMs in a ProtectionGroup must be in the same namespace.

Likely cause: You referenced VMs from more than one namespace in spec.virtualMachines. The validating webhook enforces single-namespace membership per group (tracked as TSR-162; multi-namespace support is tracked by TSR-157).

Fix: Split VMs into separate ProtectionGroups, one per namespace. You cannot work around this restriction with --force or by pre-creating the resource; the webhook rejects any create or update that would leave members spanning namespaces.

# Identify which VMs span namespaces
kubectl get protectiongroup <name> -n <namespace> \
  -o jsonpath='{.spec.virtualMachines}' \
  --kubeconfig ~/.kube/config-primary | jq .

Issue: FailoverRequest stuck in StoppingOnSource phase

Symptom: A FailoverRequest has been in the StoppingOnSource phase longer than the configured drainTimeoutSeconds.

Likely causes:

• The failover-controller cannot reach the primary cluster API server (port 6443 from quorum cluster blocked).
• A VM on the primary cluster is stuck in a shutdown loop due to a guest OS issue.
• The ProtectionGroup on the primary cluster is in a Degraded state and the protection-group-controller is not reconciling.

Fix:

# Check failover-controller logs
kubectl logs -n dr-system -l app=site-recovery-quorum-control-plane \
  --kubeconfig ~/.kube/config-quorum | grep -i "failover"

# Check connectivity from quorum cluster to primary API
kubectl get nodes --kubeconfig ~/.kube/config-primary

# Check ProtectionGroup status on primary
kubectl get protectiongroup <pg-name> -n <namespace> \
  -o jsonpath='{.status}' \
  --kubeconfig ~/.kube/config-primary | jq .

# If primary is truly unreachable, re-create the FailoverRequest with unplanned type
kubectl delete failoverrequest <name> -n dr-production --kubeconfig ~/.kube/config-quorum
# Then apply an updated manifest with failoverType: unplanned

Issue: TestFailover fails at VerifyingData phase

Symptom: A TestFailover reaches VerifyingData and then transitions to Failed. The test VMs started successfully.

Likely causes:

• A verification probe defined in spec.verification failed (application did not respond within the timeout).
• The snapshot taken at CreatingSnapshots captured a partially written state that caused a filesystem inconsistency.

Fix:

# Inspect the TestFailover status for the specific failure reason
kubectl get testfailover <name> -n dr-production \
  -o jsonpath='{.status}' \
  --kubeconfig ~/.kube/config-quorum | jq .

# If cleanupPolicy is Automatic, check logs before resources are deleted
kubectl logs -n dr-system -l app=site-recovery-quorum-control-plane \
  --kubeconfig ~/.kube/config-quorum | grep -i "testfailover"

# For deeper inspection, re-run with cleanupPolicy: Manual
# to retain test VMs and PVCs for examination

Issue: RPOEvent resources are being created continuously

Symptom: The quorum cluster accumulates a growing number of RPOEvent resources with eventType: ViolationStart and no corresponding ViolationEnd events.

Likely causes:

• Network bandwidth between primary and DR worker nodes (TCP 7000–7999) is saturated, causing DRBD replication lag to consistently exceed rpoObjectiveSeconds.
• The DRBDReplicationPolicy uses Protocol C (synchronous) but the round-trip latency between clusters has exceeded 50ms, stalling writes.
• A drbd-node-agent pod on a worker node has crashed and is not reporting replication state.

Fix:

# Check current replication group health
kubectl get replicationgroupstatus -n dr-production \
  -o jsonpath='{range .items[*]}{.metadata.name}{" health="}{.status.health}{"\n"}{end}' \
  --kubeconfig ~/.kube/config-quorum

# List recent RPO events sorted by timestamp
kubectl get rpoevents -n dr-production \
  --sort-by='.spec.timestamp' \
  --kubeconfig ~/.kube/config-quorum

# Check drbd-node-agent DaemonSet health on the primary cluster
kubectl get daemonset -n dr-system --kubeconfig ~/.kube/config-primary
kubectl get pods -n dr-system -l app=drbd-node-agent \
  --kubeconfig ~/.kube/config-primary

# Review individual agent logs for connection errors
kubectl logs -n dr-system -l app=drbd-node-agent \
  --kubeconfig ~/.kube/config-primary | grep -i "error\|disconnect"

If the replication protocol is Protocol C and latency has risen above 50ms, consider switching to Protocol A in the DRBDReplicationPolicy (replicationMode: asynchronous, drbdProtocol: A) and accepting a seconds-scale RPO until the network issue is resolved.

Issue: DRBDResource stuck with role: Secondary on both sides

Symptom: After a failover or protection operation, both the primary-side and DR-side DRBDResource CRs show role: Secondary, and VMs will not start.

Likely cause: A split-brain condition or an interrupted failover left both DRBD resource sides in Secondary role. The failover-controller's promote step did not complete.

Fix:

# Check the FailoverRequest status for the last operation
kubectl get failoverrequest -n dr-production \
  --kubeconfig ~/.kube/config-quorum

# Check drbd-node-agent logs for promotion errors
kubectl logs -n dr-system -l app=drbd-node-agent \
  --kubeconfig ~/.kube/config-dr | grep -i "promote\|primary\|split-brain"

# If the intended primary side is known and data is consistent,
# delete the stale FailoverRequest and create a new one.
# The failover-controller will re-run the promotion sequence.
kubectl delete failoverrequest <stale-name> -n dr-production \
  --kubeconfig ~/.kube/config-quorum
# Then re-apply your FailoverRequest manifest

If the issue persists after re-running the FailoverRequest, collect a tsr-gather bundle and contact Trilio support before manually modifying DRBDResource role fields.