Custom Resource Schemas
CRD structure and validation rules for site recovery resources
This page documents the complete schema for every Custom Resource Definition (CRD) in the siterecovery.trilio.io/v1alpha1 API group. These CRDs are the primary interface for all Trilio Site Recovery operations — protecting VMs, triggering failovers, monitoring replication health, and running DR validation tests are all driven by creating and observing these resources. Understanding the field shapes, enum values, validation rules, and status conditions for each CRD is essential for building automation, writing GitOps manifests, and diagnosing operational issues.
Before working with these resources, you need:
- An OpenShift 4.17 or later cluster designated as the TSR-Manager cluster, with the Trilio Site Recovery operator installed via OLM
- Two OpenShift 4.17 or later workload clusters, each with the
site-recovery-workload-control-planestack deployed (this happens automatically once yourProtectionZonereachesAvailable) - A
ProtectionZoneCR in anAvailablephase — the TSR-Manager-side CRDs (ProtectionGroup,ProtectionRequest,FailoverRequest,TestFailover,ReplicationGroupStatus,RPOEvent,DRBDReplicationPolicy) are only usable after the zone is available - The
ocCLI configured with access to the TSR-Manager cluster;ocis required for OpenShift-specific operations - Familiarity with the cluster topology: the TSR-Manager cluster hosts all zone-scoped CRs in a
dr-<name>namespace; the workload clusters hostDRBDResourceCRs in the protected VM's namespace under thetrilio-site-recovery-systemworkload stack
API group and version: All CRDs are in the siterecovery.trilio.io group, version v1alpha1. All resources are Namespaced in scope.
The CRDs are installed automatically — you do not apply CRD manifests by hand.
TSR-Manager-side CRDs (ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, ReplicationGroupStatus, RPOEvent, DRBDReplicationPolicy) are installed by the OLM operator bundle and by the embedded control-plane Helm charts. The bundle installs ProtectionZone; the site-recovery-manager-control-plane chart installs the remaining seven once the ProtectionZone controller deploys it.
Workload-side CRDs (DRBDResource) are installed by the site-recovery-workload-control-plane chart on each workload cluster.
You can verify CRD registration after the operator is installed and a ProtectionZone is Available:
# On the TSR-Manager cluster
oc get crd | grep siterecovery.trilio.io
You should see all eight TSR-Manager-side CRDs:
drbdreplicationpolicies.siterecovery.trilio.io
failoverrequests.siterecovery.trilio.io
hooks.siterecovery.trilio.io
protectiongroups.siterecovery.trilio.io
protectionrequests.siterecovery.trilio.io
protectionzones.siterecovery.trilio.io
replicationgroupstatuses.siterecovery.trilio.io
rpoevents.siterecovery.trilio.io
testfailovers.siterecovery.trilio.io
# On each workload cluster
oc get crd drbdresources.siterecovery.trilio.io
Important: Running
oc get drbdresourceagainst the TSR-Manager cluster returnsserver doesn't have a resource type—DRBDResourceis registered only on the workload clusters, never on the TSR-Manager cluster.
This section describes each CRD's spec fields, validation rules, and status shape. Fields marked (required) must be present; all others are optional and carry the defaults shown.
ProtectionZone
Short name: pz | Namespace: dr-<name> on TSR-Manager | Cluster: TSR-Manager only
A ProtectionZone is the top-level object that groups the two workload clusters into a DR deployment. Creating one triggers the protection-zone-controller to deploy the manager and workload control-plane stacks. Every other zone-scoped resource lives in the same dr-<name> namespace.
spec fields
| Field | Type | Default | Valid values | Description |
|---|---|---|---|---|
displayName | string | (required) | 5–253 characters | Human-readable name shown in the TSR web console. metadata.name remains the authoritative identifier. |
storageBackendMode | string | (required) | DRBD | Storage backend for replication. The schema also admits Linstor but the admission webhook rejects it: Linstor backend is not supported yet; use DRBD. You must set DRBD. |
description | string | — | ≤1024 characters | Free-form description of the zone's purpose. |
clusters | array | (required) | exactly 2 items | The two workload clusters. Each entry requires name, clusterID, and kubeconfigSecretRef. |
clusters[].name | string | (required) | 1–63 characters | Operator-facing identifier for this cluster within the zone. Must be unique across the two entries. |
clusters[].clusterID | string | (required) | cluster1 | cluster2 | Stable slot identifier used by all downstream CRs (sourceCluster, targetCluster). The two clusters must occupy different slots. |
clusters[].kubeconfigSecretRef.name | string | (required) | — | Name of the Secret holding the kubeconfig for this cluster. The Secret is resolved in the zone's namespace. |
clusters[].kubeconfigSecretRef.key | string | (required) | — | Key within the Secret whose value is the kubeconfig. |
clusters[].distributionName | string | RHEL-CoreOS-9 | RHEL-CoreOS-9 | RHEL-CoreOS-10 | RHCOS family for this cluster's nodes. Selects the DRBD kernel-module loader image. RHEL-CoreOS-10 is not validated in Tech Preview. Mutable — change it when an OpenShift upgrade moves the node OS. |
logging.level | string | (required when logging is set) | error | info | debug | trace | Runtime log level applied to all zone components. Propagates via the site-recovery-logging ConfigMap within ~15 seconds, with no pod restarts. |
logging.controlPlaneLevel | string | — | error | info | debug | trace | Overrides level for the manager control plane and workload control planes only. |
logging.nodeAgentLevel | string | — | error | info | debug | trace | Overrides level for the DRBD node agents only. Note: there is no warn level; the logr model uses info and error. |
hooks.jobTTLSeconds | integer | 3600 | ≥60 | How long a succeeded hook Job is retained after it finishes. |
hooks.failedJobRetentionSeconds | integer | 604800 | ≥60 | How long a failed or timed-out hook Job is retained for log inspection. |
status fields
| Field | Description |
|---|---|
phase | Zone lifecycle: Provisioning → Available → Degraded | Unavailable | Deleting |
clusters[].status | Per-cluster reachability: Reachable | Unreachable | Degraded |
clusters[].platform | Detected platform: OpenShift | Kubernetes |
clusters[].admittedSCC.nodeAgent | SCC the DRBD node-agent pod was admitted under (expected: privileged) |
clusters[].admittedSCC.controlPlane | SCC the workload control-plane pod was admitted under (expected: restricted-v2) |
clusters[].credentialExpiresAt | When the zone's kubeconfig credential expires. The zone warns 30 days before this time. |
conditions | Standard Kubernetes conditions array. |
kubectl get columns: Display Name, Backend, Clusters, Status, Age
ProtectionGroup
Short name: pg | Namespace: dr-<name> on TSR-Manager
A ProtectionGroup is a logical bundle of KubeVirt VMs that fail over together as one atomic unit. You create one per set of VMs that must maintain consistency across a failover.
spec fields
| Field | Type | Default | Valid values | Description |
|---|---|---|---|---|
virtualMachines | array | (required) | ≥1 item | The VMs to protect. Each entry requires at least name; namespace is optional (defaults to the CR's namespace). |
virtualMachines[].name | string | (required) | DNS-1123, 1–63 chars | The metadata.name of the KubeVirt VirtualMachine. |
virtualMachines[].namespace | string | — | — | The VM's namespace. Omit to use the ProtectionGroup's namespace. |
virtualMachines[].bootOrder | integer | — | ≥0 | Controls boot sequencing during failover. Lower values boot first; equal values boot in parallel; absent values form a final default batch. |
desiredState | string | running | running | stopped | Whether the VMs should be running or stopped on the active cluster. |
sourceCluster | string | — | cluster1 | cluster2 | The protect-time home cluster. Set by the operator during initial protection. |
sla.rtoTargetSeconds | integer | (required when sla is set) | ≥1 | Target recovery time objective in seconds. Recorded for compliance reporting. |
sla.rpoTargetSeconds | integer | 0 | ≥0 | Target recovery point objective in seconds. Feeds the replication monitor's objective; 0 means the Protocol C synchronous guarantee. |
hooks | array | — | ≤32 items | Lifecycle hook bindings. See the Hook section for the definition object. |
hooks[].name | string | (required) | 1–40 chars, DNS-label | Binding name, embedded in Job names. |
hooks[].hookName | string | (required) | 1–40 chars, DNS-label | References a Hook CR in the same namespace. |
hooks[].type | string | (required) | pre-failover | post-failover | pre-failback | post-failback | pre-test | post-test | test-verification | Lifecycle point this binding attaches to. |
hooks[].runOn | string | target | source | target | Which workload cluster runs the Job. source is only admitted on pre-failover and pre-failback bindings. |
hooks[].onFailure | string | Abort | Abort | Continue | What happens when the hook fails after retries. Not applicable to test-verification bindings. |
hooks[].order | integer | 0 | — | Execution order among hooks of the same type; lower runs first. |
hooks[].runOnUnplannedFailover | boolean | false | — | Opts a source-track pre-failover binding into best-effort execution on an unplanned failover. Only meaningful for runOn: source bindings. |
vmIdentity.macAddresses | string | Preserve | Preserve | Clear | How interface MAC addresses are handled on the standby DR copy. Preserve copies MACs verbatim; Clear lets the DR cluster's KubeMacPool assign fresh ones. |
status fields
| Field | Description |
|---|---|
state | Overall PG state: Protected | Syncing | Degraded | Failed |
activeClusterID | Which cluster (cluster1 | cluster2) currently runs the VMs. Empty means undetermined. |
protectedVMs[].replicationStatus | Per-VM replication summary: Protected | Syncing | Degraded | NotProtected |
lastTestFailover | Durable record of the most recent completed TestFailover for this group. Survives deletion of the TestFailover object. |
conditions | Standard Kubernetes conditions array. |
kubectl get columns: VMs, State, Active, Age
ProtectionRequest
Short name: pr, protect | Namespace: dr-<name> on TSR-Manager
A ProtectionRequest requests block-level DRBD replication protection for a single KubeVirt VM. One ProtectionRequest is created per VM. The protection-controller validates the VM, provisions the DRBDResource pair (one per workload cluster), runs initial synchronization, and switches the VM to DRBD-backed storage.
Important operational constraints:
- The VM must be running when you create the
ProtectionRequest.- Once protected, the VM is pinned to one node per cluster for the lifetime of the protection.
nodeNameis immutable once set. Live migration and cold migration of a protected VM are denied.- Re-protection always requires a full unprotect followed by a full initial resynchronization.
spec fields
| Field | Type | Default | Valid values | Description |
|---|---|---|---|---|
sourceCluster | string | (required) | cluster1 | cluster2 | The cluster where the VM currently runs. |
virtualMachine.name | string | (required) | DNS-1123, 1–63 chars | The VM to protect. |
virtualMachine.namespace | string | — | — | The VM's namespace. Defaults to the CR's namespace. |
replicationConfig.replicationProtocol | string | C | C | DRBD wire protocol. Only Protocol C (synchronous, RPO=0) is supported in Tech Preview. The webhook rejects A: ReplicationProtocolA is not supported yet. |
replicationConfig.rpo | integer | 0 | 0 | RPO in minutes. Must be 0 when replicationProtocol is C. A non-zero value is rejected. |
replicationConfig.drbdMetadataConfig.storageClassName | string | — | — | StorageClass for the per-volume DRBD external-metadata PVCs. Must bind immediately; WaitForFirstConsumer classes are rejected during validation. Defaults to the cluster's default StorageClass. |
replicationConfig.drbdMetadataConfig.size | quantity | 128Mi | ≥128Mi | Size of each DRBD metadata PVC. 128Mi accommodates backing volumes up to 4Ti. |
standbyPlacement | object | — | — | Per-key edits to the DR copy's node-level scheduling rules. Applied to the DR cluster only; the source VM is never modified. Immutable once set. |
standbyPlacement.nodeSelector.add | object | — | — | Adds or overrides nodeSelector entries on the DR VM copy. |
standbyPlacement.nodeSelector.remove | array | — | — | Removes these keys from the DR VM's nodeSelector. |
standbyPlacement.nodeAffinity.dropAll | boolean | — | — | Drops the required node affinity entirely on the DR copy. |
standbyPlacement.nodeAffinity.removeKeys | array | — | — | Drops required affinity expressions with these keys. |
standbyPlacement.tolerations.add | array | — | — | Appends tolerations to the DR VM copy. |
standbyPlacement.tolerations.removeKeys | array | — | — | Removes tolerations with these keys from the DR copy. |
status phases
The status.phase field progresses through these exact values:
| Phase | Meaning |
|---|---|
Pending | Request is queued. |
Validating | VM and configuration are being validated. |
CreatingDRBD | DRBDResource CRs are being provisioned on both workload clusters. |
Syncing | Initial block-level synchronization is in progress. |
ReadyToActivate | Initial sync is complete. The VM is ready to switch to DRBD-backed storage. |
Activating | The VM is being switched to the DRBD frontend PVC. |
Protected | The VM is fully protected and replicating. |
Unprotecting | Teardown is in progress (triggered by deleting the CR). |
Failed | Protection failed. Check status.conditions for the reason. |
key status fields
| Field | Description |
|---|---|
status.phase | Current lifecycle phase (see table above). |
status.syncProgress | Sync completion percentage (0–100) during Syncing. |
status.slowestVolumeSyncProgress | Detailed progress (percentage, bytes remaining, ETA) for the slowest volume. |
status.drbdResourceRef.name | Name of the auto-created DRBDResource CR (pattern: drbd-<namespace>-<vm>-<hash8>). |
status.drbdResourceRef.port | TCP port allocated for DRBD replication (7000–8000 inclusive). |
status.activeCluster | Which cluster currently holds the Primary DRBD role. |
status.consistencySummary | Human-readable summary of volume consistency (e.g. 3/3 volumes consistent). |
status.schedulingProbe | Outcome of the DR scheduling probe run at protect time. |
status.conditions | Ready=False with reason: RoleConflict indicates a DRBD role conflict. |
kubectl get columns: VM, Namespace, Phase, Sync, Age
FailoverRequest
Short name: fr | Namespace: dr-<name> on TSR-Manager
A FailoverRequest initiates a planned or unplanned failover of a ProtectionGroup from its active cluster to a target cluster. Every failover — including failback — is initiated by creating this resource. The failover-controller watches FailoverRequest CRs and orchestrates the operation.
Spec is immutable. All fields are fixed at creation time. To retry a failed failover, delete the
FailoverRequestand create a new one.
spec fields
| Field | Type | Default | Valid values | Description |
|---|---|---|---|---|
protectionGroupRef.name | string | (required) | — | Name of the ProtectionGroup to fail over. |
protectionGroupRef.namespace | string | — | — | Namespace of the ProtectionGroup. Defaults to the CR's namespace. |
targetCluster | string | (required) | cluster1 | cluster2 | The cluster to fail over to. |
failoverType | string | planned | planned | unplanned | planned: graceful shutdown of source VMs, waits for DRBD sync, then promotes on target. unplanned: force-promotes on target without waiting for the source, for use when the primary cluster is unavailable. |
batchBootTimeoutSeconds | integer | 300 | 30–3600 | Maximum seconds to wait for one boot-order batch to reach Running on the target before proceeding with a warning. Only meaningful when ProtectionGroup.spec.virtualMachines[].bootOrder is set. |
skipHooks | boolean | false | — | Records all planned hooks as Skipped instead of running them. Use during an incident to bypass a broken or slow hook without losing the audit record. |
Note on
drainTimeoutSeconds: This field is accepted and range-checked by the admission webhook (30–3600, default 300) but is not consumed by the failover controller in Tech Preview. It has no effect on drain or failover behavior and should not be relied upon.
status fields
| Field | Description |
|---|---|
status.state | Overall lifecycle: Pending | InProgress | Completed | Failed | Cancelled |
status.phase | Granular phase: Initializing → Validating → RunningPreHooks → StoppingSource → WaitingForLocks → RemovingTaints → StartingTarget → Monitoring → RunningPostHooks → Completed | Failed |
status.sourceCluster | The cluster the VMs were on when the failover began. |
status.vmStatuses | Per-VM status. Each entry has name, namespace, and phase (Pending | Stopping | Stopped | Starting | Running | Failed). Per-VM detail is here — there is no status.failedVMs field and no status.message field. |
status.startTime / status.completionTime | Timestamps for the operation. |
status.hookStatuses | Per-hook execution records snapshotted at validation time. The plan is immutable even if the ProtectionGroup is edited mid-flight. |
status.conditions | Standard Kubernetes conditions. |
kubectl get columns: Protection Group, Target, State, Phase, Age
TestFailover
Short name: tfo | Namespace: dr-<name> on TSR-Manager
A TestFailover triggers a non-disruptive DR validation. The test-failover-controller creates volume snapshots, provisions test PVCs and VMs in an isolated namespace on the target cluster, runs verification checks, and cleans up. Production VMs and replication are not interrupted.
Test namespace: The controller generates the test namespace as
site-recovery-<protection-group>-<random>and records it instatus.testNamespace. You do not choose or construct this name.
spec fields
| Field | Type | Default | Valid values | Description |
|---|---|---|---|---|
protectionGroupRef.name | string | (required) | — | The ProtectionGroup to test. |
cleanupPolicy | string | Automatic | Automatic | Manual | Automatic: test resources are removed after retentionTime following completionTime. Manual: resources are kept until you set spec.cleanupRequested: true or delete the CR. |
retentionTime | string | 2h | pattern ^[0-9]+(s|m|h|d)$ | How long test resources are kept before automatic cleanup. |
cleanupRequested | boolean | — | — | When set to true, triggers cleanup of test workload resources while preserving the TestFailover CR as a record. Can be set but never unset. |
timeout | string | — | pattern ^[0-9]+(s|m|h)$ | Maximum duration before the test is marked Failed. No default is applied — omit this unless you need to enforce a wall-clock SLA, since large protection groups can take significant time for snapshots and clones. |
schedulingCheck | string | Fail | Fail | Warn | How to handle a VM the DR scheduler cannot place. Fail: exclude the VM and mark the test Failed. Warn: record the verdict and drill everything anyway. |
verification.reachability | array | — | — | Optional network probes to run against test VMs after the automatic boot/agent/filesystem checks pass. |
verification.reachability[].type | string | (required) | tcp | http | Probe protocol. |
verification.reachability[].port | integer | (required) | 1–65535 | Port to probe. |
verification.reachability[].vmName | string | — | — | Restrict probe to this protected VM (source name from the ProtectionGroup). Empty means probe all test VMs. |
verification.reachability[].endpoint | string | — | — | HTTP path. http only; defaults to /. |
verification.reachability[].expectedStatus | integer | — | 100–599 | Expected HTTP status code. Defaults to 200. |
verification.reachability[].timeout | string | 30s | pattern ^[0-9]+(s|m|h)$ | Probe timeout. |
skipHooks | boolean | false | — | Records all planned hooks (including test-verification hooks) as Skipped. |
batchBootTimeoutSeconds | integer | 300 | 30–3600 | Per boot-order batch timeout for test VMs. |
status phases
The status.phase field progresses through these exact values:
| Phase | Meaning |
|---|---|
Queued | Test is waiting to start. |
PreparingEnvironment | Isolated test namespace and NetworkPolicy fence are being created on the target cluster. |
CreatingSnapshots | Volume snapshots of the protected PVCs are being created. |
CreatingVolumes | Test PVCs are being provisioned from snapshots. |
CreatingVMs | Test VM objects are being created in the isolated namespace. |
StartingVMs | Test VMs are being started. |
Verifying | Automatic checks (boot, guest agent, filesystem) and user-configured reachability probes are running. |
Succeeded | Test passed all checks. |
Failed | Test failed. See status.verificationResults and status.schedulingCheck. |
CleaningUp | Test workload resources are being removed. |
Cleaned | All test resources have been removed. The CR remains as a record. |
key status fields
| Field | Description |
|---|---|
status.testNamespace | The generated namespace where test VMs ran (e.g. site-recovery-mygroup-a3f2). Controller-generated; do not construct this value yourself. |
status.targetCluster | Which cluster (cluster1 | cluster2) the test ran on. |
status.result | Terminal result: Succeeded | Failed. |
status.verificationResults.passed | Whether all verification checks passed. |
status.schedulingCheck.verdict | Passed | Failed | Warned | Skipped | Pending | NotEvaluated |
status.schedulingCheck.excludedVMs | VMs that could not be placed, with the scheduler's reason. |
kubectl get columns: Protection Group, Phase, Result, Passed, Scheduling, Age
DRBDReplicationPolicy
Short name: drp, drbdpolicy | Namespace: dr-<name> on TSR-Manager
A DRBDReplicationPolicy defines how volumes replicate between the two workload clusters. You do not create or modify this resource. The ProtectionZone controller creates exactly one default policy named <zoneName>-default with isDefault: true in the zone's namespace when the zone becomes Available. Only the isDefault policy in the ProtectionRequest's namespace is ever consulted.
This schema is documented here for reference when reading oc get drp output or diagnosing replication configuration.
spec fields (read-only reference)
| Field | Default | Description |
|---|---|---|
clusters | (required, 2 items) | The two workload clusters by clusterID (cluster1 | cluster2). Each may carry storageClassMappings (maps source StorageClass to DR StorageClass). |
replicationMode | Synchronous | Synchronous is the only supported mode in Tech Preview. |
drbdProtocol | C | DRBD wire protocol. C (synchronous) is the only supported value. |
isDefault | false | Whether this is the namespace-default policy. The auto-created policy is true. |
networkConfig.connectTimeout | 10s | DRBD connection timeout. |
networkConfig.pingTimeout | 5s | Connection-health ping timeout. |
networkConfig.maxBuffers | 8192 | Maximum network buffers. |
networkConfig.maxEpochSize | 8192 | Maximum write-reordering epoch size. |
networkConfig.sndbufSize | 1M | TCP send buffer size. |
networkConfig.rcvbufSize | 1M | TCP receive buffer size. |
diskConfig.onIoError | detach | Action on lower-level I/O error: detach | pass_on | call-local-io-error. |
diskConfig.fencing | resource-only | DRBD fencing policy: dont-care | resource-only | resource-and-stonith. |
diskConfig.diskFlushes | true | Enable disk flushes for write ordering. |
diskConfig.diskBarrier | true | Enable disk barriers for write ordering. |
resyncConfig.rate | 100M | Maximum resync rate. |
resyncConfig.verify | false | Enable periodic online verification. |
resyncConfig.verifyInterval | 24h | Online-verify interval. |
Note on
networkConfig.portRange: This field is accepted by the schema but is never read by the port allocator. DRBD replication ports are always allocated from the fixed range TCP 7000–8000 inclusive (one port per replicated VM), regardless of whatportRangesays. Port 8000 is allocatable; a firewall rule stopping at 7999 will silently prevent replication on one volume.
kubectl get columns: Clusters, Mode, Protocol, Default, Phase, Age
DRBDResource
Short name: dres | Namespace: VM's namespace on workload clusters only
A DRBDResource represents all the replicated disks of a single VM as one DRBD resource. One CR exists per cluster side (source and target) in the VM's namespace on each workload cluster. The naming pattern is drbd-<namespace>-<vm>-<hash8>. This resource is created and managed by the protection-controller; you do not author it.
Cluster placement:
DRBDResourceCRs exist only on workload clusters. Runningoc get dresagainst the TSR-Manager cluster returnsserver doesn't have a resource type. The labelsiterecovery.trilio.io/protected-vm=trueis stamped on the KubeVirtVirtualMachineobject, not on theDRBDResource.
key spec fields (read-only reference)
| Field | Description |
|---|---|
spec.side | Source | Target — protection direction of this copy. Immutable; does not change on failover. |
spec.role | Desired DRBD role: Primary | Secondary. Changed only by the protection-controller during role flips. |
spec.nodeName | The node this copy is pinned to. Immutable once set. |
spec.port | TCP port for DRBD replication (7000–8000). One port per VM. |
spec.volumes | Per-disk configuration. Up to 15 volumes per resource. Each entry has volumeNumber (stable cross-copy join key), pvcName (this cluster's local backing PVC), and metadata (external-metadata PVC config). |
spec.paused | Suspends agent convergence. Maintenance knob. |
spec.replicationIntent | Enabled | Suspended. Suspended renders the resource without a connection mesh; the kernel sits StandAlone. Written only by the protection-controller. |
spec.replication.protocol | A | C. Materialized from the DRBDReplicationPolicy; C is the only supported value. |
key status fields
| Field | Description |
|---|---|
status.phase | Pending | Provisioning | ReadyToActivate | Syncing | Consistent | Degraded | Failed | Deleting |
status.connectionState | Resource-scoped connection: Connected | Connecting | Disconnected | StandAlone | Unknown |
status.role | Observed DRBD role: Primary | Secondary | Unknown |
status.volumes[].replicationState | Per-volume replication state: Established | SyncSource | SyncTarget | PausedSyncS | PausedSyncT | Off | Unknown |
status.volumes[].syncProgress | Per-volume sync progress (percentage, bytes remaining, ETA). |
status.volumes[].metadata.pvcName | External-metadata PVC name (pattern: drbd-md-<resourceName>-v<volumeNumber>). |
status.conditions | Includes ConditionPlacementReady, which gates metadata-PVC provisioning. |
kubectl get columns: Side, Role, Node, State, Phase, Age
ReplicationGroupStatus
Short name: rgs | Namespace: dr-<name> on TSR-Manager
A ReplicationGroupStatus aggregates replication health for a ProtectionGroup into a single resource. The replication-monitor reconciler auto-creates exactly one per ProtectionGroup, named <protection-group-name>-replication. You do not create this resource. Use the label siterecovery.trilio.io/protection-group: <group-name> to filter them — field selectors on spec.* or status.* do not work for custom resources.
spec fields (read-only reference)
| Field | Default | Description |
|---|---|---|
protectionGroupRef.name | (required) | The monitored ProtectionGroup. |
rpoObjectiveSeconds | 0 | Target RPO in seconds. Fed from ProtectionGroup.spec.sla.rpoTargetSeconds. Drives status.rpoStatus. 0 means Protocol C synchronous guarantee. |
pollingIntervalSeconds | 15 | How often the monitor polls DRBD metrics (5–300 seconds). |
replicationProtocol | C | The DRBD protocol in use. |
key status fields
| Field | Description |
|---|---|
status.rpoStatus | Met | Degraded | Violated | Unknown — current RPO compliance. Driven by spec.rpoObjectiveSeconds. |
status.currentRPOSeconds | Estimated current RPO in seconds based on out-of-sync bytes and throughput. |
status.aggregated.outOfSyncBytes | Total out-of-sync bytes across all VMs in the group. |
status.aggregated.totalVolumeCount | Number of replicated DRBD resources (one per VM, not one per disk). |
status.aggregated.syncedVolumeCount | Number of VMs fully in sync (all disks). |
status.aggregated.connectionState | Worst-case connection state: Connected | Degraded | Disconnected |
status.volumeReplicationStatuses | One entry per VM (per DRBD resource). Each entry covers the entire VM's disks, not a single disk. |
status.lastFullySyncedAt | When every VM in the group was last fully in sync. |
kubectl get columns: Protection-Group, RPO-Status, Out-Of-Sync, Synced, Connection, Last-Sync, Age
RPOEvent
Short name: rpoe | Namespace: dr-<name> on TSR-Manager
An RPOEvent is an immutable audit record of a single replication event — a connection loss, an RPO violation, a resync start or completion, or a sync-state transition. The replication-monitor creates these automatically. All data is in spec (there is no status subresource). Filter by the label siterecovery.trilio.io/protection-group: <group-name>.
spec fields (read-only reference)
| Field | Required | Description |
|---|---|---|
protectionGroupRef.name | Yes | The ProtectionGroup this event pertains to. |
eventType | Yes | SyncCompleted | RPODegraded | RPOViolated | ConnectionLost | ConnectionRestored | ResyncStarted | ResyncCompleted | VolumeAdded | VolumeRemoved | MetadataUndersized | ResizePending | ResizeComplete |
timestamp | Yes | When the event occurred. |
rpoAtEvent | — | Estimated RPO in seconds at event time. |
outOfSyncBytesAtEvent | — | Total out-of-sync bytes at event time. |
previousRPOStatus / newRPOStatus | — | RPO compliance before and after this event: Met | Degraded | Violated | Unknown |
violationReason | — | ConnectionLost | NetworkTimeout | SyncLagExceeded | ProtocolCOutOfSync | ResyncInProgress | Unknown |
affectedVolumes | — | Per-VM replication state snapshot at event time. |
recoveryEstimate | — | Estimated seconds to full resync and the assumed resync rate. |
message | — | Human-readable event description. |
kubectl get columns: Protection-Group, Event-Type, RPO-Seconds, Out-Of-Sync, Reason, Timestamp, Age
Hook
Short name: none | Namespace: dr-<name> on TSR-Manager
A Hook is a reusable, lifecycle-agnostic definition of a customer Job that runs at fixed points of the failover, failback, or DR-test lifecycle. It has no status and no controller. ProtectionGroup.spec.hooks bindings reference Hook objects by name. The Hook admission webhook denies spec edits while an operation is actively using the hook, and denies deletion while it is referenced.
spec fields
| Field | Type | Default | Description |
|---|---|---|---|
jobTemplate.image | string | (required) | Container image for the hook Job. Must be pullable from the executing workload cluster. Pin to a release tag or digest; do not use floating tags. |
jobTemplate.command | array | — | Overrides the image entrypoint. |
jobTemplate.args | array | — | Arguments passed to the command. |
jobTemplate.env | array | — | Environment variables appended after the controller-injected context. fieldRef and resourceFieldRef sources are rejected. |
jobTemplate.envFrom | array | — | configMapRef or secretRef sources. Referenced objects must pre-exist on the executing cluster. |
jobTemplate.volumes | array | — | Volumes for the hook container. Admitted sources: secret, configMap, emptyDir, projected. hostPath and other node-reaching sources are not representable. |
jobTemplate.volumeMounts | array | — | Mounts for the declared volumes. |
jobTemplate.resources | object | — | CPU and memory requests and limits for the hook container. |
jobTemplate.imagePullPolicy | string | — | Always | IfNotPresent | Never. |
timeoutSeconds | integer | 300 | Per-attempt time budget in seconds (1–3600). |
retries | integer | 0 | Number of retry attempts after an initial failure (0–5). |
retryDelaySeconds | integer | 5 | Pause between retry attempts in seconds (0–300). |
description | string | — | Human-readable summary shown in the TSR web console. |
kubectl get columns: Image, Timeout, Retries, Age
This section shows the most common patterns for creating and observing the key CRDs. All commands target the TSR-Manager cluster unless stated otherwise.
Querying zone and cluster status
Check your ProtectionZone is Available before creating any zone-scoped resources:
oc get pz -n dr-myzone
Inspect cluster reachability:
oc get pz myzone -n dr-myzone -o jsonpath='{.status.clusters[*].status}'
Creating a ProtectionGroup
Create a ProtectionGroup that bundles two VMs with boot ordering:
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
name: web-tier
namespace: dr-myzone
spec:
sourceCluster: cluster1
virtualMachines:
- name: web-db
namespace: production
bootOrder: 1
- name: web-app
namespace: production
bootOrder: 2
desiredState: running
sla:
rtoTargetSeconds: 300
rpoTargetSeconds: 0
oc apply -f protection-group.yaml -n dr-myzone
oc get pg -n dr-myzone
Protecting a VM
Create a ProtectionRequest for each VM in the group. The VM must be running:
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
name: protect-web-app
namespace: dr-myzone
spec:
sourceCluster: cluster1
virtualMachine:
name: web-app
namespace: production
replicationConfig:
replicationProtocol: C
rpo: 0
drbdMetadataConfig:
storageClassName: fast-ssd
size: 128Mi
oc apply -f protection-request.yaml -n dr-myzone
# Watch the phase progress
oc get pr -n dr-myzone -w
A VM is fully protected when status.phase reaches Protected. While in Syncing, check progress:
oc get pr protect-web-app -n dr-myzone \
-o jsonpath='{.status.slowestVolumeSyncProgress}'
To find protected VMs on a workload cluster:
# Run against the workload cluster
oc get vm -A -l siterecovery.trilio.io/protected-vm=true
Monitoring replication health
The replication monitor auto-creates a ReplicationGroupStatus named <pg-name>-replication:
oc get rgs -n dr-myzone
oc get rgs web-tier-replication -n dr-myzone -o yaml
Check the RPO compliance status and out-of-sync data:
oc get rgs web-tier-replication -n dr-myzone \
-o jsonpath='{.status.rpoStatus} {.status.aggregated.outOfSyncBytes}'
List recent RPO events for a protection group:
oc get rpoe -n dr-myzone \
-l siterecovery.trilio.io/protection-group=web-tier \
--sort-by=.spec.timestamp
Check per-VM replication detail on a workload cluster:
# Run against the workload cluster; DRBDResource lives in the VM's namespace
oc get dres -n production
Running a test failover
Create a TestFailover to validate DR without disrupting production:
apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
name: web-tier-drill-1
namespace: dr-myzone
spec:
protectionGroupRef:
name: web-tier
cleanupPolicy: Automatic
retentionTime: 2h
schedulingCheck: Fail
verification:
reachability:
- type: http
vmName: web-app
port: 8080
endpoint: /healthz
expectedStatus: 200
timeout: 30s
oc apply -f test-failover.yaml -n dr-myzone
oc get tfo -n dr-myzone -w
After the test completes, retrieve the test namespace and results:
oc get tfo web-tier-drill-1 -n dr-myzone \
-o jsonpath='{.status.testNamespace} {.status.result} {.status.schedulingCheck.verdict}'
To keep the test resources for manual inspection, set cleanupPolicy: Manual and trigger cleanup when ready:
oc patch tfo web-tier-drill-1 -n dr-myzone \
--type merge -p '{"spec":{"cleanupRequested":true}}'
Executing a planned failover
Create a FailoverRequest to move the protection group to cluster2:
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
name: planned-fo-1
namespace: dr-myzone
spec:
protectionGroupRef:
name: web-tier
targetCluster: cluster2
failoverType: planned
batchBootTimeoutSeconds: 300
oc apply -f failover-request.yaml -n dr-myzone
oc get fr -n dr-myzone -w
Watch per-VM progress:
oc get fr planned-fo-1 -n dr-myzone \
-o jsonpath='{.status.vmStatuses}'
Executing an unplanned failover
When the primary cluster is unavailable, create a FailoverRequest with failoverType: unplanned. DRBD volumes are force-promoted on the target without waiting for the primary to shut down:
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
name: emergency-fo-1
namespace: dr-myzone
spec:
protectionGroupRef:
name: web-tier
targetCluster: cluster2
failoverType: unplanned
If replication was Healthy and Consistent at the moment of failure, zero data loss occurs. If replication was Degraded, minimal data loss may occur.
Example 1: Minimal ProtectionZone
The smallest valid ProtectionZone with two workload clusters. The storageBackendMode field is required and must be DRBD.
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
name: zone-east-west
namespace: dr-zone-east-west
spec:
displayName: "East-West DR Zone"
storageBackendMode: DRBD
clusters:
- name: primary
clusterID: cluster1
distributionName: RHEL-CoreOS-9
kubeconfigSecretRef:
name: cluster1-kubeconfig
key: kubeconfig
- name: dr
clusterID: cluster2
distributionName: RHEL-CoreOS-9
kubeconfigSecretRef:
name: cluster2-kubeconfig
key: kubeconfig
Example 2: ProtectionGroup with boot ordering and SLA
Three VMs that must start in order: database first, then cache, then application. The SLA records compliance targets.
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
name: app-stack
namespace: dr-zone-east-west
spec:
sourceCluster: cluster1
virtualMachines:
- name: app-db
namespace: production
bootOrder: 1
- name: app-cache
namespace: production
bootOrder: 2
- name: app-server
namespace: production
bootOrder: 3
desiredState: running
sla:
rtoTargetSeconds: 300
rpoTargetSeconds: 0
vmIdentity:
macAddresses: Preserve
Example 3: ProtectionRequest with custom metadata storage and standby placement
Protect a VM, using a dedicated fast StorageClass for DRBD metadata PVCs and relaxing node affinity on the DR copy.
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
name: protect-app-server
namespace: dr-zone-east-west
spec:
sourceCluster: cluster1
virtualMachine:
name: app-server
namespace: production
replicationConfig:
replicationProtocol: C
rpo: 0
drbdMetadataConfig:
storageClassName: fast-metadata
size: 128Mi
standbyPlacement:
nodeAffinity:
dropAll: true
Example 4: TestFailover with TCP and HTTP probes
Run a DR drill with automatic cleanup after 4 hours, with a TCP probe on port 5432 (database) and an HTTP health check on the application server.
apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
name: app-stack-drill
namespace: dr-zone-east-west
spec:
protectionGroupRef:
name: app-stack
cleanupPolicy: Automatic
retentionTime: 4h
schedulingCheck: Fail
verification:
reachability:
- type: tcp
vmName: app-db
port: 5432
timeout: 30s
- type: http
vmName: app-server
port: 8080
endpoint: /health
expectedStatus: 200
timeout: 30s
Example 5: FailoverRequest — planned failover to DR cluster
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
name: planned-failover-to-dr
namespace: dr-zone-east-west
spec:
protectionGroupRef:
name: app-stack
targetCluster: cluster2
failoverType: planned
batchBootTimeoutSeconds: 300
Example 6: FailoverRequest — unplanned (emergency) failover
For use when the primary cluster is unreachable. Force-promotes on the DR cluster immediately.
apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
name: emergency-failover
namespace: dr-zone-east-west
spec:
protectionGroupRef:
name: app-stack
targetCluster: cluster2
failoverType: unplanned
Example 7: Hook definition with a credential volume
A reusable hook that runs a database quiesce script, mounted from a Secret.
apiVersion: siterecovery.trilio.io/v1alpha1
kind: Hook
metadata:
name: db-quiesce
namespace: dr-zone-east-west
spec:
description: "Quiesce the PostgreSQL database before failover"
timeoutSeconds: 120
retries: 2
retryDelaySeconds: 10
jobTemplate:
image: quay.io/myorg/db-tools:1.2.3
command: ["/scripts/quiesce.sh"]
volumes:
- name: db-creds
secret:
secretName: db-admin-secret
volumeMounts:
- name: db-creds
mountPath: /etc/db-creds
readOnly: true
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 500m
memory: 128Mi
Bind it to a ProtectionGroup as a pre-failover hook:
# In ProtectionGroup.spec.hooks:
hooks:
- name: quiesce-db
hookName: db-quiesce
type: pre-failover
runOn: source
onFailure: Abort
order: 1
Example 8: Reading DRBDResource status on a workload cluster
Query the DRBDResource for a VM named app-db in the production namespace. Run this against the workload cluster, not the TSR-Manager cluster.
# Switch context to the workload cluster
oc get dres -n production
oc get dres drbd-production-app-db-a1b2c3d4 -n production -o yaml
Check per-volume sync state:
oc get dres drbd-production-app-db-a1b2c3d4 -n production \
-o jsonpath='{.status.volumes[*].replicationState}'
Example 9: ProtectionZone with runtime log level override
Enable debug logging for the DRBD node agents without changing the control-plane log level:
apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
name: zone-east-west
namespace: dr-zone-east-west
spec:
displayName: "East-West DR Zone"
storageBackendMode: DRBD
clusters:
- name: primary
clusterID: cluster1
kubeconfigSecretRef:
name: cluster1-kubeconfig
key: kubeconfig
- name: dr
clusterID: cluster2
kubeconfigSecretRef:
name: cluster2-kubeconfig
key: kubeconfig
logging:
level: info
nodeAgentLevel: debug
The change takes effect within approximately 15 seconds with no pod restarts.
Use the following format for each issue: Symptom, Likely cause, Fix.
Issue 1: ProtectionZone admission denied — Linstor backend is not supported yet; use DRBD
Symptom: oc apply of a ProtectionZone CR returns an admission webhook error containing the message above.
Likely cause: spec.storageBackendMode is set to Linstor.
Fix: Change spec.storageBackendMode to DRBD. This is the only accepted value in Tech Preview.
spec:
storageBackendMode: DRBD
Issue 2: ProtectionRequest stuck in Failed phase with Validated=False
Symptom: The ProtectionRequest reaches the Validating phase and then transitions to Failed. A Validated=False condition and a warning event describe the rejection.
Likely causes:
- One or more of the VM's disks is not
volumeMode: Block. - The VM is not running at the time of protection (the VM must be running).
- The
sourceClusterfield references aclusterIDthat does not match eithercluster1orcluster2.
Fix: Correct the VM configuration or the CR field that the warning event identifies. Delete the failed ProtectionRequest and create a new one—the controller does not retry failed requests.
# Read the event for the exact reason
oc describe protectionrequest <name> -n <namespace> --context=primary-cluster
Issue 3: oc adm drain does not complete on a node hosting a protected VM
Symptom: oc adm drain <node> hangs or reports that eviction of a VM pod was denied.
Likely cause: The node hosts a running protected VM. All KubeVirt migration paths (user-initiated, evacuation, drain) are realised as VirtualMachineInstanceMigration objects, and the TSR admission webhook denies these for protected VMs.
Fix: There is no way to drain a node hosting a running protected VM without first unprotecting the VM. To perform node maintenance:
- For in-place maintenance (reboot, kernel update): proceed with the reboot. DRBD modules reload on boot, the anchor pod returns to the same node, and DRBD reconnects and resynchronises the delta.
- For permanent node replacement: unprotect the VM, perform the node replacement, then protect the VM again (full initial resync required).
Issue 4: TestFailover stuck in CleaningUp or not cleaning up
Symptom: The TestFailover CR has been in CleaningUp for an extended period, or resources remain in the test namespace after the test completed.
Likely cause: cleanupPolicy: Manual was set, or the test failed partway through cleanup.
Fix: Do not patch finalizers. Delete the TestFailover CR through the normal API and the controller will handle cleanup:
oc delete testfailover <name> -n dr-production --context=quorum-cluster
If cleanup is blocked, inspect the test namespace (read from status.testNamespace) for stuck resources:
TEST_NS=$(oc get testfailover <name> -n dr-production \
--context=quorum-cluster \
-o jsonpath='{.status.testNamespace}')
oc get all -n "$TEST_NS" --context=dr-cluster
Issue 5: FailoverRequest transitions to Failed before completing
Symptom: The FailoverRequest transitions from InProgress directly to Failed.
Likely causes:
- The failover exceeded the 5-minute wall-clock budget.
- VMs failed to start on the target cluster within
batchBootTimeoutSeconds. - DRBD volume promotion failed on the target side.
Fix:
- Check per-VM detail in
status.vmStatuses:
oc get failoverrequest <name> -n dr-production \
--context=quorum-cluster \
-o jsonpath='{.status.vmStatuses}'
- Check the failover-controller logs on the quorum cluster:
oc logs -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=quorum-control-plane \
-n dr-production \
--context=quorum-cluster \
-c manager
- Check DRBD resource state on both workload clusters:
oc get drbdresource -n <vm-namespace> --context=primary-cluster
oc get drbdresource -n <vm-namespace> --context=dr-cluster
- If the primary cluster is unreachable and you used
failoverType: planned, re-apply the request withfailoverType: unplannedto force-promote without waiting for the primary.
Issue 6: ReplicationGroupStatus shows Degraded or Critical
Symptom: oc get replicationgroupstatus reports Degraded or Critical health for a Protection Group.
Likely causes:
- Network connectivity between primary and DR worker nodes on TCP ports 7000–8000 has been interrupted.
- The DRBD node agent on a worker node is not running or has not converged (check for the
siterecovery.trilio.io/not-ready:NoScheduletaint on nodes). - A worker node hosting a protected VM was lost.
Fix:
- Check
RPOEventrecords for the affected group:
oc get rpoevent -n <vm-namespace> --context=primary-cluster
- Check node agent DaemonSet pod status on both workload clusters:
oc get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=workload-control-plane \
-n <agent-namespace> --context=primary-cluster
oc get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=workload-control-plane \
-n <agent-namespace> --context=dr-cluster
- Check for the node taint:
oc get nodes --context=primary-cluster \
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.taints}{"\n"}{end}'
- If a node was permanently lost, follow the node-loss recovery runbook at
docs/runbooks/drbdresource-node-loss.md. Remove only the agent finalizer—removing the placement finalizer will permanently leak a PVC/PV pair.
Issue 7: replicationProtocol: A rejected at admission
Symptom: Applying a DRBDReplicationPolicy with drbdProtocol: A results in an admission error: ReplicationProtocolA is not supported yet.
Likely cause: Protocol A (asynchronous replication) is not supported in Tech Preview. The DRBDReplicationPolicy is also created automatically by the ProtectionZone controller—you should not be creating it manually.
Fix: Do not create or modify DRBDReplicationPolicy objects. The ProtectionZone controller creates the default policy (<zoneName>-default) automatically when the zone reaches Available. Only synchronous replication (Protocol C, RPO=0) is supported in Tech Preview.
Issue 8: Collecting a diagnostic bundle
Symptom: You need to provide a support bundle to Trilio support.
Fix: Run tsr-gather via oc adm must-gather against the quorum cluster. This collects logs, CRD state, and configuration from all clusters in the DR estate without capturing secrets or credentials:
oc adm must-gather \
--image=quay.io/triliovault/tsr-gather:latest \
--context=quorum-cluster \
-- /usr/bin/tsr-gather
The resulting bundle in ./must-gather.local.* can be shared with support.