Trilio Site Recovery for Kubernetes
Guide

Custom Resource Schemas

CRD structure and validation rules for site recovery resources


Overview

This page documents the complete schema for every Custom Resource Definition (CRD) in the siterecovery.trilio.io/v1alpha1 API group. These CRDs are the primary interface for all Trilio Site Recovery operations — protecting VMs, triggering failovers, monitoring replication health, and running DR validation tests are all driven by creating and observing these resources. Understanding the field shapes, enum values, validation rules, and status conditions for each CRD is essential for building automation, writing GitOps manifests, and diagnosing operational issues.


Prerequisites

Before working with these resources, you need:

  • An OpenShift 4.17 or later cluster designated as the TSR-Manager cluster, with the Trilio Site Recovery operator installed via OLM
  • Two OpenShift 4.17 or later workload clusters, each with the site-recovery-workload-control-plane stack deployed (this happens automatically once your ProtectionZone reaches Available)
  • A ProtectionZone CR in an Available phase — the TSR-Manager-side CRDs (ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, ReplicationGroupStatus, RPOEvent, DRBDReplicationPolicy) are only usable after the zone is available
  • The oc CLI configured with access to the TSR-Manager cluster; oc is required for OpenShift-specific operations
  • Familiarity with the cluster topology: the TSR-Manager cluster hosts all zone-scoped CRs in a dr-<name> namespace; the workload clusters host DRBDResource CRs in the protected VM's namespace under the trilio-site-recovery-system workload stack

API group and version: All CRDs are in the siterecovery.trilio.io group, version v1alpha1. All resources are Namespaced in scope.


Installation

The CRDs are installed automatically — you do not apply CRD manifests by hand.

TSR-Manager-side CRDs (ProtectionZone, ProtectionGroup, ProtectionRequest, FailoverRequest, TestFailover, ReplicationGroupStatus, RPOEvent, DRBDReplicationPolicy) are installed by the OLM operator bundle and by the embedded control-plane Helm charts. The bundle installs ProtectionZone; the site-recovery-manager-control-plane chart installs the remaining seven once the ProtectionZone controller deploys it.

Workload-side CRDs (DRBDResource) are installed by the site-recovery-workload-control-plane chart on each workload cluster.

You can verify CRD registration after the operator is installed and a ProtectionZone is Available:

# On the TSR-Manager cluster
oc get crd | grep siterecovery.trilio.io

You should see all eight TSR-Manager-side CRDs:

drbdreplicationpolicies.siterecovery.trilio.io
failoverrequests.siterecovery.trilio.io
hooks.siterecovery.trilio.io
protectiongroups.siterecovery.trilio.io
protectionrequests.siterecovery.trilio.io
protectionzones.siterecovery.trilio.io
replicationgroupstatuses.siterecovery.trilio.io
rpoevents.siterecovery.trilio.io
testfailovers.siterecovery.trilio.io
# On each workload cluster
oc get crd drbdresources.siterecovery.trilio.io

Important: Running oc get drbdresource against the TSR-Manager cluster returns server doesn't have a resource type — DRBDResource is registered only on the workload clusters, never on the TSR-Manager cluster.


Configuration

This section describes each CRD's spec fields, validation rules, and status shape. Fields marked (required) must be present; all others are optional and carry the defaults shown.


ProtectionZone

Short name: pz | Namespace: dr-<name> on TSR-Manager | Cluster: TSR-Manager only

A ProtectionZone is the top-level object that groups the two workload clusters into a DR deployment. Creating one triggers the protection-zone-controller to deploy the manager and workload control-plane stacks. Every other zone-scoped resource lives in the same dr-<name> namespace.

spec fields

FieldTypeDefaultValid valuesDescription
displayNamestring(required)5–253 charactersHuman-readable name shown in the TSR web console. metadata.name remains the authoritative identifier.
storageBackendModestring(required)DRBDStorage backend for replication. The schema also admits Linstor but the admission webhook rejects it: Linstor backend is not supported yet; use DRBD. You must set DRBD.
descriptionstring—≤1024 charactersFree-form description of the zone's purpose.
clustersarray(required)exactly 2 itemsThe two workload clusters. Each entry requires name, clusterID, and kubeconfigSecretRef.
clusters[].namestring(required)1–63 charactersOperator-facing identifier for this cluster within the zone. Must be unique across the two entries.
clusters[].clusterIDstring(required)cluster1 | cluster2Stable slot identifier used by all downstream CRs (sourceCluster, targetCluster). The two clusters must occupy different slots.
clusters[].kubeconfigSecretRef.namestring(required)—Name of the Secret holding the kubeconfig for this cluster. The Secret is resolved in the zone's namespace.
clusters[].kubeconfigSecretRef.keystring(required)—Key within the Secret whose value is the kubeconfig.
clusters[].distributionNamestringRHEL-CoreOS-9RHEL-CoreOS-9 | RHEL-CoreOS-10RHCOS family for this cluster's nodes. Selects the DRBD kernel-module loader image. RHEL-CoreOS-10 is not validated in Tech Preview. Mutable — change it when an OpenShift upgrade moves the node OS.
logging.levelstring(required when logging is set)error | info | debug | traceRuntime log level applied to all zone components. Propagates via the site-recovery-logging ConfigMap within ~15 seconds, with no pod restarts.
logging.controlPlaneLevelstring—error | info | debug | traceOverrides level for the manager control plane and workload control planes only.
logging.nodeAgentLevelstring—error | info | debug | traceOverrides level for the DRBD node agents only. Note: there is no warn level; the logr model uses info and error.
hooks.jobTTLSecondsinteger3600≥60How long a succeeded hook Job is retained after it finishes.
hooks.failedJobRetentionSecondsinteger604800≥60How long a failed or timed-out hook Job is retained for log inspection.

status fields

FieldDescription
phaseZone lifecycle: Provisioning → Available → Degraded | Unavailable | Deleting
clusters[].statusPer-cluster reachability: Reachable | Unreachable | Degraded
clusters[].platformDetected platform: OpenShift | Kubernetes
clusters[].admittedSCC.nodeAgentSCC the DRBD node-agent pod was admitted under (expected: privileged)
clusters[].admittedSCC.controlPlaneSCC the workload control-plane pod was admitted under (expected: restricted-v2)
clusters[].credentialExpiresAtWhen the zone's kubeconfig credential expires. The zone warns 30 days before this time.
conditionsStandard Kubernetes conditions array.

kubectl get columns: Display Name, Backend, Clusters, Status, Age


ProtectionGroup

Short name: pg | Namespace: dr-<name> on TSR-Manager

A ProtectionGroup is a logical bundle of KubeVirt VMs that fail over together as one atomic unit. You create one per set of VMs that must maintain consistency across a failover.

spec fields

FieldTypeDefaultValid valuesDescription
virtualMachinesarray(required)≥1 itemThe VMs to protect. Each entry requires at least name; namespace is optional (defaults to the CR's namespace).
virtualMachines[].namestring(required)DNS-1123, 1–63 charsThe metadata.name of the KubeVirt VirtualMachine.
virtualMachines[].namespacestring——The VM's namespace. Omit to use the ProtectionGroup's namespace.
virtualMachines[].bootOrderinteger—≥0Controls boot sequencing during failover. Lower values boot first; equal values boot in parallel; absent values form a final default batch.
desiredStatestringrunningrunning | stoppedWhether the VMs should be running or stopped on the active cluster.
sourceClusterstring—cluster1 | cluster2The protect-time home cluster. Set by the operator during initial protection.
sla.rtoTargetSecondsinteger(required when sla is set)≥1Target recovery time objective in seconds. Recorded for compliance reporting.
sla.rpoTargetSecondsinteger0≥0Target recovery point objective in seconds. Feeds the replication monitor's objective; 0 means the Protocol C synchronous guarantee.
hooksarray—≤32 itemsLifecycle hook bindings. See the Hook section for the definition object.
hooks[].namestring(required)1–40 chars, DNS-labelBinding name, embedded in Job names.
hooks[].hookNamestring(required)1–40 chars, DNS-labelReferences a Hook CR in the same namespace.
hooks[].typestring(required)pre-failover | post-failover | pre-failback | post-failback | pre-test | post-test | test-verificationLifecycle point this binding attaches to.
hooks[].runOnstringtargetsource | targetWhich workload cluster runs the Job. source is only admitted on pre-failover and pre-failback bindings.
hooks[].onFailurestringAbortAbort | ContinueWhat happens when the hook fails after retries. Not applicable to test-verification bindings.
hooks[].orderinteger0—Execution order among hooks of the same type; lower runs first.
hooks[].runOnUnplannedFailoverbooleanfalse—Opts a source-track pre-failover binding into best-effort execution on an unplanned failover. Only meaningful for runOn: source bindings.
vmIdentity.macAddressesstringPreservePreserve | ClearHow interface MAC addresses are handled on the standby DR copy. Preserve copies MACs verbatim; Clear lets the DR cluster's KubeMacPool assign fresh ones.

status fields

FieldDescription
stateOverall PG state: Protected | Syncing | Degraded | Failed
activeClusterIDWhich cluster (cluster1 | cluster2) currently runs the VMs. Empty means undetermined.
protectedVMs[].replicationStatusPer-VM replication summary: Protected | Syncing | Degraded | NotProtected
lastTestFailoverDurable record of the most recent completed TestFailover for this group. Survives deletion of the TestFailover object.
conditionsStandard Kubernetes conditions array.

kubectl get columns: VMs, State, Active, Age


ProtectionRequest

Short name: pr, protect | Namespace: dr-<name> on TSR-Manager

A ProtectionRequest requests block-level DRBD replication protection for a single KubeVirt VM. One ProtectionRequest is created per VM. The protection-controller validates the VM, provisions the DRBDResource pair (one per workload cluster), runs initial synchronization, and switches the VM to DRBD-backed storage.

Important operational constraints:

  • The VM must be running when you create the ProtectionRequest.
  • Once protected, the VM is pinned to one node per cluster for the lifetime of the protection. nodeName is immutable once set. Live migration and cold migration of a protected VM are denied.
  • Re-protection always requires a full unprotect followed by a full initial resynchronization.

spec fields

FieldTypeDefaultValid valuesDescription
sourceClusterstring(required)cluster1 | cluster2The cluster where the VM currently runs.
virtualMachine.namestring(required)DNS-1123, 1–63 charsThe VM to protect.
virtualMachine.namespacestring——The VM's namespace. Defaults to the CR's namespace.
replicationConfig.replicationProtocolstringCCDRBD wire protocol. Only Protocol C (synchronous, RPO=0) is supported in Tech Preview. The webhook rejects A: ReplicationProtocolA is not supported yet.
replicationConfig.rpointeger00RPO in minutes. Must be 0 when replicationProtocol is C. A non-zero value is rejected.
replicationConfig.drbdMetadataConfig.storageClassNamestring——StorageClass for the per-volume DRBD external-metadata PVCs. Must bind immediately; WaitForFirstConsumer classes are rejected during validation. Defaults to the cluster's default StorageClass.
replicationConfig.drbdMetadataConfig.sizequantity128Mi≥128MiSize of each DRBD metadata PVC. 128Mi accommodates backing volumes up to 4Ti.
standbyPlacementobject——Per-key edits to the DR copy's node-level scheduling rules. Applied to the DR cluster only; the source VM is never modified. Immutable once set.
standbyPlacement.nodeSelector.addobject——Adds or overrides nodeSelector entries on the DR VM copy.
standbyPlacement.nodeSelector.removearray——Removes these keys from the DR VM's nodeSelector.
standbyPlacement.nodeAffinity.dropAllboolean——Drops the required node affinity entirely on the DR copy.
standbyPlacement.nodeAffinity.removeKeysarray——Drops required affinity expressions with these keys.
standbyPlacement.tolerations.addarray——Appends tolerations to the DR VM copy.
standbyPlacement.tolerations.removeKeysarray——Removes tolerations with these keys from the DR copy.

status phases

The status.phase field progresses through these exact values:

PhaseMeaning
PendingRequest is queued.
ValidatingVM and configuration are being validated.
CreatingDRBDDRBDResource CRs are being provisioned on both workload clusters.
SyncingInitial block-level synchronization is in progress.
ReadyToActivateInitial sync is complete. The VM is ready to switch to DRBD-backed storage.
ActivatingThe VM is being switched to the DRBD frontend PVC.
ProtectedThe VM is fully protected and replicating.
UnprotectingTeardown is in progress (triggered by deleting the CR).
FailedProtection failed. Check status.conditions for the reason.

key status fields

FieldDescription
status.phaseCurrent lifecycle phase (see table above).
status.syncProgressSync completion percentage (0–100) during Syncing.
status.slowestVolumeSyncProgressDetailed progress (percentage, bytes remaining, ETA) for the slowest volume.
status.drbdResourceRef.nameName of the auto-created DRBDResource CR (pattern: drbd-<namespace>-<vm>-<hash8>).
status.drbdResourceRef.portTCP port allocated for DRBD replication (7000–8000 inclusive).
status.activeClusterWhich cluster currently holds the Primary DRBD role.
status.consistencySummaryHuman-readable summary of volume consistency (e.g. 3/3 volumes consistent).
status.schedulingProbeOutcome of the DR scheduling probe run at protect time.
status.conditionsReady=False with reason: RoleConflict indicates a DRBD role conflict.

kubectl get columns: VM, Namespace, Phase, Sync, Age


FailoverRequest

Short name: fr | Namespace: dr-<name> on TSR-Manager

A FailoverRequest initiates a planned or unplanned failover of a ProtectionGroup from its active cluster to a target cluster. Every failover — including failback — is initiated by creating this resource. The failover-controller watches FailoverRequest CRs and orchestrates the operation.

Spec is immutable. All fields are fixed at creation time. To retry a failed failover, delete the FailoverRequest and create a new one.

spec fields

FieldTypeDefaultValid valuesDescription
protectionGroupRef.namestring(required)—Name of the ProtectionGroup to fail over.
protectionGroupRef.namespacestring——Namespace of the ProtectionGroup. Defaults to the CR's namespace.
targetClusterstring(required)cluster1 | cluster2The cluster to fail over to.
failoverTypestringplannedplanned | unplannedplanned: graceful shutdown of source VMs, waits for DRBD sync, then promotes on target. unplanned: force-promotes on target without waiting for the source, for use when the primary cluster is unavailable.
batchBootTimeoutSecondsinteger30030–3600Maximum seconds to wait for one boot-order batch to reach Running on the target before proceeding with a warning. Only meaningful when ProtectionGroup.spec.virtualMachines[].bootOrder is set.
skipHooksbooleanfalse—Records all planned hooks as Skipped instead of running them. Use during an incident to bypass a broken or slow hook without losing the audit record.

Note on drainTimeoutSeconds: This field is accepted and range-checked by the admission webhook (30–3600, default 300) but is not consumed by the failover controller in Tech Preview. It has no effect on drain or failover behavior and should not be relied upon.

status fields

FieldDescription
status.stateOverall lifecycle: Pending | InProgress | Completed | Failed | Cancelled
status.phaseGranular phase: Initializing → Validating → RunningPreHooks → StoppingSource → WaitingForLocks → RemovingTaints → StartingTarget → Monitoring → RunningPostHooks → Completed | Failed
status.sourceClusterThe cluster the VMs were on when the failover began.
status.vmStatusesPer-VM status. Each entry has name, namespace, and phase (Pending | Stopping | Stopped | Starting | Running | Failed). Per-VM detail is here — there is no status.failedVMs field and no status.message field.
status.startTime / status.completionTimeTimestamps for the operation.
status.hookStatusesPer-hook execution records snapshotted at validation time. The plan is immutable even if the ProtectionGroup is edited mid-flight.
status.conditionsStandard Kubernetes conditions.

kubectl get columns: Protection Group, Target, State, Phase, Age


TestFailover

Short name: tfo | Namespace: dr-<name> on TSR-Manager

A TestFailover triggers a non-disruptive DR validation. The test-failover-controller creates volume snapshots, provisions test PVCs and VMs in an isolated namespace on the target cluster, runs verification checks, and cleans up. Production VMs and replication are not interrupted.

Test namespace: The controller generates the test namespace as site-recovery-<protection-group>-<random> and records it in status.testNamespace. You do not choose or construct this name.

spec fields

FieldTypeDefaultValid valuesDescription
protectionGroupRef.namestring(required)—The ProtectionGroup to test.
cleanupPolicystringAutomaticAutomatic | ManualAutomatic: test resources are removed after retentionTime following completionTime. Manual: resources are kept until you set spec.cleanupRequested: true or delete the CR.
retentionTimestring2hpattern ^[0-9]+(s|m|h|d)$How long test resources are kept before automatic cleanup.
cleanupRequestedboolean——When set to true, triggers cleanup of test workload resources while preserving the TestFailover CR as a record. Can be set but never unset.
timeoutstring—pattern ^[0-9]+(s|m|h)$Maximum duration before the test is marked Failed. No default is applied — omit this unless you need to enforce a wall-clock SLA, since large protection groups can take significant time for snapshots and clones.
schedulingCheckstringFailFail | WarnHow to handle a VM the DR scheduler cannot place. Fail: exclude the VM and mark the test Failed. Warn: record the verdict and drill everything anyway.
verification.reachabilityarray——Optional network probes to run against test VMs after the automatic boot/agent/filesystem checks pass.
verification.reachability[].typestring(required)tcp | httpProbe protocol.
verification.reachability[].portinteger(required)1–65535Port to probe.
verification.reachability[].vmNamestring——Restrict probe to this protected VM (source name from the ProtectionGroup). Empty means probe all test VMs.
verification.reachability[].endpointstring——HTTP path. http only; defaults to /.
verification.reachability[].expectedStatusinteger—100–599Expected HTTP status code. Defaults to 200.
verification.reachability[].timeoutstring30spattern ^[0-9]+(s|m|h)$Probe timeout.
skipHooksbooleanfalse—Records all planned hooks (including test-verification hooks) as Skipped.
batchBootTimeoutSecondsinteger30030–3600Per boot-order batch timeout for test VMs.

status phases

The status.phase field progresses through these exact values:

PhaseMeaning
QueuedTest is waiting to start.
PreparingEnvironmentIsolated test namespace and NetworkPolicy fence are being created on the target cluster.
CreatingSnapshotsVolume snapshots of the protected PVCs are being created.
CreatingVolumesTest PVCs are being provisioned from snapshots.
CreatingVMsTest VM objects are being created in the isolated namespace.
StartingVMsTest VMs are being started.
VerifyingAutomatic checks (boot, guest agent, filesystem) and user-configured reachability probes are running.
SucceededTest passed all checks.
FailedTest failed. See status.verificationResults and status.schedulingCheck.
CleaningUpTest workload resources are being removed.
CleanedAll test resources have been removed. The CR remains as a record.

key status fields

FieldDescription
status.testNamespaceThe generated namespace where test VMs ran (e.g. site-recovery-mygroup-a3f2). Controller-generated; do not construct this value yourself.
status.targetClusterWhich cluster (cluster1 | cluster2) the test ran on.
status.resultTerminal result: Succeeded | Failed.
status.verificationResults.passedWhether all verification checks passed.
status.schedulingCheck.verdictPassed | Failed | Warned | Skipped | Pending | NotEvaluated
status.schedulingCheck.excludedVMsVMs that could not be placed, with the scheduler's reason.

kubectl get columns: Protection Group, Phase, Result, Passed, Scheduling, Age


DRBDReplicationPolicy

Short name: drp, drbdpolicy | Namespace: dr-<name> on TSR-Manager

A DRBDReplicationPolicy defines how volumes replicate between the two workload clusters. You do not create or modify this resource. The ProtectionZone controller creates exactly one default policy named <zoneName>-default with isDefault: true in the zone's namespace when the zone becomes Available. Only the isDefault policy in the ProtectionRequest's namespace is ever consulted.

This schema is documented here for reference when reading oc get drp output or diagnosing replication configuration.

spec fields (read-only reference)

FieldDefaultDescription
clusters(required, 2 items)The two workload clusters by clusterID (cluster1 | cluster2). Each may carry storageClassMappings (maps source StorageClass to DR StorageClass).
replicationModeSynchronousSynchronous is the only supported mode in Tech Preview.
drbdProtocolCDRBD wire protocol. C (synchronous) is the only supported value.
isDefaultfalseWhether this is the namespace-default policy. The auto-created policy is true.
networkConfig.connectTimeout10sDRBD connection timeout.
networkConfig.pingTimeout5sConnection-health ping timeout.
networkConfig.maxBuffers8192Maximum network buffers.
networkConfig.maxEpochSize8192Maximum write-reordering epoch size.
networkConfig.sndbufSize1MTCP send buffer size.
networkConfig.rcvbufSize1MTCP receive buffer size.
diskConfig.onIoErrordetachAction on lower-level I/O error: detach | pass_on | call-local-io-error.
diskConfig.fencingresource-onlyDRBD fencing policy: dont-care | resource-only | resource-and-stonith.
diskConfig.diskFlushestrueEnable disk flushes for write ordering.
diskConfig.diskBarriertrueEnable disk barriers for write ordering.
resyncConfig.rate100MMaximum resync rate.
resyncConfig.verifyfalseEnable periodic online verification.
resyncConfig.verifyInterval24hOnline-verify interval.

Note on networkConfig.portRange: This field is accepted by the schema but is never read by the port allocator. DRBD replication ports are always allocated from the fixed range TCP 7000–8000 inclusive (one port per replicated VM), regardless of what portRange says. Port 8000 is allocatable; a firewall rule stopping at 7999 will silently prevent replication on one volume.

kubectl get columns: Clusters, Mode, Protocol, Default, Phase, Age


DRBDResource

Short name: dres | Namespace: VM's namespace on workload clusters only

A DRBDResource represents all the replicated disks of a single VM as one DRBD resource. One CR exists per cluster side (source and target) in the VM's namespace on each workload cluster. The naming pattern is drbd-<namespace>-<vm>-<hash8>. This resource is created and managed by the protection-controller; you do not author it.

Cluster placement: DRBDResource CRs exist only on workload clusters. Running oc get dres against the TSR-Manager cluster returns server doesn't have a resource type. The label siterecovery.trilio.io/protected-vm=true is stamped on the KubeVirt VirtualMachine object, not on the DRBDResource.

key spec fields (read-only reference)

FieldDescription
spec.sideSource | Target — protection direction of this copy. Immutable; does not change on failover.
spec.roleDesired DRBD role: Primary | Secondary. Changed only by the protection-controller during role flips.
spec.nodeNameThe node this copy is pinned to. Immutable once set.
spec.portTCP port for DRBD replication (7000–8000). One port per VM.
spec.volumesPer-disk configuration. Up to 15 volumes per resource. Each entry has volumeNumber (stable cross-copy join key), pvcName (this cluster's local backing PVC), and metadata (external-metadata PVC config).
spec.pausedSuspends agent convergence. Maintenance knob.
spec.replicationIntentEnabled | Suspended. Suspended renders the resource without a connection mesh; the kernel sits StandAlone. Written only by the protection-controller.
spec.replication.protocolA | C. Materialized from the DRBDReplicationPolicy; C is the only supported value.

key status fields

FieldDescription
status.phasePending | Provisioning | ReadyToActivate | Syncing | Consistent | Degraded | Failed | Deleting
status.connectionStateResource-scoped connection: Connected | Connecting | Disconnected | StandAlone | Unknown
status.roleObserved DRBD role: Primary | Secondary | Unknown
status.volumes[].replicationStatePer-volume replication state: Established | SyncSource | SyncTarget | PausedSyncS | PausedSyncT | Off | Unknown
status.volumes[].syncProgressPer-volume sync progress (percentage, bytes remaining, ETA).
status.volumes[].metadata.pvcNameExternal-metadata PVC name (pattern: drbd-md-<resourceName>-v<volumeNumber>).
status.conditionsIncludes ConditionPlacementReady, which gates metadata-PVC provisioning.

kubectl get columns: Side, Role, Node, State, Phase, Age


ReplicationGroupStatus

Short name: rgs | Namespace: dr-<name> on TSR-Manager

A ReplicationGroupStatus aggregates replication health for a ProtectionGroup into a single resource. The replication-monitor reconciler auto-creates exactly one per ProtectionGroup, named <protection-group-name>-replication. You do not create this resource. Use the label siterecovery.trilio.io/protection-group: <group-name> to filter them — field selectors on spec.* or status.* do not work for custom resources.

spec fields (read-only reference)

FieldDefaultDescription
protectionGroupRef.name(required)The monitored ProtectionGroup.
rpoObjectiveSeconds0Target RPO in seconds. Fed from ProtectionGroup.spec.sla.rpoTargetSeconds. Drives status.rpoStatus. 0 means Protocol C synchronous guarantee.
pollingIntervalSeconds15How often the monitor polls DRBD metrics (5–300 seconds).
replicationProtocolCThe DRBD protocol in use.

key status fields

FieldDescription
status.rpoStatusMet | Degraded | Violated | Unknown — current RPO compliance. Driven by spec.rpoObjectiveSeconds.
status.currentRPOSecondsEstimated current RPO in seconds based on out-of-sync bytes and throughput.
status.aggregated.outOfSyncBytesTotal out-of-sync bytes across all VMs in the group.
status.aggregated.totalVolumeCountNumber of replicated DRBD resources (one per VM, not one per disk).
status.aggregated.syncedVolumeCountNumber of VMs fully in sync (all disks).
status.aggregated.connectionStateWorst-case connection state: Connected | Degraded | Disconnected
status.volumeReplicationStatusesOne entry per VM (per DRBD resource). Each entry covers the entire VM's disks, not a single disk.
status.lastFullySyncedAtWhen every VM in the group was last fully in sync.

kubectl get columns: Protection-Group, RPO-Status, Out-Of-Sync, Synced, Connection, Last-Sync, Age


RPOEvent

Short name: rpoe | Namespace: dr-<name> on TSR-Manager

An RPOEvent is an immutable audit record of a single replication event — a connection loss, an RPO violation, a resync start or completion, or a sync-state transition. The replication-monitor creates these automatically. All data is in spec (there is no status subresource). Filter by the label siterecovery.trilio.io/protection-group: <group-name>.

spec fields (read-only reference)

FieldRequiredDescription
protectionGroupRef.nameYesThe ProtectionGroup this event pertains to.
eventTypeYesSyncCompleted | RPODegraded | RPOViolated | ConnectionLost | ConnectionRestored | ResyncStarted | ResyncCompleted | VolumeAdded | VolumeRemoved | MetadataUndersized | ResizePending | ResizeComplete
timestampYesWhen the event occurred.
rpoAtEvent—Estimated RPO in seconds at event time.
outOfSyncBytesAtEvent—Total out-of-sync bytes at event time.
previousRPOStatus / newRPOStatus—RPO compliance before and after this event: Met | Degraded | Violated | Unknown
violationReason—ConnectionLost | NetworkTimeout | SyncLagExceeded | ProtocolCOutOfSync | ResyncInProgress | Unknown
affectedVolumes—Per-VM replication state snapshot at event time.
recoveryEstimate—Estimated seconds to full resync and the assumed resync rate.
message—Human-readable event description.

kubectl get columns: Protection-Group, Event-Type, RPO-Seconds, Out-Of-Sync, Reason, Timestamp, Age


Hook

Short name: none | Namespace: dr-<name> on TSR-Manager

A Hook is a reusable, lifecycle-agnostic definition of a customer Job that runs at fixed points of the failover, failback, or DR-test lifecycle. It has no status and no controller. ProtectionGroup.spec.hooks bindings reference Hook objects by name. The Hook admission webhook denies spec edits while an operation is actively using the hook, and denies deletion while it is referenced.

spec fields

FieldTypeDefaultDescription
jobTemplate.imagestring(required)Container image for the hook Job. Must be pullable from the executing workload cluster. Pin to a release tag or digest; do not use floating tags.
jobTemplate.commandarray—Overrides the image entrypoint.
jobTemplate.argsarray—Arguments passed to the command.
jobTemplate.envarray—Environment variables appended after the controller-injected context. fieldRef and resourceFieldRef sources are rejected.
jobTemplate.envFromarray—configMapRef or secretRef sources. Referenced objects must pre-exist on the executing cluster.
jobTemplate.volumesarray—Volumes for the hook container. Admitted sources: secret, configMap, emptyDir, projected. hostPath and other node-reaching sources are not representable.
jobTemplate.volumeMountsarray—Mounts for the declared volumes.
jobTemplate.resourcesobject—CPU and memory requests and limits for the hook container.
jobTemplate.imagePullPolicystring—Always | IfNotPresent | Never.
timeoutSecondsinteger300Per-attempt time budget in seconds (1–3600).
retriesinteger0Number of retry attempts after an initial failure (0–5).
retryDelaySecondsinteger5Pause between retry attempts in seconds (0–300).
descriptionstring—Human-readable summary shown in the TSR web console.

kubectl get columns: Image, Timeout, Retries, Age


Usage

This section shows the most common patterns for creating and observing the key CRDs. All commands target the TSR-Manager cluster unless stated otherwise.


Querying zone and cluster status

Check your ProtectionZone is Available before creating any zone-scoped resources:

oc get pz -n dr-myzone

Inspect cluster reachability:

oc get pz myzone -n dr-myzone -o jsonpath='{.status.clusters[*].status}'

Creating a ProtectionGroup

Create a ProtectionGroup that bundles two VMs with boot ordering:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: web-tier
  namespace: dr-myzone
spec:
  sourceCluster: cluster1
  virtualMachines:
    - name: web-db
      namespace: production
      bootOrder: 1
    - name: web-app
      namespace: production
      bootOrder: 2
  desiredState: running
  sla:
    rtoTargetSeconds: 300
    rpoTargetSeconds: 0
oc apply -f protection-group.yaml -n dr-myzone
oc get pg -n dr-myzone

Protecting a VM

Create a ProtectionRequest for each VM in the group. The VM must be running:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-app
  namespace: dr-myzone
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: web-app
    namespace: production
  replicationConfig:
    replicationProtocol: C
    rpo: 0
    drbdMetadataConfig:
      storageClassName: fast-ssd
      size: 128Mi
oc apply -f protection-request.yaml -n dr-myzone
# Watch the phase progress
oc get pr -n dr-myzone -w

A VM is fully protected when status.phase reaches Protected. While in Syncing, check progress:

oc get pr protect-web-app -n dr-myzone \
  -o jsonpath='{.status.slowestVolumeSyncProgress}'

To find protected VMs on a workload cluster:

# Run against the workload cluster
oc get vm -A -l siterecovery.trilio.io/protected-vm=true

Monitoring replication health

The replication monitor auto-creates a ReplicationGroupStatus named <pg-name>-replication:

oc get rgs -n dr-myzone
oc get rgs web-tier-replication -n dr-myzone -o yaml

Check the RPO compliance status and out-of-sync data:

oc get rgs web-tier-replication -n dr-myzone \
  -o jsonpath='{.status.rpoStatus} {.status.aggregated.outOfSyncBytes}'

List recent RPO events for a protection group:

oc get rpoe -n dr-myzone \
  -l siterecovery.trilio.io/protection-group=web-tier \
  --sort-by=.spec.timestamp

Check per-VM replication detail on a workload cluster:

# Run against the workload cluster; DRBDResource lives in the VM's namespace
oc get dres -n production

Running a test failover

Create a TestFailover to validate DR without disrupting production:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: web-tier-drill-1
  namespace: dr-myzone
spec:
  protectionGroupRef:
    name: web-tier
  cleanupPolicy: Automatic
  retentionTime: 2h
  schedulingCheck: Fail
  verification:
    reachability:
      - type: http
        vmName: web-app
        port: 8080
        endpoint: /healthz
        expectedStatus: 200
        timeout: 30s
oc apply -f test-failover.yaml -n dr-myzone
oc get tfo -n dr-myzone -w

After the test completes, retrieve the test namespace and results:

oc get tfo web-tier-drill-1 -n dr-myzone \
  -o jsonpath='{.status.testNamespace} {.status.result} {.status.schedulingCheck.verdict}'

To keep the test resources for manual inspection, set cleanupPolicy: Manual and trigger cleanup when ready:

oc patch tfo web-tier-drill-1 -n dr-myzone \
  --type merge -p '{"spec":{"cleanupRequested":true}}'

Executing a planned failover

Create a FailoverRequest to move the protection group to cluster2:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-fo-1
  namespace: dr-myzone
spec:
  protectionGroupRef:
    name: web-tier
  targetCluster: cluster2
  failoverType: planned
  batchBootTimeoutSeconds: 300
oc apply -f failover-request.yaml -n dr-myzone
oc get fr -n dr-myzone -w

Watch per-VM progress:

oc get fr planned-fo-1 -n dr-myzone \
  -o jsonpath='{.status.vmStatuses}'

Executing an unplanned failover

When the primary cluster is unavailable, create a FailoverRequest with failoverType: unplanned. DRBD volumes are force-promoted on the target without waiting for the primary to shut down:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: emergency-fo-1
  namespace: dr-myzone
spec:
  protectionGroupRef:
    name: web-tier
  targetCluster: cluster2
  failoverType: unplanned

If replication was Healthy and Consistent at the moment of failure, zero data loss occurs. If replication was Degraded, minimal data loss may occur.


Examples

Example 1: Minimal ProtectionZone

The smallest valid ProtectionZone with two workload clusters. The storageBackendMode field is required and must be DRBD.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: zone-east-west
  namespace: dr-zone-east-west
spec:
  displayName: "East-West DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Example 2: ProtectionGroup with boot ordering and SLA

Three VMs that must start in order: database first, then cache, then application. The SLA records compliance targets.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-stack
  namespace: dr-zone-east-west
spec:
  sourceCluster: cluster1
  virtualMachines:
    - name: app-db
      namespace: production
      bootOrder: 1
    - name: app-cache
      namespace: production
      bootOrder: 2
    - name: app-server
      namespace: production
      bootOrder: 3
  desiredState: running
  sla:
    rtoTargetSeconds: 300
    rpoTargetSeconds: 0
  vmIdentity:
    macAddresses: Preserve

Example 3: ProtectionRequest with custom metadata storage and standby placement

Protect a VM, using a dedicated fast StorageClass for DRBD metadata PVCs and relaxing node affinity on the DR copy.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-app-server
  namespace: dr-zone-east-west
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: app-server
    namespace: production
  replicationConfig:
    replicationProtocol: C
    rpo: 0
    drbdMetadataConfig:
      storageClassName: fast-metadata
      size: 128Mi
  standbyPlacement:
    nodeAffinity:
      dropAll: true

Example 4: TestFailover with TCP and HTTP probes

Run a DR drill with automatic cleanup after 4 hours, with a TCP probe on port 5432 (database) and an HTTP health check on the application server.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: app-stack-drill
  namespace: dr-zone-east-west
spec:
  protectionGroupRef:
    name: app-stack
  cleanupPolicy: Automatic
  retentionTime: 4h
  schedulingCheck: Fail
  verification:
    reachability:
      - type: tcp
        vmName: app-db
        port: 5432
        timeout: 30s
      - type: http
        vmName: app-server
        port: 8080
        endpoint: /health
        expectedStatus: 200
        timeout: 30s

Example 5: FailoverRequest — planned failover to DR cluster

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: planned-failover-to-dr
  namespace: dr-zone-east-west
spec:
  protectionGroupRef:
    name: app-stack
  targetCluster: cluster2
  failoverType: planned
  batchBootTimeoutSeconds: 300

Example 6: FailoverRequest — unplanned (emergency) failover

For use when the primary cluster is unreachable. Force-promotes on the DR cluster immediately.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: emergency-failover
  namespace: dr-zone-east-west
spec:
  protectionGroupRef:
    name: app-stack
  targetCluster: cluster2
  failoverType: unplanned

Example 7: Hook definition with a credential volume

A reusable hook that runs a database quiesce script, mounted from a Secret.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: Hook
metadata:
  name: db-quiesce
  namespace: dr-zone-east-west
spec:
  description: "Quiesce the PostgreSQL database before failover"
  timeoutSeconds: 120
  retries: 2
  retryDelaySeconds: 10
  jobTemplate:
    image: quay.io/myorg/db-tools:1.2.3
    command: ["/scripts/quiesce.sh"]
    volumes:
      - name: db-creds
        secret:
          secretName: db-admin-secret
    volumeMounts:
      - name: db-creds
        mountPath: /etc/db-creds
        readOnly: true
    resources:
      requests:
        cpu: 100m
        memory: 64Mi
      limits:
        cpu: 500m
        memory: 128Mi

Bind it to a ProtectionGroup as a pre-failover hook:

# In ProtectionGroup.spec.hooks:
hooks:
  - name: quiesce-db
    hookName: db-quiesce
    type: pre-failover
    runOn: source
    onFailure: Abort
    order: 1

Example 8: Reading DRBDResource status on a workload cluster

Query the DRBDResource for a VM named app-db in the production namespace. Run this against the workload cluster, not the TSR-Manager cluster.

# Switch context to the workload cluster
oc get dres -n production
oc get dres drbd-production-app-db-a1b2c3d4 -n production -o yaml

Check per-volume sync state:

oc get dres drbd-production-app-db-a1b2c3d4 -n production \
  -o jsonpath='{.status.volumes[*].replicationState}'

Example 9: ProtectionZone with runtime log level override

Enable debug logging for the DRBD node agents without changing the control-plane log level:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: zone-east-west
  namespace: dr-zone-east-west
spec:
  displayName: "East-West DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig
  logging:
    level: info
    nodeAgentLevel: debug

The change takes effect within approximately 15 seconds with no pod restarts.


Troubleshooting

Use the following format for each issue: Symptom, Likely cause, Fix.


Issue 1: ProtectionZone admission denied — Linstor backend is not supported yet; use DRBD

Symptom: oc apply of a ProtectionZone CR returns an admission webhook error containing the message above.

Likely cause: spec.storageBackendMode is set to Linstor.

Fix: Change spec.storageBackendMode to DRBD. This is the only accepted value in Tech Preview.

spec:
  storageBackendMode: DRBD

Issue 2: ProtectionRequest stuck in Failed phase with Validated=False

Symptom: The ProtectionRequest reaches the Validating phase and then transitions to Failed. A Validated=False condition and a warning event describe the rejection.

Likely causes:

  • One or more of the VM's disks is not volumeMode: Block.
  • The VM is not running at the time of protection (the VM must be running).
  • The sourceCluster field references a clusterID that does not match either cluster1 or cluster2.

Fix: Correct the VM configuration or the CR field that the warning event identifies. Delete the failed ProtectionRequest and create a new one—the controller does not retry failed requests.

# Read the event for the exact reason
oc describe protectionrequest <name> -n <namespace> --context=primary-cluster

Issue 3: oc adm drain does not complete on a node hosting a protected VM

Symptom: oc adm drain <node> hangs or reports that eviction of a VM pod was denied.

Likely cause: The node hosts a running protected VM. All KubeVirt migration paths (user-initiated, evacuation, drain) are realised as VirtualMachineInstanceMigration objects, and the TSR admission webhook denies these for protected VMs.

Fix: There is no way to drain a node hosting a running protected VM without first unprotecting the VM. To perform node maintenance:

  • For in-place maintenance (reboot, kernel update): proceed with the reboot. DRBD modules reload on boot, the anchor pod returns to the same node, and DRBD reconnects and resynchronises the delta.
  • For permanent node replacement: unprotect the VM, perform the node replacement, then protect the VM again (full initial resync required).

Issue 4: TestFailover stuck in CleaningUp or not cleaning up

Symptom: The TestFailover CR has been in CleaningUp for an extended period, or resources remain in the test namespace after the test completed.

Likely cause: cleanupPolicy: Manual was set, or the test failed partway through cleanup.

Fix: Do not patch finalizers. Delete the TestFailover CR through the normal API and the controller will handle cleanup:

oc delete testfailover <name> -n dr-production --context=quorum-cluster

If cleanup is blocked, inspect the test namespace (read from status.testNamespace) for stuck resources:

TEST_NS=$(oc get testfailover <name> -n dr-production \
  --context=quorum-cluster \
  -o jsonpath='{.status.testNamespace}')

oc get all -n "$TEST_NS" --context=dr-cluster

Issue 5: FailoverRequest transitions to Failed before completing

Symptom: The FailoverRequest transitions from InProgress directly to Failed.

Likely causes:

  • The failover exceeded the 5-minute wall-clock budget.
  • VMs failed to start on the target cluster within batchBootTimeoutSeconds.
  • DRBD volume promotion failed on the target side.

Fix:

  1. Check per-VM detail in status.vmStatuses:
oc get failoverrequest <name> -n dr-production \
  --context=quorum-cluster \
  -o jsonpath='{.status.vmStatuses}'
  1. Check the failover-controller logs on the quorum cluster:
oc logs -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=quorum-control-plane \
  -n dr-production \
  --context=quorum-cluster \
  -c manager
  1. Check DRBD resource state on both workload clusters:
oc get drbdresource -n <vm-namespace> --context=primary-cluster
oc get drbdresource -n <vm-namespace> --context=dr-cluster
  1. If the primary cluster is unreachable and you used failoverType: planned, re-apply the request with failoverType: unplanned to force-promote without waiting for the primary.

Issue 6: ReplicationGroupStatus shows Degraded or Critical

Symptom: oc get replicationgroupstatus reports Degraded or Critical health for a Protection Group.

Likely causes:

  • Network connectivity between primary and DR worker nodes on TCP ports 7000–8000 has been interrupted.
  • The DRBD node agent on a worker node is not running or has not converged (check for the siterecovery.trilio.io/not-ready:NoSchedule taint on nodes).
  • A worker node hosting a protected VM was lost.

Fix:

  1. Check RPOEvent records for the affected group:
oc get rpoevent -n <vm-namespace> --context=primary-cluster
  1. Check node agent DaemonSet pod status on both workload clusters:
oc get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=workload-control-plane \
  -n <agent-namespace> --context=primary-cluster

oc get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=workload-control-plane \
  -n <agent-namespace> --context=dr-cluster
  1. Check for the node taint:
oc get nodes --context=primary-cluster \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.taints}{"\n"}{end}'
  1. If a node was permanently lost, follow the node-loss recovery runbook at docs/runbooks/drbdresource-node-loss.md. Remove only the agent finalizer—removing the placement finalizer will permanently leak a PVC/PV pair.

Issue 7: replicationProtocol: A rejected at admission

Symptom: Applying a DRBDReplicationPolicy with drbdProtocol: A results in an admission error: ReplicationProtocolA is not supported yet.

Likely cause: Protocol A (asynchronous replication) is not supported in Tech Preview. The DRBDReplicationPolicy is also created automatically by the ProtectionZone controller—you should not be creating it manually.

Fix: Do not create or modify DRBDReplicationPolicy objects. The ProtectionZone controller creates the default policy (<zoneName>-default) automatically when the zone reaches Available. Only synchronous replication (Protocol C, RPO=0) is supported in Tech Preview.


Issue 8: Collecting a diagnostic bundle

Symptom: You need to provide a support bundle to Trilio support.

Fix: Run tsr-gather via oc adm must-gather against the quorum cluster. This collects logs, CRD state, and configuration from all clusters in the DR estate without capturing secrets or credentials:

oc adm must-gather \
  --image=quay.io/triliovault/tsr-gather:latest \
  --context=quorum-cluster \
  -- /usr/bin/tsr-gather

The resulting bundle in ./must-gather.local.* can be shared with support.