Prerequisites

This page lists every requirement that must be satisfied before you install Trilio Site Recovery (Tech Preview). Meeting these requirements is a hard prerequisite: the operator will reject configurations that violate enforced constraints, and configurations outside the validated set are unsupported. Review each section carefully before provisioning hardware, configuring networking, or installing software.

Prerequisites Cluster topology Trilio Site Recovery requires three OpenShift clusters: Role Purpose Quorum cluster (Site C) Hosts the management plane: the OLM-installed operator, the TSR web console, and the quorum control plane. Runs no application VMs. Primary cluster (cluster1) Runs the active VM workloads and the workload control plane. DR cluster (cluster2) Hosts standby VMs and a mirror workload control plane. Receives replicated data from the primary. All three clusters must be reachable from one another at the API and storage layers (see Network requirements below).

OpenShift version Two version ranges apply, and you should satisfy both: OLM-installable range — read from the com.redhat.openshift.versions annotation in deploy/olm-catalog/site-recovery/metadata/annotations.yaml, currently v4.14-v4.19. The operator bundle can be installed on clusters in this range. Validated range — OpenShift 4.17–4.20 (Kubernetes 1.30–1.33), per docs/tech-preview-supported-configurations.md. This is the range Tech Preview testing covers. OpenShift 4.14–4.16 sits within the supported floor but is not covered by Tech Preview testing. Versions beyond 4.20 are expected to work but have not been validated. Use 4.17–4.20 on all three clusters unless you have a specific reason not to.

Note: Upstream Kubernetes workload clusters (non-OpenShift) are not a supported configuration. The TSR web console is served behind the OpenShift oauth-proxy, and the Route and console link are created by the ProtectionZone controller. Neither an ingress path nor an authentication path exists on upstream Kubernetes.

Virtualization OpenShift Virtualization (CNV) ≥ 1.0 installed on the primary cluster and the DR cluster. The quorum cluster does not run VMs and does not require OpenShift Virtualization.

Storage DRBD kernel module DRBD kernel module ≥ 9.x loaded on every worker node of the primary cluster and the DR cluster. The drbd-node-agent DaemonSet manages block replication at the kernel level. If the module is absent or at an unsupported version, the DaemonSet pods will enter a crash loop and no replication can be established. LVM thin pool A dedicated LVM thin pool backed by a dedicated block device must be configured on every worker node of the primary cluster and the DR cluster before you deploy the workload control plane. DRBD uses this thin pool as its backing storage for replicated VM volumes. The thin pool must be large enough to accommodate all VM disk images you intend to protect, plus snapshot overhead during test failovers.

CLI tools Tool Version Notes oc Matching your OpenShift version Required. oc is the only supported CLI for OpenShift-specific paths (oc adm must-gather, Routes). kubectl-only operation is not a supported mode. kubectl Compatible with OpenShift 4.17+ API server Used for CRD and custom resource operations alongside oc.

You do not need Helm. The Helm charts for the quorum control plane, the workload control planes and the DRBD node agents are embedded in the operator binary, and the ProtectionZone controller installs them for you once a ProtectionZone reaches Available. There is no chart repository to add and no helm install step in any supported customer procedure.

Kubeconfig files You need a valid kubeconfig file for each of the three clusters:

~/.kube/config-quorum    # Quorum cluster
~/.kube/config-cluster1  # Primary cluster
~/.kube/config-cluster2  # DR cluster

Have these ready before you create the ProtectionZone. You store the two workload kubeconfigs as Secrets in the dr-<name> namespace on the quorum cluster and reference them from the ProtectionZone's clusters[].kubeconfigSecretRef; the controller reads them from there.

Network requirements Between primary and DR worker nodes (DRBD replication traffic) TCP ports 7000–8000 must be open in both directions between every worker node on the primary cluster and every worker node on the DR cluster. DRBD replication uses one port per replicated volume from this range. Blocking even a subset of these ports will cause replication to stall and Protection Groups to enter a Degraded state. Note that the range is inclusive of port 8000 — a firewall rule that stops at 7999 will leave one volume in a large estate unable to replicate. Between the quorum cluster and workload cluster API servers TCP port 6443 (Kubernetes API) must be open from the quorum cluster to the API server of both the primary cluster and the DR cluster. The failover, protection, pg-sync, and replication-monitor controllers running on the quorum cluster communicate with workload clusters exclusively through their API servers. Latency requirements Average round-trip time between primary and DR worker nodes must be below 50 ms. Synchronous DRBD Protocol C is the only supported replication mode. RPO is always 0 and is not tunable. Asynchronous replication (Protocol A) is rejected by the admission webhook with ReplicationProtocolA is not supported yet, so there is no long-distance, higher-RPO configuration to trade off against. Because every write is acknowledged only after it has committed on both clusters, exceeding 50 ms RTT will cause write I/O to stall on the primary. This is a hard constraint on how far apart the two sites can be, not a tuning preference. Measure it before you deploy — see Example 5.

Permissions The accounts you use to install and operate Site Recovery need the following access levels: Action Required permission Install the OLM operator bundle on the quorum cluster cluster-admin on the quorum cluster Create the dr-<name> namespace and the kubeconfig Secrets within it Namespace admin or cluster-admin on the quorum cluster Apply CRD manifests (ProtectionZone, ProtectionRequest, FailoverRequest, TestFailover) At least edit on the target namespace; cluster-admin for CRD-level operations Run oc adm must-gather with tsr-gather cluster-admin on the quorum cluster The kubeconfig you supply for each workload cluster must itself carry cluster-admin on that cluster. The ProtectionZone controller uses it to deploy the workload control plane and the DRBD node agent DaemonSet, and the DaemonSet requires a privileged SCC. You do not install anything on the workload clusters by hand.

Optional components The following components are not required for a basic deployment but unlock specific capabilities: Component Purpose Containerized Data Importer (CDI) Required if your VMs use DataVolumes. The pg-sync-controller converts DataVolume references to PVC references when building standby VM specs on the DR cluster. Multus CNI Required if your VMs use secondary network interfaces that must be preserved across failover.

Trilio Site Recovery is installed via the Operator Lifecycle Manager (OLM) on the TSR-Manager cluster. The OLM bundle installs the protection-zone controller, the TSR web console (site-recovery-backend and site-recovery-frontend), and the ProtectionZone CRD. Once the operator is running, you create a ProtectionZone CR, and the ProtectionZone controller automatically deploys the control-plane components on all three clusters — you never run helm install directly.

Overview of the install sequence

1. Install the OLM operator bundle on the TSR-Manager cluster.
2. Create kubeconfig Secrets for both workload clusters.
3. Create a ProtectionZone CR — the controller does the rest.

Step 1 — Create the operator namespace

All TSR operator components install into trilio-site-recovery-system on the TSR-Manager cluster.

oc create namespace trilio-site-recovery-system

Step 2 — Create the CatalogSource

This makes the TSR operator bundle available in OperatorHub on the TSR-Manager cluster.

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: trilio-site-recovery-catalog
  namespace: trilio-site-recovery-system
spec:
  sourceType: grpc
  image: quay.io/triliovault/site-recovery-catalog:<release-tag>
  displayName: Trilio Site Recovery
  publisher: Trilio Data, Inc.
  updateStrategy:
    registryPoll:
      interval: 10m

oc apply -f catalogsource.yaml

Replace <release-tag> with the pinned release tag for your deployment. Do not use a floating tag such as latest.

Step 3 — Create the OperatorGroup

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: trilio-site-recovery-og
  namespace: trilio-site-recovery-system
spec:
  targetNamespaces:
    - trilio-site-recovery-system

oc apply -f operatorgroup.yaml

Step 4 — Create the Subscription

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: site-recovery
  namespace: trilio-site-recovery-system
spec:
  channel: tech-preview
  name: site-recovery
  source: trilio-site-recovery-catalog
  sourceNamespace: trilio-site-recovery-system

oc apply -f subscription.yaml

Wait for the operator pod to reach Running state:

oc -n trilio-site-recovery-system get pods -l app.kubernetes.io/part-of=site-recovery -w

You should see the protection-zone controller, site-recovery-backend, and site-recovery-frontend pods running (1/1 Ready each).

Step 5 — Create zone credentials on the workload clusters

Each workload cluster must have a zone credential — a time-limited kubeconfig holding a ServiceAccount token with exactly the permissions TSR requires. Use tsrctl to create these credentials. Run the following command with cluster-admin access on each workload cluster:

# Create the zone credential on the primary workload cluster
tsrctl zone-credential create

# Create the zone credential on the DR workload cluster
tsrctl zone-credential create

The zone-credential create command creates a per-zone ServiceAccount in trilio-site-recovery-access on the target cluster, binds the required ClusterRole, and produces a time-limited kubeconfig (default: one year) with the cluster CA embedded. Save the output kubeconfig from each workload cluster — you will store them as Secrets on the TSR-Manager cluster in the next step.

Note: The zone warns 30 days before the token expires. Use tsrctl zone-credential rotate with the --refresh-rbac or --revoke-previous flags to issue a new token when needed.

Step 6 — Store kubeconfig Secrets on the TSR-Manager cluster

Create a namespace for this protection zone. Zone namespaces must use the dr-<name> prefix:

oc create namespace dr-prod

Store the workload cluster kubeconfigs as Secrets in this namespace:

oc -n dr-prod create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=/path/to/cluster1-kubeconfig

oc -n dr-prod create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=/path/to/cluster2-kubeconfig

Step 7 — Provision the webhook serving certificate

The protection-zone controller and the workload control-plane webhooks require a TLS serving certificate. This is not provisioned automatically — you must create it out of band before the ProtectionZone CR is applied.

Create a Secret named webhook-server-cert (the default webhook.certSecretName) containing tls.crt and tls.key in the zone namespace. The certificate's Subject Alternative Names must cover:

<release>-site-recovery-protectionzone-controller-webhook.dr-prod.svc
<release>-site-recovery-protectionzone-controller-webhook.dr-prod.svc.cluster.local

After you have a certificate and key:

oc -n dr-prod create secret tls webhook-server-cert \
  --cert=/path/to/tls.crt \
  --key=/path/to/tls.key

Step 8 — Create the ProtectionZone CR

The ProtectionZone CR drives all subsequent automation. Once it reaches the Available phase, the controller automatically deploys the site-recovery-manager-control-plane on the TSR-Manager cluster and the site-recovery-workload-control-plane (including the DRBD node agent DaemonSet) on both workload clusters.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: "Production DR Zone"
  description: "DRBD protection zone pairing primary and DR sites."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

oc apply -f protectionzone.yaml

Monitor the ProtectionZone status:

oc -n dr-prod get pz prod -w

The zone progresses through Provisioning → Available. When it reaches Available, all control-plane components are running.

Step 9 — Verify the deployment

Verify the TSR-Manager-side components in the zone namespace:

oc -n dr-prod get pods -l app.kubernetes.io/part-of=site-recovery

Verify the workload control-plane components on each workload cluster (run against each cluster's kubeconfig):

oc -n trilio-site-recovery-system get pods -l app.kubernetes.io/part-of=site-recovery

Verify the DRBD node agent DaemonSet on each workload cluster:

oc -n trilio-site-recovery-system get daemonset -l app.kubernetes.io/component=drbd-node-agent

Verify the replication policy was created automatically:

oc -n dr-prod get drp

You should see a policy named prod-default with Default: true. This policy is managed by the controller — do not edit or delete it.

Configuration in Trilio Site Recovery is expressed through Kubernetes Custom Resources, not through Helm values files or command-line flags that you set directly. The Helm charts are embedded in the operator binary and deployed automatically by the ProtectionZone controller when the zone reaches Available — you never run helm install or pass --set flags.

The key configuration objects and their effects are described below.

ProtectionZone

The ProtectionZone CR is the primary configuration object. It controls which clusters participate in a DR zone, how they are reached, and the runtime behavior of all components in that zone.

Required fields:

Optional fields:

Immutable fields (cannot be changed after creation):

• spec.storageBackendMode
• spec.clusters[].name
• spec.clusters[].clusterID

Mutable fields (can be changed after creation):

• spec.displayName, spec.description
• spec.clusters[].kubeconfigSecretRef (to rotate credentials)
• spec.clusters[].distributionName (to follow an OS upgrade)
• spec.logging.*
• spec.hooks.*

ProtectionRequest

A ProtectionRequest CR requests DRBD block-level protection for a single VM. You create one per VM.

DRBDReplicationPolicy

You do not create or modify this object. The ProtectionZone controller automatically creates one policy named <zoneName>-default with isDefault: true in the zone namespace when the zone reaches Available. Only the default policy in a namespace is ever consulted by the protection controller.

If you inspect this object, you will see it uses Protocol C (synchronous) and maps storage classes between clusters. These values reflect the zone configuration and are not user-editable.

Webhook serving certificate

The webhook serving certificate is not managed by any chart or controller. Before the ProtectionZone controller can function, the webhook-server-cert Secret (the default webhook.certSecretName) must exist in each zone namespace and on each workload cluster namespace (trilio-site-recovery-system). It must contain tls.crt and tls.key. You are responsible for creating and rotating this Secret. There is no cert-manager integration.

Runtime log level

You can change the log level of all zone components without restarting pods by updating spec.logging.level on the ProtectionZone CR. The change propagates via the site-recovery-logging ConfigMap within approximately 15 seconds.

Accepted levels: error, info, debug, trace. There is no warn level.

oc -n dr-prod patch pz prod --type merge -p '{"spec":{"logging":{"level":"debug"}}}'

To set a component-specific level, use spec.logging.controlPlaneLevel or spec.logging.nodeAgentLevel.

Node agent resource limits

The DRBD node agent DaemonSet is deployed by the ProtectionZone controller using the values embedded in the operator. The default resource allocation is:

If your nodes require different values, contact Trilio support — chart values are an internal surface and are not exposed for customer modification.

Zone configuration tuning

The workload control plane installs a site-recovery-zone-config ConfigMap in trilio-site-recovery-system on each workload cluster. This ConfigMap controls provisioning deadlines:

These values are re-applied on every operator reconcile. To make a persistent change, update the ProtectionZone CR or contact Trilio support.

After installation, all DR operations are performed by creating or deleting Kubernetes Custom Resources on the TSR-Manager cluster. The oc and kubectl CLIs are your primary tools.

Protecting a virtual machine

To protect a VM, create a ProtectionRequest CR in the zone namespace on the TSR-Manager cluster. The VM must be running at protect time — its running node becomes its pinned source node.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-web-server
  namespace: dr-prod
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: web-server
    namespace: production

Apply it against the TSR-Manager cluster:

oc -n dr-prod apply -f protect-web-server.yaml

Watch protection progress:

oc -n dr-prod get pr protect-web-server -w

The ProtectionRequest progresses through: Pending → Validating → CreatingDRBD → Syncing → ReadyToActivate → Activating → Protected.

Once Protected, the VM carries the label siterecovery.trilio.io/protected-vm=true on the source workload cluster. Check it:

# Run against the primary workload cluster
oc get vm -A -l siterecovery.trilio.io/protected-vm=true

Grouping VMs into a ProtectionGroup

VMs that must fail over together are grouped in a ProtectionGroup. Create the ProtectionGroup in the zone namespace on the TSR-Manager cluster, listing the VMs by name and namespace.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-tier
  namespace: dr-prod
spec:
  virtualMachines:
    - name: web-server
      namespace: production
    - name: database
      namespace: production
  sourceCluster: cluster1
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

oc -n dr-prod apply -f app-tier-pg.yaml

Monitoring replication health

The replication monitor automatically creates a ReplicationGroupStatus (short name: rgs) for each ProtectionGroup, named <protection-group-name>-replication:

oc -n dr-prod get rgs

For RPO violation history, check RPOEvent records (short name: rpoe):

oc -n dr-prod get rpoe -l siterecovery.trilio.io/protection-group=app-tier

Check DRBDResource status on the workload clusters (not on the TSR-Manager cluster — this CRD does not exist there):

# Run against the primary workload cluster
oc -n production get dres

Running a test failover

A test failover is non-disruptive. It creates volume snapshots, starts test VMs in an isolated namespace, runs verification checks, and cleans up. Create a TestFailover CR on the TSR-Manager cluster:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: app-tier-drill
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-prod
  cleanupPolicy: Automatic
  retentionTime: 2h

oc -n dr-prod apply -f test-failover.yaml

Monitor progress:

oc -n dr-prod get tfo app-tier-drill -w

The controller generates the test namespace automatically — its name is available in status.testNamespace. Do not attempt to construct or guess the name.

Test failover phases: Queued → PreparingEnvironment → CreatingSnapshots → CreatingVolumes → CreatingVMs → StartingVMs → Verifying → Succeeded (or Failed) → CleaningUp → Cleaned.

Triggering a planned failover

A planned failover gracefully shuts down VMs on the source cluster, waits for replication to be consistent, and starts VMs on the target cluster. Initiate it by creating a FailoverRequest on the TSR-Manager cluster:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: app-tier-failover
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: planned

oc -n dr-prod apply -f failover.yaml

Watch progress:

oc -n dr-prod get fr app-tier-failover -w

Per-VM status is in status.vmStatuses. There is no status.message or status.failedVMs field on a FailoverRequest.

Triggering an unplanned failover

Use failoverType: unplanned when the primary cluster is unavailable. DRBD volumes are force-promoted on the DR cluster without waiting for the primary to shut down. If replication was Healthy and Consistent at the moment of failure, data loss is zero. If replication was Degraded, minimal data loss may occur.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: app-tier-emergency
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: unplanned

Managing zone credentials

Zone credentials have a default lifetime of one year. The ProtectionZone status warns 30 days before expiry. To rotate a credential:

# Rotate and revoke the previous token
tsrctl zone-credential rotate --revoke-previous

# Rotate and also refresh the RBAC to pick up any new permissions
tsrctl zone-credential rotate --refresh-rbac

After rotating, update the kubeconfig Secret in the zone namespace with the new kubeconfig output, and the zone will pick up the new credential on the next reconcile.

To inspect the current credential:

tsrctl zone-credential inspect

To delete a credential:

tsrctl zone-credential delete
# Use --force to remove even if the zone still references it
tsrctl zone-credential delete --force

Using the TSR web console

On OpenShift, the ProtectionZone controller creates a Route and a console link for the TSR web console. After the zone reaches Available, the console link appears in the OpenShift web console navigation. Browser access is authenticated through the OpenShift oauth-proxy.

The web console is installed as part of the OLM operator bundle — there is no opt-out. On upstream Kubernetes, the console pods are deployed but no ingress or authentication path is provided, and browser access is not a supported use case.

Zone-scoped pages in the console derive the dr-<zoneId> namespace from the zone ID. A zone in a differently named namespace will appear in the list but its pages will not work.

Example 1 — Minimal ProtectionZone CR

A DRBD protection zone pairing two RHCOS 9 clusters:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: "Production DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Example 2 — Check ProtectionZone status

oc -n dr-prod get pz prod -o wide

Example 3 — Protect a single VM

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-database
  namespace: dr-prod
spec:
  sourceCluster: cluster1
  virtualMachine:
    name: database
    namespace: production
  replicationConfig:
    replicationProtocol: C
    rpo: 0
    drbdMetadataConfig:
      size: 128Mi

oc -n dr-prod apply -f protect-database.yaml
oc -n dr-prod get pr protect-database -w

Example 4 — Create a ProtectionGroup for coordinated failover

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-tier
  namespace: dr-prod
spec:
  virtualMachines:
    - name: web-server
      namespace: production
    - name: database
      namespace: production
  sourceCluster: cluster1
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

oc -n dr-prod apply -f app-tier-pg.yaml
oc -n dr-prod get pg app-tier

Example 5 — View replication health

# Check the auto-created ReplicationGroupStatus for app-tier
oc -n dr-prod get rgs app-tier-replication -o yaml

# Check per-VM replication status
oc -n dr-prod get pg app-tier -o jsonpath='{.status.protectedVMs}'

# List all RPO violation events for app-tier
oc -n dr-prod get rpoe -l siterecovery.trilio.io/protection-group=app-tier

Example 6 — Non-disruptive test failover with manual cleanup

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: app-tier-drill
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-prod
  cleanupPolicy: Manual
  retentionTime: 4h
  schedulingCheck: Fail

oc -n dr-prod apply -f test-failover.yaml

# Watch progress
oc -n dr-prod get tfo app-tier-drill -w

# Retrieve the auto-generated test namespace
oc -n dr-prod get tfo app-tier-drill -o jsonpath='{.status.testNamespace}'

# When done inspecting, trigger cleanup while keeping the record
oc -n dr-prod patch tfo app-tier-drill --type merge -p '{"spec":{"cleanupRequested":true}}'

Example 7 — Planned failover to the DR cluster

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: app-tier-planned
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: planned

oc -n dr-prod apply -f failover.yaml
oc -n dr-prod get fr app-tier-planned -w

# Check per-VM status
oc -n dr-prod get fr app-tier-planned -o jsonpath='{.status.vmStatuses}'

Example 8 — Unplanned failover (emergency)

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: app-tier-emergency
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-tier
    namespace: dr-prod
  targetCluster: cluster2
  failoverType: unplanned

oc -n dr-prod apply -f emergency-failover.yaml
oc -n dr-prod get fr app-tier-emergency -w

Example 9 — Collect a support bundle

The tsr-gather tool is shipped as an oc adm must-gather image. Run it against the TSR-Manager cluster:

oc adm must-gather --image=quay.io/triliovault/site-recovery-must-gather:<release-tag>

No -- <command> override is needed; the gather binary is the image entrypoint and runs automatically. To narrow the collection:

# Collect only the last 2 hours of logs
oc adm must-gather --image=quay.io/triliovault/site-recovery-must-gather:<release-tag> -- /usr/bin/gather --since 2h

# Collect only a specific zone
oc adm must-gather --image=quay.io/triliovault/site-recovery-must-gather:<release-tag> -- /usr/bin/gather --zone prod

Replace <release-tag> with the pinned release tag for your deployment.

Use a consistent troubleshooting workflow: check the ProtectionZone status conditions first, then check the relevant CR's status conditions, then collect component logs.

ProtectionZone stays in Provisioning and does not reach Available

Symptom: oc -n dr-prod get pz prod shows Status: Provisioning for more than a few minutes.

Likely causes and fixes:

DRBD node agent pods are not running on workload nodes

Symptom: oc -n trilio-site-recovery-system get pods -l app.kubernetes.io/component=drbd-node-agent shows pods in Init:Error or CrashLoopBackOff.

Likely causes and fixes:

To view init container logs:

# Get the agent pod name on a specific node
oc -n trilio-site-recovery-system get pods --field-selector spec.nodeName=<node-name> -o jsonpath='{.items[0].metadata.name}'

# View the module loader init container logs
oc -n trilio-site-recovery-system logs <pod-name> -c drbd-module-loader

ProtectionRequest stuck in Syncing or CreatingDRBD

Symptom: oc -n dr-prod get pr <name> shows Phase: Syncing or Phase: CreatingDRBD for an extended period.

Likely causes and fixes:

Check DRBDResource on the workload cluster (not on the TSR-Manager cluster):

# Run against the primary workload cluster
oc -n production get dres -l siterecovery.trilio.io/protection-group=app-tier

oc adm drain does not complete on a workload node

Symptom: oc adm drain <node> hangs and does not complete.

Cause: The node hosts a running protected VM. The DRBD node agent anchor pod cannot be evicted while the VM is protected and running.

Fix: You cannot drain a node hosting a running protected VM without first stopping the VM or triggering a failover. To proceed:

1. Perform a planned failover of any ProtectionGroups whose VMs are running on this node.
2. Or unprotect the VMs, drain the node, then re-protect (at the cost of a full resynchronization).

Live migration of a protected VM is denied

Symptom: virtctl migrate <vm> or creating a VirtualMachineInstanceMigration fails with an admission error.

Cause: Live migration of protected VMs is denied by the workload admission webhook. Protected VMs are pinned to their source node for the lifetime of the protection.

Fix: There is no supported migration path for a protected VM. If you need to move the VM to a different node, unprotect it, move it using your normal migration procedure, then protect it again. This costs a full initial resynchronization.

FailoverRequest stuck in InProgress

Symptom: oc -n dr-prod get fr <name> shows State: InProgress for longer than the 5-minute failover wall-clock budget.

Likely causes and fixes:

Collect a diagnostic bundle:

oc adm must-gather --image=quay.io/triliovault/site-recovery-must-gather:<release-tag> -- /usr/bin/gather --zone prod

Zone credential expiring

Symptom: oc -n dr-prod get pz prod -o yaml shows a condition with reason CredentialExpiring.

Fix: Rotate the credential before it expires:

tsrctl zone-credential rotate --revoke-previous

Update the kubeconfig Secret in the zone namespace with the new kubeconfig, then verify the zone condition clears.

kubectl get drbdresource returns server doesn't have a resource type

Symptom: Running oc get dres against the TSR-Manager cluster returns a "resource type not found" error.

Cause: DRBDResource CRDs are installed only on workload clusters, not on the TSR-Manager cluster. This is by design.

Fix: Run the command against a workload cluster kubeconfig:

oc --kubeconfig /path/to/cluster1-kubeconfig -n production get dres