Installation

This page walks you through installing Trilio Site Recovery (Tech Preview) on OpenShift. Installation has two complementary phases: first, the OLM operator bundle is installed on the TSR-Manager cluster, which deploys the ProtectionZone controller and the TSR web console; second, you create kubeconfig Secrets and a ProtectionZone custom resource, which causes the ProtectionZone controller to automatically deploy the control plane and DRBD node agents to both workload clusters. You do not run Helm directly — the charts are embedded in the operator binary and applied for you. All three clusters (TSR-Manager, primary, and DR) must be OpenShift 4.17 or later.

Before you begin, verify that your environment meets every requirement below. Items marked Enforced are rejected by the product if absent.

Clusters

• Three OpenShift clusters, each running OpenShift 4.17 or later:
  • TSR-Manager cluster — hosts the operator, ProtectionZone controller, and TSR web console. Requires no storage, no VMs, and no DRBD.
  • Primary cluster — hosts your active KubeVirt VMs.
  • DR cluster — hosts standby VMs and mirrors of protected volumes.
• Single-Node OpenShift (SNO) is supported for all three roles.
• The com.redhat.openshift.versions annotation in the OLM bundle metadata specifies the installable range as v4.14-v4.19. The validated range for Tech Preview is OpenShift 4.17–4.20. Versions 4.14–4.16 are within the installable range but are not validated for Tech Preview and are not supported. Versions beyond 4.20 are expected to work but have not been validated.
• Upstream Kubernetes (non-OpenShift) is not supported. The Helm charts install on plain Kubernetes but the configuration is untested, the DRBD node agent requires the OpenShift privileged SCC, and test-failover network isolation silently fails on non-OVN CNI.

Virtualization

• OpenShift Virtualization (KubeVirt) installed on both workload clusters (primary and DR). The TSR-Manager cluster does not need KubeVirt.
• The kubevirt.io/v1 API must serve VirtualMachine, VirtualMachineInstance, and VirtualMachineInstanceMigration kinds, including the virtualmachines/status subresource.

Storage — DRBD (required on both workload clusters)

• DRBD is the only supported storage backend. ProtectionZone.spec.storageBackendMode must be DRBD; the value Linstor is rejected by the admission webhook with: Linstor backend is not supported yet; use DRBD. Enforced.
• DRBD kernel module version 9 or later loaded on every worker node of the primary and DR clusters. An RHEL 9-based RHCOS node (el9 kernel) is required; RHEL 10-based nodes are not validated in Tech Preview.
• LVM thin pool configured on each worker node of both workload clusters, backed by a dedicated block device, for DRBD backing storage.
• Mixed OS families inside one cluster are not supported — each workload cluster must be internally uniform.

Network

• TCP ports 7000–8000 inclusive open between worker nodes of the primary and DR clusters. One port is allocated per replicated VM volume. Port 8000 is mandatory — a firewall rule that stops at 7999 will silently prevent replication on one volume.
• TCP port 6443 (Kubernetes API) open from the TSR-Manager cluster to the primary and DR API servers.
• Round-trip latency below 50 ms between primary and DR clusters. This is required for Protocol C synchronous replication (RPO=0), the only supported replication mode.

Tools and access

• oc CLI. oc is required for OpenShift-specific operations (Routes, oc adm must-gather, SCC bindings). kubectl-only operation is not a supported mode.
• Kubeconfig files for all three clusters.
• Cluster-admin rights on all three clusters for the initial installation.
• Access to quay.io/triliovault to pull operator and component images. Workload nodes must also be able to pull the DRBD module-loader image (pull policy Always).
• NTP/chrony running on every node of all three clusters. Clock skew causes credential and timestamp failures that do not announce themselves as TSR errors.

Replication constraints to understand before you begin

• Protocol C synchronous replication is the only supported mode. replicationProtocol: A is rejected at admission: ReplicationProtocolA is not supported yet. RPO is always 0 and is not tunable.
• A protected VM is pinned to one node per cluster for the lifetime of the protection. Live migration and cold migration of a protected VM are denied. oc adm drain of a node hosting a running protected VM will not complete.
• Scale limits: approximately 1000 protected VMs per deployment (one DRBD port per VM from the 7000–8000 range), 15 replicated disks per VM, exactly two workload clusters per zone.

Installation proceeds in four stages:

1. Install the OLM operator bundle on the TSR-Manager cluster.
2. Create kubeconfig Secrets for the two workload clusters.
3. Create a ProtectionZone CR — this triggers automatic deployment of the control plane and DRBD node agents.
4. Verify the installation.

Stage 1 — Install the OLM operator bundle on the TSR-Manager cluster

All commands in this stage run against the TSR-Manager cluster.

1.1 — Create the operator namespace

The operator must be installed in trilio-site-recovery-system.

oc --context <tsr-manager-context> create namespace trilio-site-recovery-system

1.2 — Create the CatalogSource

The catalog image is hosted on quay.io/triliovault. Replace <tag> with the release tag provided by Trilio.

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: trilio-site-recovery-catalog
  namespace: trilio-site-recovery-system
spec:
  sourceType: grpc
  image: quay.io/triliovault/site-recovery-catalog:<tag>
  displayName: Trilio Site Recovery
  publisher: Trilio Data, Inc.

oc --context <tsr-manager-context> apply -f catalogsource.yaml

1.3 — Create the OperatorGroup

apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: trilio-site-recovery-og
  namespace: trilio-site-recovery-system
spec:
  targetNamespaces:
    - trilio-site-recovery-system

oc --context <tsr-manager-context> apply -f operatorgroup.yaml

1.4 — Create the Subscription

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: site-recovery
  namespace: trilio-site-recovery-system
spec:
  channel: tech-preview
  name: site-recovery
  source: trilio-site-recovery-catalog
  sourceNamespace: trilio-site-recovery-system

oc --context <tsr-manager-context> apply -f subscription.yaml

1.5 — Wait for the operator to become ready

oc --context <tsr-manager-context> -n trilio-site-recovery-system \
  rollout status deployment/protection-zone-controller

Once the operator is ready, the following are running in trilio-site-recovery-system:

• protection-zone-controller Deployment — the ProtectionZone reconciler and admission webhook.
• site-recovery-frontend Deployment — the Next.js UI with OpenShift oauth-proxy sidecar.
• site-recovery-backend Deployment — the Flask API.

The TSR web console is installed at this point with no opt-out. The OpenShift Route and console link are created later, when the first ProtectionZone reaches Available.

Stage 2 — Create kubeconfig Secrets for the workload clusters

Each workload cluster is reached through a kubeconfig held in a Secret on the TSR-Manager cluster. Create one Secret per cluster in the zone namespace (dr-<name>) you will use in Stage 3.

Use tsrctl to create a least-privilege zone credential on each workload cluster. Run tsrctl with cluster-admin rights.

2.1 — Create the zone namespace on the TSR-Manager cluster

All zone-scoped resources — the ProtectionZone CR, control plane Deployments, and the kubeconfig Secrets — live in a namespace named dr-<zone-name>. Choose a zone name (for example prod) and create the namespace:

oc --context <tsr-manager-context> create namespace dr-prod

2.2 — Create zone credentials on each workload cluster

tsrctl zone-credential create creates a per-zone ServiceAccount, ClusterRole, and time-limited token on the target cluster, then writes the resulting kubeconfig.

# Create credential for the primary cluster (cluster1)
tsrctl zone-credential create \
  --zone prod \
  --kubeconfig <path-to-primary-kubeconfig> \
  --context <primary-context>

# Create credential for the DR cluster (cluster2)
tsrctl zone-credential create \
  --zone prod \
  --kubeconfig <path-to-dr-kubeconfig> \
  --context <dr-context>

Each command produces a kubeconfig file. Store them as Secrets in the dr-prod namespace on the TSR-Manager cluster:

# Primary cluster kubeconfig Secret
oc --context <tsr-manager-context> -n dr-prod \
  create secret generic cluster1-kubeconfig \
  --from-file=kubeconfig=<path-to-primary-zone-kubeconfig>

# DR cluster kubeconfig Secret
oc --context <tsr-manager-context> -n dr-prod \
  create secret generic cluster2-kubeconfig \
  --from-file=kubeconfig=<path-to-dr-zone-kubeconfig>

Stage 3 — Create the ProtectionZone CR

Creating the ProtectionZone CR triggers the ProtectionZone controller to:

1. Validate connectivity and permissions for both workload clusters.
2. Deploy the site-recovery control plane (manager Deployment and admission webhooks) into trilio-site-recovery-system on each workload cluster.
3. Deploy the DRBD node agent DaemonSet on every schedulable node of each workload cluster.
4. Create the default DRBDReplicationPolicy named prod-default with isDefault: true in the dr-prod namespace.
5. Create the OpenShift Route and console link for the TSR web console.

You do not run Helm. All chart installation is performed automatically by the ProtectionZone controller from charts embedded in the operator binary.

3.1 — Prepare the ProtectionZone manifest

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: "Production DR Zone"
  description: "Primary and DR site pairing for production VM workloads."
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Key field notes:

• storageBackendMode must be DRBD. The value Linstor is rejected at admission.
• clusterID must be cluster1 or cluster2. Each cluster in the zone must use a different slot.
• distributionName selects the DRBD kernel-module loader image: RHEL-CoreOS-9 (supported) or RHEL-CoreOS-10 (not validated in Tech Preview). This field is mutable if a cluster later moves to a newer RHEL base.
• kubeconfigSecretRef.name and .key must match the Secret you created in Stage 2.

3.2 — Apply the ProtectionZone

oc --context <tsr-manager-context> apply -f protectionzone.yaml

3.3 — Create the webhook TLS serving certificate Secret

The admission webhooks require a TLS serving certificate that the charts do not provision. You must create this Secret before the ProtectionZone controller can complete webhook installation. The Secret must contain tls.crt and tls.key and must be created in each release namespace.

On the TSR-Manager cluster, create the Secret in dr-prod:

# Replace with your actual certificate and key
oc --context <tsr-manager-context> -n dr-prod \
  create secret tls webhook-server-cert \
  --cert=<path-to-tls.crt> \
  --key=<path-to-tls.key>

On each workload cluster, create the Secret in trilio-site-recovery-system:

oc --context <primary-context> -n trilio-site-recovery-system \
  create secret tls webhook-server-cert \
  --cert=<path-to-tls.crt> \
  --key=<path-to-tls.key>

oc --context <dr-context> -n trilio-site-recovery-system \
  create secret tls webhook-server-cert \
  --cert=<path-to-tls.crt> \
  --key=<path-to-tls.key>

The certificate SAN must cover the webhook Service DNS names. For the workload chart these are:

<release>-site-recovery-workload-control-plane-webhook.trilio-site-recovery-system.svc
<release>-site-recovery-workload-control-plane-webhook.trilio-site-recovery-system.svc.cluster.local

For the protectionzone-controller chart on the TSR-Manager cluster:

<release>-site-recovery-protectionzone-controller-webhook.dr-prod.svc
<release>-site-recovery-protectionzone-controller-webhook.dr-prod.svc.cluster.local

Stage 4 — Verify the installation

4.1 — Check the ProtectionZone status

oc --context <tsr-manager-context> -n dr-prod get pz prod

Wait for STATUS to show Available. Provisioning typically takes 2–5 minutes.

4.2 — Verify the TSR-Manager control plane

The control plane Deployment runs in the zone namespace (dr-prod) on the TSR-Manager cluster:

oc --context <tsr-manager-context> -n dr-prod \
  get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=manager-control-plane

Each pod shows 1/1 Running. The single container is named manager and hosts all reconcilers (failover, protection, pg-sync, replication-monitor, test-failover).

4.3 — Verify workload control planes on both workload clusters

The workload components install in trilio-site-recovery-system on each workload cluster:

# Primary cluster — workload manager Deployment
oc --context <primary-context> -n trilio-site-recovery-system \
  get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=workload-control-plane

# Primary cluster — DRBD node agent DaemonSet (one pod per schedulable node)
oc --context <primary-context> -n trilio-site-recovery-system \
  get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=drbd-node-agent

# DR cluster — same checks
oc --context <dr-context> -n trilio-site-recovery-system \
  get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=workload-control-plane

oc --context <dr-context> -n trilio-site-recovery-system \
  get pods -l app.kubernetes.io/part-of=site-recovery,app.kubernetes.io/component=drbd-node-agent

All workload manager pods show 1/1 Running. The node agent DaemonSet shows one pod per schedulable node; all pods show 1/1 Running once the DRBD module is loaded.

4.4 — Verify the default replication policy

The ProtectionZone controller creates a DRBDReplicationPolicy named prod-default with isDefault: true:

oc --context <tsr-manager-context> -n dr-prod get drp

You should see one policy with Default set to true and Protocol set to C. Do not modify this policy.

4.5 — Verify the TSR web console

On the TSR-Manager cluster, check that the OpenShift Route exists:

oc --context <tsr-manager-context> -n trilio-site-recovery-system get route

A site-recovery-oauth-proxy Route should be present. Open it in a browser to access the TSR web console. The console link also appears in the OpenShift web console application menu.

The ProtectionZone CR is the primary configuration surface. You do not configure Helm values directly — the charts are embedded in the operator binary and deployed automatically. The fields below represent the configuration choices available to you.

ProtectionZone configuration

All fields live under spec.

Webhook TLS certificate configuration

The admission webhooks require a pre-created TLS Secret. The chart does not provision certificates and has no cert-manager integration.

The caBundle for each webhook configuration must be the base64-encoded CA that signed the serving certificate.

DRBD node agent configuration

These settings are applied by the ProtectionZone controller when deploying the workload chart. You cannot set them directly as a customer; they are listed here so you understand operational defaults.

Node taint behavior

The DRBD node agent applies a siterecovery.trilio.io/not-ready:NoSchedule taint to its own node at startup. The taint is removed once every DRBD resource on that node is connected. Workloads that do not tolerate this taint will not schedule on a freshly started node until DRBD has converged. Plan for this during initial deployment and after node reboots.

What you do not configure

The following are managed automatically and must not be modified:

• DRBDReplicationPolicy — created by the ProtectionZone controller as <zoneName>-default with isDefault: true and Protocol C. Customers do not author replication policies.
• Replication protocol — always Protocol C (synchronous, RPO=0). Protocol A is rejected at admission.
• DRBD port allocation — ports are allocated from the fixed range 7000–8000, one per replicated VM volume. The networkConfig.portRange field on the policy is accepted but never read by the allocator.

Once the ProtectionZone is Available, you interact with Trilio Site Recovery through Custom Resources. The primary workflows below show the most common operations.

Check zone and cluster status

All zone-scoped operations run against the TSR-Manager cluster in the zone namespace (dr-<name>).

# Overall zone status
oc --context <tsr-manager-context> -n dr-prod get pz prod

# Detailed zone conditions and per-cluster reachability
oc --context <tsr-manager-context> -n dr-prod describe pz prod

The zone status.phase values are: Provisioning, Available, Unavailable, Degraded, Deleting.

Create a ProtectionGroup

A ProtectionGroup defines the set of VMs that fail over together as a unit. It lives on the TSR-Manager cluster in the zone namespace.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: web-tier
  namespace: dr-prod
spec:
  desiredState: running
  sourceCluster: cluster1
  virtualMachines:
    - name: vm-web-01
      namespace: production
    - name: vm-web-02
      namespace: production
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

oc --context <tsr-manager-context> apply -f protection-group.yaml
oc --context <tsr-manager-context> -n dr-prod get pg web-tier

Protect a VM (create a ProtectionRequest)

A ProtectionRequest enables block-level DRBD replication for a single VM. The VM must be running when you protect it, because the controller captures the running node assignment at protection time.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-vm-web-01
  namespace: dr-prod
spec:
  virtualMachine:
    name: vm-web-01
    namespace: production
  sourceCluster: cluster1
  replicationConfig:
    replicationProtocol: C
    rpo: 0

oc --context <tsr-manager-context> apply -f protectionrequest.yaml

# Watch protection progress
oc --context <tsr-manager-context> -n dr-prod get pr protect-vm-web-01 -w

Protection phases are: Pending → Validating → CreatingDRBD → Syncing → ReadyToActivate → Activating → Protected.

Once Protected, the VM is pinned to its current node on the primary cluster. The node binding is immutable for the lifetime of the protection. Live migration and cold migration of a protected VM are denied.

Monitor replication health

The ReplicationGroupStatus named <protection-group-name>-replication is auto-created by the replication monitor for each ProtectionGroup.

oc --context <tsr-manager-context> -n dr-prod get rgs
oc --context <tsr-manager-context> -n dr-prod describe rgs web-tier-replication

The status.rpoStatus field reports Met, Degraded, Violated, or Unknown. The status.aggregated.connectionState field reports Connected, Degraded, or Disconnected.

Use the label siterecovery.trilio.io/protection-group=<pg-name> to filter RPOEvents for a specific group:

oc --context <tsr-manager-context> -n dr-prod get rpoe \
  -l siterecovery.trilio.io/protection-group=web-tier

Check DRBD resource state on a workload cluster

DRBDResource CRs live on each workload cluster in the protected VM's namespace — not on the TSR-Manager cluster.

# List DRBDResources for protected VMs in a namespace
oc --context <primary-context> -n production get dres

# Check a specific resource
oc --context <primary-context> -n production describe dres drbd-production-vm-web-01-<hash>

The DRBD resource name follows the pattern drbd-<namespace>-<vm>-<hash8>. Do not query for DRBDResources on the TSR-Manager cluster — the CRD is not registered there.

To find protected VMs on a workload cluster:

oc --context <primary-context> get vm -A \
  -l siterecovery.trilio.io/protected-vm=true

Run a test failover

A TestFailover validates your DR capability non-disruptively using volume snapshots. Production VMs continue running during the test.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: test-web-tier
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: web-tier
    namespace: dr-prod
  cleanupPolicy: Automatic
  retentionTime: 2h

oc --context <tsr-manager-context> apply -f testfailover.yaml
oc --context <tsr-manager-context> -n dr-prod get tfo test-web-tier -w

Test phases are: Queued → PreparingEnvironment → CreatingSnapshots → CreatingVolumes → CreatingVMs → StartingVMs → Verifying → Succeeded (or Failed) → CleaningUp → Cleaned.

The test namespace is generated by the controller and available in status.testNamespace. Do not attempt to construct or predict this name.

Execute a planned failover

A planned failover gracefully shuts down VMs on the source cluster, waits for DRBD sync to complete, promotes the DR volumes, and starts VMs on the target cluster. Every failover is initiated by creating a FailoverRequest.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: failover-web-tier
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: web-tier
  targetCluster: cluster2
  failoverType: planned

oc --context <tsr-manager-context> apply -f failoverrequest.yaml
oc --context <tsr-manager-context> -n dr-prod get fr failover-web-tier -w

The status.state field reports Pending, InProgress, Completed, or Failed. Per-VM status is in status.vmStatuses.

Rotate zone credentials before expiry

The ProtectionZone warns approximately 30 days before a zone credential's token expires (configurable via credentialExpiryWarning, default 720h). Rotate the credential before it expires:

tsrctl zone-credential rotate \
  --zone prod \
  --kubeconfig <path-to-cluster-kubeconfig> \
  --context <cluster-context>

Then update the Secret on the TSR-Manager cluster with the new kubeconfig. The zone manager restarts onto the new credential automatically.

Example 1 — Minimal ProtectionZone

A ProtectionZone pairing a primary and DR cluster, both running RHCOS 9:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionZone
metadata:
  name: prod
  namespace: dr-prod
spec:
  displayName: "Production DR Zone"
  storageBackendMode: DRBD
  clusters:
    - name: primary
      clusterID: cluster1
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster1-kubeconfig
        key: kubeconfig
    - name: dr
      clusterID: cluster2
      distributionName: RHEL-CoreOS-9
      kubeconfigSecretRef:
        name: cluster2-kubeconfig
        key: kubeconfig

Example 2 — ProtectionGroup with boot ordering

A ProtectionGroup where a database VM starts before application VMs:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionGroup
metadata:
  name: app-stack
  namespace: dr-prod
spec:
  desiredState: running
  sourceCluster: cluster1
  virtualMachines:
    - name: vm-database
      namespace: production
      bootOrder: 1
    - name: vm-app-01
      namespace: production
      bootOrder: 2
    - name: vm-app-02
      namespace: production
      bootOrder: 2
  sla:
    rpoTargetSeconds: 0
    rtoTargetSeconds: 300

Example 3 — ProtectionRequest with metadata storage class override

Specifying a dedicated storage class for the DRBD external-metadata PVCs:

apiVersion: siterecovery.trilio.io/v1alpha1
kind: ProtectionRequest
metadata:
  name: protect-vm-database
  namespace: dr-prod
spec:
  virtualMachine:
    name: vm-database
    namespace: production
  sourceCluster: cluster1
  replicationConfig:
    replicationProtocol: C
    rpo: 0
    drbdMetadataConfig:
      storageClassName: fast-local
      size: 256Mi

Example 4 — Unplanned failover (emergency)

Used when the primary cluster is unavailable. DRBD volumes are force-promoted on the DR cluster without waiting for the primary to shut down. If replication was healthy and consistent at the moment of failure, zero data loss occurs.

apiVersion: siterecovery.trilio.io/v1alpha1
kind: FailoverRequest
metadata:
  name: emergency-failover
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-stack
  targetCluster: cluster2
  failoverType: unplanned

oc --context <tsr-manager-context> apply -f emergency-failover.yaml
oc --context <tsr-manager-context> -n dr-prod get fr emergency-failover -w

Example 5 — Test failover with HTTP reachability probe

apiVersion: siterecovery.trilio.io/v1alpha1
kind: TestFailover
metadata:
  name: test-app-stack
  namespace: dr-prod
spec:
  protectionGroupRef:
    name: app-stack
    namespace: dr-prod
  cleanupPolicy: Manual
  retentionTime: 4h
  schedulingCheck: Fail
  verification:
    reachability:
      - type: http
        port: 8080
        endpoint: /healthz
        expectedStatus: 200
        timeout: 30s
        vmName: vm-app-01

Example 6 — Collecting a diagnostic bundle

Run oc adm must-gather on the TSR-Manager cluster. The gather binary inside the image collects logs, CRD state, and configuration from all clusters in the DR estate.

oc --context <tsr-manager-context> adm must-gather \
  --image=quay.io/triliovault/site-recovery-must-gather:<tag>

To narrow the collection to a shorter log window or a single zone:

# Last 2 hours of logs
oc --context <tsr-manager-context> adm must-gather \
  --image=quay.io/triliovault/site-recovery-must-gather:<tag> \
  -- /usr/bin/gather --since 2h

# Single zone only
oc --context <tsr-manager-context> adm must-gather \
  --image=quay.io/triliovault/site-recovery-must-gather:<tag> \
  -- /usr/bin/gather --zone prod

Replace <tag> with the pinned release tag matching your installed operator version.

Example 7 — Adjusting runtime log level without a pod restart

The logging field on the ProtectionZone propagates a new log level to all zone components within approximately 15 seconds:

oc --context <tsr-manager-context> -n dr-prod patch pz prod --type=merge \
  -p '{"spec":{"logging":{"level":"debug"}}}'

To set a per-component override (debug for node agents only, info for control planes):

oc --context <tsr-manager-context> -n dr-prod patch pz prod --type=merge \
  -p '{"spec":{"logging":{"level":"info","nodeAgentLevel":"debug"}}}'

The following issues are the most common during and after installation. Each entry shows the symptom, the most likely cause, and the remediation step.

ProtectionZone stays in Provisioning phase

Symptom: oc get pz prod shows Status: Provisioning for more than 5 minutes.

Likely causes and fixes:

1. Webhook TLS Secret is missing. The ProtectionZone controller cannot install the admission webhook without the serving certificate.

   Copy

   oc --context <tsr-manager-context> -n dr-prod get secret webhook-server-cert
   

   If absent, create it as described in Stage 3.3 of the installation steps.

2. kubeconfig Secret is unreachable or has wrong permissions. The controller cannot reach a workload cluster.

   Copy

   oc --context <tsr-manager-context> -n dr-prod describe pz prod
   

   Check status.clusters[].status and status.clusters[].message. If Unreachable, verify the Secret name and key in spec.clusters[].kubeconfigSecretRef and confirm the zone credential has the required permissions.

3. Namespace trilio-site-recovery-system does not exist on a workload cluster. The ProtectionZone reachability probe GETs this namespace by name. If the namespace is absent the zone cannot confirm the cluster.

   Copy

   oc --context <primary-context> get namespace trilio-site-recovery-system
   

   The namespace is created automatically by the workload chart install. If the zone cannot reach the cluster at all, this will not yet exist.

4. ProtectionZone controller logs show an error. Inspect the logs:

   Copy

   oc --context <tsr-manager-context> -n trilio-site-recovery-system \
     logs deployment/protection-zone-controller -c manager
   

DRBD node agent pods stuck in Init phase

Symptom: Node agent pods on a workload cluster show Init:0/1 or Init:Error.

Likely causes:

1. No compatible DRBD kernel module for the running kernel. The init container (drbd-module-loader) uses kABI matching to find a precompiled module. If no match exists for the node's kernel, the init container exits with an error.

   Copy

   oc --context <primary-context> -n trilio-site-recovery-system \
     logs <node-agent-pod> -c drbd-module-loader
   

   Verify the node's kernel version and that distributionName: RHEL-CoreOS-9 is set in the ProtectionZone spec. RHEL 10 kernels are not validated in Tech Preview.

2. A foreign DRBD module is already loaded with usermode_helper enabled. This is a hard failure by design. The node must be rebooted after removing whatever loaded the foreign module.

3. Workload nodes cannot pull the module-loader image from quay.io/triliovault. Verify egress access from workload nodes to quay.io/triliovault. The module-loader image has pull policy Always.

DRBD node agent pod is refused by SCC admission

Symptom: The node agent DaemonSet shows FailedCreate events. The ProtectionZone status may show SCCAdmissionDenied.

Cause: The dedicated agent ServiceAccount (<release>-site-recovery-workload-control-plane-agent) must be bound to the OpenShift privileged SCC. This binding is applied automatically by the workload chart when the zone credential holds the required RBAC grants.

Fix: Verify the zone credential was created with tsrctl zone-credential create and that the ClusterRole it produced includes bind on system:openshift:scc:privileged. Re-create the credential if necessary, update the Secret, and trigger a zone reconciliation by annotating the ProtectionZone.

oc --context <primary-context> -n trilio-site-recovery-system \
  get events --field-selector reason=FailedCreate

ProtectionRequest stuck in Validating phase

Symptom: oc get pr <name> shows Phase: Validating and does not advance.

Likely causes:

1. The VM is not running. The VM must be running to be protected — the controller captures the node assignment from the running VMI. Start the VM and re-apply the ProtectionRequest.

2. The VM's namespace is not a dr--prefixed namespace on the TSR-Manager cluster. DRBDResources are rejected outside a dr- prefixed namespace. Ensure the ProtectionRequest is created in dr-<zone-name>.

3. The webhook-server-cert Secret is missing on the relevant workload cluster. The workload admission webhook cannot start without it, causing ProtectionRequest admission to fail.

4. RoleConflict condition on the ProtectionRequest. Check the Ready condition:

   Copy

   oc --context <tsr-manager-context> -n dr-prod describe pr <name>
   

   A Ready=False condition with reason RoleConflict indicates a DRBD role conflict on the DRBDResource. Contact Trilio support with a diagnostic bundle.

oc adm drain does not complete on a workload node

Symptom: oc adm drain <node> hangs indefinitely on a workload cluster node.

Cause: A running protected VM is pinned to the node being drained. The workload admission webhook denies eviction of a running protected VM. Live migration is also denied for protected VMs.

Fix: To drain the node you must first unprotect the VM (which removes the pin), drain the node, then re-protect the VM from its new location. Re-protecting triggers a full initial DRBD resynchronisation.

Replication shows Degraded or Disconnected connection state

Symptom: oc get rgs shows Connection: Degraded or Connection: Disconnected.

Likely causes:

1. DRBD replication ports are blocked. Verify that TCP ports 7000–8000 inclusive are open between all worker nodes of the primary and DR clusters. Port 8000 is mandatory; a rule stopping at 7999 leaves one volume unable to replicate.

   Copy

   # From a primary cluster worker node, test connectivity to a DR worker node
   # on a port in the range, for example port 7000
   nc -zv <dr-worker-ip> 7000
   

2. Network latency exceeds 50 ms. Protocol C synchronous replication requires sub-50 ms round-trip latency between primary and DR worker nodes. Measure RTT and address network path issues.

3. The node agent on one side has not yet converged. The node agent applies a siterecovery.trilio.io/not-ready:NoSchedule taint to its own node until DRBD converges. Check agent pod status and logs:

   Copy

   # Find the agent pod on the affected node
   oc --context <primary-context> -n trilio-site-recovery-system \
     get pods --field-selector spec.nodeName=<node-name> \
     -l app.kubernetes.io/component=drbd-node-agent -o name
   
   # Then retrieve its logs
   oc --context <primary-context> -n trilio-site-recovery-system \
     logs <pod-name> -c agent
   

TSR web console is not accessible

Symptom: The TSR web console URL is not reachable or no Route exists.

Likely cause: The Route and OpenShift console link are created by the ProtectionZone controller after the zone reaches Available. If the zone is not yet Available, the Route does not exist.

Fix: Wait for oc get pz prod to show Status: Available, then check for the Route:

oc --context <tsr-manager-context> -n trilio-site-recovery-system get route

If the zone is Available but the Route is absent, check the ProtectionZone controller logs for errors related to Route or ConsoleLink creation.

Note: Browser access to the TSR web console requires OpenShift and the oauth-proxy Route. On upstream Kubernetes, no ingress is created and browser access is not a supported use case.