RHOSO18

This page explains how to deploy Trilio Site Recovery for OpenStack on Red Hat OpenStack Services on OpenShift 18 (RHOSO18). The service runs as two Kubernetes Deployments — trilio-site-recovery-api and trilio-site-recovery-engine — inside a dedicated trilio-site-recovery namespace, sharing the Galera cluster and RabbitMQ cluster already present in the trilio-openstack namespace. Because this deployment path targets an internal development environment and is not the validated customer-facing method, you should treat these instructions as a technical reference for contributors and evaluators rather than a production runbook. The Juju charm deployment remains the only validated, customer-facing installation method for the tech preview.

Before you begin, confirm that the following are in place on every site where you intend to run the service.

OpenShift and OpenStack

• Red Hat OpenShift Container Platform with RHOSO18 installed and healthy
• Two independent OpenStack deployments ≥ Victoria (one primary, one secondary), each with Nova, Cinder, Keystone, and Neutron
• Cinder configured with a volume type that has replication_enabled='<is> True' as an extra-spec, backed by Pure Storage FlashArray
• Pure Storage FlashArray arrays with an async replication connection between sites (Purity//FA 6.0+ required for ActiveDR; any supported Purity//FA version for async snapshot replication)

Shared infrastructure (already deployed in trilio-openstack namespace)

• trilio-galera-cluster — MariaDB ≥ 10.3 Galera cluster; the install script creates the protector database and user automatically
• trilio-rabbitmq-cluster — RabbitMQ cluster with management API enabled on port 15671 (TLS)
• combined-ca-bundle Secret in the trilio-openstack namespace containing tls-ca-bundle.pem
• trilio-openstack-secret Secret in the trilio-openstack namespace (shared with other Trilio products; the install script adds SrDatabasePassword, SrKeystonePassword, and SrRabbitPassword keys)
• osp-secret in the openstack namespace containing AdminPassword

Tooling

• oc CLI installed and logged in (oc whoami must succeed)
• podman installed if you need to build the container image locally
• Network access from the trilio-site-recovery namespace to:
  • keystone-public.openstack.svc:5000 (Keystone)
  • trilio-galera-cluster.trilio-openstack.svc:3306 (MariaDB)
  • trilio-rabbitmq-cluster.trilio-openstack.svc:5671 (AMQP) and :15671 (management API)
  • Port 8788 on the remote site's protector-api (bidirectional — one-way connectivity causes metadata sync failures)

Keystone roles

• The Keystone roles dr_operator and dr_site_admin must exist on both sites before you run any DR operation. The service looks them up and grants them; it does not create them. Missing roles cause trust creation to fail with a message naming the absent role.

CLI (optional, on any workstation)

• python-openstackclient installed via pip (not the snap — the snap is incompatible with the CLI plugin)
• pip install triliositerecovery to install the openstack dr plugin separately from the server packages

The install script at deploy/openshift/scripts/install.sh is idempotent — you can re-run it safely. It performs every bootstrap step in order: namespace, RBAC, CA bundle, credential provisioning, init jobs (database, RabbitMQ, Keystone), schema migration, service/route, and deployment rollout.

Step 1 — Clone the repository and navigate to the scripts directory

git clone <repository-url>
cd deploy/openshift/scripts

Step 2 — Log in to OpenShift

oc login --token=<token> --server=https://<api-server>:6443
oc whoami   # must succeed before continuing

Step 3 — Run the install script with default settings

The defaults assume the Trilio shared infrastructure lives in trilio-openstack and the OpenStack control plane lives in openstack. Adjust with flags if your environment differs.

bash install.sh

To override any of the defaults:

bash install.sh \
  --namespace trilio-site-recovery \
  --trilio-ns trilio-openstack \
  --openstack-ns openstack

To use a custom image (for example, a build from your own registry):

bash install.sh --image registry.example.com/trilio/trilio-site-recovery-openstack:1.0.0

The script performs the following steps automatically; you do not need to run any of them manually:

1. Creates the trilio-site-recovery namespace from manifests/namespace.yaml.
2. Creates the ServiceAccount and RBAC from manifests/serviceaccount.yaml.
3. Copies tls-ca-bundle.pem from combined-ca-bundle in trilio-openstack into the trilio-site-recovery-ca-bundle Secret in the service namespace.
4. Reads admin credentials from trilio-galera-cluster-mariadb-root-db-secret, trilio-rabbitmq-cluster-default-user, and osp-secret.
5. Generates or reuses Sr* passwords (SrDatabasePassword, SrKeystonePassword, SrRabbitPassword) in trilio-openstack-secret. Use --rotate-passwords to force regeneration.
6. Creates the transient trilio-site-recovery-init-creds Secret used by the init Jobs, then deletes it at the end.
7. Applies the ConfigMap (manifests/configmap.yaml) with the base protector.conf and api-paste.ini.
8. Renders and applies trilio-site-recovery-secrets with live Sr* passwords substituted into protector-secrets.conf.
9. Runs the trilio-site-recovery-db-init Job — creates the protector database and user on the Galera cluster using the Galera root credentials.
10. Runs the trilio-site-recovery-rabbitmq-init Job — creates the protector RabbitMQ user and grants permissions on the default vhost.
11. Runs the trilio-site-recovery-keystone-init Job — creates or updates the protector Keystone user, ensures dr_operator and dr_site_admin roles exist, registers the disaster-recovery service, and upserts the public/internal/admin endpoints.
12. Runs the trilio-site-recovery-db-sync Job — migrates the database schema using protector-manage db sync (the canonical form of the schema migration command).
13. Applies the Service and Route from manifests/service.yaml — exposes port 8788 via a ClusterIP Service and an edge-terminated OpenShift Route.
14. Deploys the API (manifests/deployment-api.yaml) — 2 replicas of the api container listening on port 8788.
15. Deploys the Engine (manifests/deployment-engine.yaml) — 1 replica (singleton; multiple engine replicas cause duplicate RPC handling).
16. Waits for rollouts to complete (oc rollout status).
17. Deletes the transient trilio-site-recovery-init-creds Secret.

Step 4 — Verify the installation

# Check that both pods are Running
oc get pods -n trilio-site-recovery -l app.kubernetes.io/name=trilio-site-recovery

# Get the public Route URL
ROUTE=$(oc get route trilio-site-recovery-api \
  -n trilio-site-recovery \
  -o jsonpath='{.spec.host}')
echo "API URL: https://${ROUTE}"

# Version discovery — no authentication required
curl -kSs "https://${ROUTE}/" | python3 -m json.tool

A healthy response looks like:

{
  "versions": [
    {
      "id": "v1.0",
      "status": "CURRENT",
      "version": "1.1",
      "min_version": "1.0",
      "links": [
        {
          "rel": "self",
          "href": "https://<route-host>/v1/"
        }
      ]
    }
  ]
}

Building the container image locally

If you need to build and push your own image:

cd deploy/openshift/scripts
bash build.sh --registry registry.example.com/trilio --tag 1.0.0 --push

Then pass --image registry.example.com/trilio/trilio-site-recovery-openstack:1.0.0 to install.sh.

Uninstalling

To remove in-namespace resources while leaving the database, RabbitMQ user, Keystone artefacts, and Sr* secret keys intact:

bash uninstall.sh

To also drop the protector database and user on Galera, delete the RabbitMQ user, remove the Keystone service and endpoints, and strip the Sr* keys from trilio-openstack-secret:

bash uninstall.sh --purge

To additionally delete the namespace itself:

bash uninstall.sh --purge --delete-namespace

Configuration is split into two Kubernetes objects so that secrets never appear in the ConfigMap:

• trilio-site-recovery-config ConfigMap — base protector.conf containing non-sensitive options, plus api-paste.ini and policy.yaml.
• trilio-site-recovery-secrets Secret — protector-secrets.conf containing the database connection string, RabbitMQ transport URL, Keystone credentials, and Pure Storage tokens. This file is applied on top of the base config at startup via a second --config-file argument, so its values override any matching key in the ConfigMap.

Both are projected into the container at /etc/trilio-site-recovery.

Important: Do not edit these objects directly in a running cluster and then re-run the install script — the script will overwrite your changes. Make all persistent configuration changes in the manifests before running install.sh, or patch the objects with oc patch / oc edit if you are making a one-off adjustment to a running deployment.

Base configuration (ConfigMap — protector.conf)

[DEFAULT]

[oslo_messaging_rabbit]

[api]

[engine]

[keystone_authtoken]

[service_credentials]

[pure_storage]

[mock_storage]

Secrets overlay (protector-secrets.conf)

The install script renders manifests/secret.yaml with live Sr* passwords and applies it. The resulting Secret contains a single key protector-secrets.conf that overrides the ConfigMap at startup. The sensitive sections are:

[DEFAULT]

transport_url = rabbit://protector:<SrRabbitPassword>@trilio-rabbitmq-cluster.trilio-openstack.svc:5671//

[database]

connection = mysql+pymysql://protector:<SrDatabasePassword>@trilio-galera-cluster.trilio-openstack.svc:3306/protector?ssl_ca=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

[keystone_authtoken]

auth_url = https://keystone-public.openstack.svc:5000
www_authenticate_uri = https://keystone-public-openstack.apps.<cluster-domain>
cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
username = protector
password = <SrKeystonePassword>

[service_credentials]

auth_url = https://keystone-public.openstack.svc:5000
cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
username = protector
password = <SrKeystonePassword>
project_name = service
user_domain_name = Default
project_domain_name = Default

[pure_storage] — production values (uncomment when ready)

# primary_management_ip = 10.21.67.50
# primary_api_token = <token>
# primary_site_name = cluster1
# secondary_management_ip = 10.21.67.60
# secondary_api_token = <token>
# secondary_site_name = cluster2

When you are ready to connect real Pure Storage arrays, edit the Secret and also flip use_mock_storage = false and use_mock_cinder = false in the ConfigMap, then update [storage] with storage_driver = pure:

oc edit configmap trilio-site-recovery-config -n trilio-site-recovery
# Change use_mock_storage = false, use_mock_cinder = false
# Add:
# [storage]
# storage_driver = pure

oc edit secret trilio-site-recovery-secrets -n trilio-site-recovery
# Populate [pure_storage] with real IPs and API tokens (base64-encode the file)

# Restart both workloads to pick up the new config
oc rollout restart deployment/trilio-site-recovery-api -n trilio-site-recovery
oc rollout restart deployment/trilio-site-recovery-engine -n trilio-site-recovery

Policy

The policy.yaml shipped in the ConfigMap grants all protection-group and member operations to project members (rule:default = is_admin:True or project_id:%(project_id)s). Site-level operations (sites:action, sites:failover) and site operation listing require role:dr_site_admin or role:admin. Adjust this file to tighten or loosen access control for your environment.

Once the service is running and endpoints are registered in Keystone, you interact with it via the openstack dr CLI plugin or directly through the REST API.

Installing the CLI plugin

The CLI plugin is a separate Python package. Install it on any workstation that has python-openstackclient available (install via pip, not the snap):

pip install triliositerecovery

Verify the plugin is loaded:

openstack dr site list

Authenticating

All API calls require an X-Auth-Token obtained from Keystone on port 5000. The CLI handles this automatically when your OS_* environment variables are set. For direct API calls:

export OS_AUTH_URL=https://keystone-public-openstack.apps.<cluster-domain>
export OS_USERNAME=<your-username>
export OS_PASSWORD=<your-password>
export OS_PROJECT_NAME=<your-project>
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_IDENTITY_API_VERSION=3

TOKEN=$(openstack token issue -f value -c id)

Registering sites

Register both your primary and secondary OpenStack sites. The dr site create command requires --site-type, --auth-url, and additional mandatory arguments — run openstack dr site create --help for the full list before proceeding.

openstack dr site create primary-site \
  --site-type PRIMARY \
  --auth-url https://keystone-primary.example.com:5000 \
  --service-username protector \
  --service-password <password> \
  --service-project-name service \
  --service-user-domain-name Default \
  --service-project-domain-name Default

openstack dr site create secondary-site \
  --site-type SECONDARY \
  --auth-url https://keystone-secondary.example.com:5000 \
  --service-username protector \
  --service-password <password> \
  --service-project-name service \
  --service-user-domain-name Default \
  --service-project-domain-name Default

Creating a tenant mapping

Because each site has an independent Keystone, the same tenant has different UUIDs on each site. Register that mapping:

openstack dr tenant mapping create \
  --local-tenant <primary-project-id> \
  --remote-site <secondary-site-id> \
  --remote-tenant <secondary-project-id>

Creating a Protection Group

A Protection Group names the collection of VMs you want to replicate and fail over together:

openstack dr protection group create my-app-pg \
  --primary-site <primary-site-id> \
  --secondary-site <secondary-site-id> \
  --replication-type async

Adding VMs to a Protection Group

Every VM must boot from a Cinder volume (not ephemeral storage), and all attached volumes must use a volume type with replication_enabled='<is> True'. Validate eligibility first to avoid side effects:

# Pre-check — returns HTTP 200 with an 'errors' array; no side effects
openstack dr protection group member validate my-app-pg \
  --instance <nova-instance-id>

# Add if validation passes
openstack dr protection group member add my-app-pg \
  --instance <nova-instance-id> \
  --boot-order 1

VMs with the same --boot-order value are started and stopped in parallel during failover; lower-numbered tiers complete before higher-numbered tiers begin.

Running a test failover (non-disruptive DR validation)

A test failover clones replicated volumes and boots isolated test VMs on a separate network without touching production:

openstack dr test failover <pg-id> --auto-network

You must supply either --auto-network or --network-mapping; the command is rejected without one of them.

Clean up test resources when done:

openstack dr test failover cleanup <pg-id>

To target a specific test failover operation rather than the most recent one:

openstack dr test failover cleanup <pg-id> --operation <operation-id>

Executing a planned failover

openstack dr failover <pg-id> --failover-type planned --wait

--failover-type planned gracefully shuts down VMs and takes a final snapshot before promoting the replica — zero data loss. Use --failover-type unplanned when the primary site is unavailable and you accept data loss up to the RPO.

Monitoring replication health

openstack dr replication health <pg-id>

Check replication_health.failover_validation.can_failover programmatically to gate automated failover workflows:

curl -s \
  -H "X-Auth-Token: ${TOKEN}" \
  "https://${ROUTE}/v1/${OS_PROJECT_ID}/protection-groups/<pg-id>/replication-health" \
  | python3 -m json.tool

Tracking operations

All long-running DR operations (failover, failback, test failover, sync) return an operation_id immediately with HTTP 202. Poll the operation until it reaches 100% progress:

openstack dr operation show <operation-id>

# Or via the API
curl -s \
  -H "X-Auth-Token: ${TOKEN}" \
  "https://${ROUTE}/v1/${OS_PROJECT_ID}/operations/<operation-id>" \
  | python3 -m json.tool

To cancel an in-progress operation:

openstack dr operation cancel <operation-id>

Example 1 — Verify service health (no authentication required)

The /healthz endpoint returns {"status": "ok"} when the service is up and the database is reachable.

ROUTE=$(oc get route trilio-site-recovery-api \
  -n trilio-site-recovery \
  -o jsonpath='{.spec.host}')

curl -kSs "https://${ROUTE}/healthz"

Expected output:

{"status": "ok", "checks": {"database": "ok"}}

If the database is unreachable the status code is 503:

{"status": "unhealthy", "checks": {"database": "OperationalError"}}

Example 2 — Version discovery (no authentication required)

curl -kSs "https://${ROUTE}/" | python3 -m json.tool

Expected output:

{
  "versions": [
    {
      "id": "v1.0",
      "status": "CURRENT",
      "version": "1.1",
      "min_version": "1.0",
      "links": [
        {
          "rel": "self",
          "href": "https://<route-host>/v1/"
        }
      ]
    }
  ]
}

Example 3 — List Protection Groups via the REST API

TOKEN=$(openstack token issue -f value -c id)
PROJECT_ID=$(openstack token issue -f value -c project_id)

curl -kSs \
  -H "X-Auth-Token: ${TOKEN}" \
  -H "OpenStack-API-Version: protector 1.1" \
  "https://${ROUTE}/v1/${PROJECT_ID}/protection-groups" \
  | python3 -m json.tool

The OpenStack-API-Version header is optional. When omitted the service defaults to microversion 1.0 and processes the request normally.

Expected output (empty project):

{"protection_groups": []}

Example 4 — Trigger a planned failover and poll to completion

PG_ID="<protection-group-id>"

# Trigger the failover
RESPONSE=$(curl -ksSf \
  -X POST \
  -H "X-Auth-Token: ${TOKEN}" \
  -H "Content-Type: application/json" \
  "https://${ROUTE}/v1/${PROJECT_ID}/protection-groups/${PG_ID}/action" \
  -d '{"failover": {"failover_type": "planned"}}')

echo "$RESPONSE" | python3 -m json.tool
OP_ID=$(echo "$RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['operation_id'])")

Initial response (HTTP 202):

{
  "message": "Failover initiated",
  "status": "pending",
  "protection_group_id": "<pg-id>",
  "operation_id": "<op-id>",
  "operation": { ... }
}

The initial status is pending, not running. Poll until progress reaches 100:

while true; do
  STATUS=$(curl -ksSf \
    -H "X-Auth-Token: ${TOKEN}" \
    "https://${ROUTE}/v1/${PROJECT_ID}/operations/${OP_ID}" \
    | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['status'], d.get('progress',0))")
  echo "$(date): ${STATUS}"
  [[ "$STATUS" == completed* || "$STATUS" == failed* ]] && break
  sleep 10
done

Alternatively, pass --wait to the CLI and it blocks automatically:

openstack dr failover "${PG_ID}" --failover-type planned --wait

Example 5 — Pre-validate a member before adding it

Use the validate endpoint to check eligibility without creating anything:

curl -ksSf \
  -X POST \
  -H "X-Auth-Token: ${TOKEN}" \
  -H "Content-Type: application/json" \
  "https://${ROUTE}/v1/${PROJECT_ID}/protection-groups/${PG_ID}/members/validate" \
  -d '{"instance_id": "<nova-instance-id>"}' \
  | python3 -m json.tool

Expected output when eligible (HTTP 200):

{"errors": []}

Output when ineligible (HTTP 200, errors populated):

{
  "errors": [
    {
      "type": "InstanceNotCinderBoot",
      "message": "Instance <id> is not a Cinder boot instance. All protected VMs must boot from a Cinder volume."
    }
  ]
}

Example 6 — Check replication health programmatically

HEALTH=$(curl -ksSf \
  -H "X-Auth-Token: ${TOKEN}" \
  "https://${ROUTE}/v1/${PROJECT_ID}/protection-groups/${PG_ID}/replication-health")

echo "$HEALTH" | python3 -m json.tool

# Extract the failover readiness flag
CAN_FAILOVER=$(echo "$HEALTH" | python3 -c \
  "import sys,json; d=json.load(sys.stdin); \
   print(d['replication_health']['failover_validation']['can_failover'])")

echo "Can failover: ${CAN_FAILOVER}"

Example 7 — Rotate Sr* passwords

To regenerate all Sr* passwords (database, Keystone, RabbitMQ) without a full reinstall:

bash install.sh --rotate-passwords

The script generates new random passwords, patches trilio-openstack-secret, re-renders trilio-site-recovery-secrets, and re-runs the init jobs. Both deployments are then restarted automatically as part of the rollout steps.

Use a consistent diagnostic workflow: check pod status → inspect logs → verify the specific component or secret involved.

# Pod overview
oc get pods -n trilio-site-recovery

# API logs (last 100 lines)
oc logs -n trilio-site-recovery \
  -l app.kubernetes.io/component=api --tail=100

# Engine logs
oc logs -n trilio-site-recovery \
  -l app.kubernetes.io/component=engine --tail=100

# Init job logs (replace <component> with db-init, keystone-init, etc.)
oc logs -n trilio-site-recovery \
  -l app.kubernetes.io/component=<component> --tail=200

Issue: Install script fails with "combined-ca-bundle not found"

Symptom: install.sh exits with combined-ca-bundle not found in trilio-openstack.

Cause: The shared trilio-openstack namespace does not contain the expected CA bundle Secret, or the --trilio-ns flag points to the wrong namespace.

Fix:

1. Confirm the Secret exists: oc get secret combined-ca-bundle -n trilio-openstack
2. If you used a non-default namespace for shared infra, pass the correct value: bash install.sh --trilio-ns <your-trilio-ns>

Issue: trilio-site-recovery-db-init Job fails

Symptom: Job enters Failed state; logs show Access denied for user 'root' or a connection refused error.

Cause: Either the Galera root password Secret key is wrong, or the Galera service is not reachable from the trilio-site-recovery namespace.

Fix:

1. Verify the Secret key: oc get secret trilio-galera-cluster-mariadb-root-db-secret -n trilio-openstack -o jsonpath='{.data.DatabasePassword}' | base64 -d
2. Check network policy: from a debug pod in trilio-site-recovery, test nc -zv trilio-galera-cluster.trilio-openstack.svc 3306
3. Re-run the install script after fixing connectivity — it is idempotent.

Issue: trilio-site-recovery-keystone-init Job fails with "Role dr_operator does not exist"

Symptom: The keystone-init Job log shows Role dr_operator does not exist on site <site> and exits non-zero.

Cause: The Keystone init job creates the roles if they are absent on the local site (see the for R in dr_operator dr_site_admin loop in job-keystone-init.yaml). However, if the remote site does not have the roles, trust creation fails during the first cross-site DR operation.

Fix: Create both roles on the remote site using an admin account on that site:

openstack role create dr_operator
openstack role create dr_site_admin

Issue: API pod starts but returns HTTP 500 on all requests

Symptom: curl https://<route>/ returns a version document, but authenticated calls to /v1/<project>/protection-groups return 500.

Cause: Most commonly a database connectivity failure — the [database] connection string in the secrets overlay is incorrect, or the schema has not been migrated.

Fix:

1. Check API logs for OperationalError or Can't connect to MySQL.
2. Verify the database connection string in the Secret: oc get secret trilio-site-recovery-secrets -n trilio-site-recovery -o jsonpath='{.data.protector-secrets\.conf}' | base64 -d
3. If the db-sync Job failed, check its logs and re-run the install script to retry it.

Issue: Engine pod is in CrashLoopBackOff

Symptom: oc get pods shows trilio-site-recovery-engine-<hash> with status CrashLoopBackOff.

Cause: Usually a misconfigured transport_url — the engine cannot connect to RabbitMQ — or a missing [storage] section when use_mock_storage = false.

Fix:

1. oc logs -n trilio-site-recovery -l app.kubernetes.io/component=engine --previous --tail=50
2. Look for AMQP server ... is unreachable → verify the transport_url in trilio-site-recovery-secrets.
3. Look for StorageDriverConfigError → confirm storage_driver = pure and all [pure_storage] options are populated in the Secret.
4. After editing the Secret, restart the engine: oc rollout restart deployment/trilio-site-recovery-engine -n trilio-site-recovery

Issue: openstack dr site create fails with "Authentication failed"

Symptom: SiteAuthenticationFailed (HTTP 401) when registering a remote site.

Cause: The --service-username / --service-password credentials are wrong, or the protector user does not exist on the remote site's Keystone.

Fix:

1. Confirm the user exists on the remote site: openstack user show protector (authenticated against the remote Keystone).
2. Re-run the keystone-init Job on the remote site, or create the user manually and set the password.
3. Retry openstack dr site set <site> --service-password <correct-password>.

Issue: Member add fails with HTTP 422 InstanceNotCinderBoot

Symptom: openstack dr protection group member add returns InstanceNotCinderBoot.

Cause: The Nova instance uses ephemeral (local) storage for its boot disk rather than a Cinder volume.

Fix: You cannot protect this instance without migrating it to a Cinder-backed boot volume. Use openstack server rebuild or create a new instance from a Cinder volume, then add it to the Protection Group.

Issue: Member add fails with HTTP 422 VolumeTypeNotReplicated

Symptom: openstack dr protection group member add returns VolumeTypeNotReplicated.

Cause: One or more Cinder volumes attached to the instance use a volume type that lacks replication_enabled='<is> True' as an extra-spec.

Fix:

1. Identify the volume type: openstack volume show <volume-id> -f value -c volume_type
2. Check its extra-specs: openstack volume type show <type> -f json
3. Either set the extra-spec on the existing type (admin operation): openstack volume type set --property replication_enabled='<is> True' <type> — or retype the volume to a replicated type before adding the instance.

Issue: Cross-site metadata sync fails / tenant mapping does not propagate

Symptom: openstack dr tenant mapping create succeeds locally but operations on the remote site report TenantMappingNotFound.

Cause: Port 8788 connectivity between sites is one-directional or blocked. Cross-site metadata sync requires bidirectional connectivity on port 8788.

Fix:

1. From the engine pod on site A, test reachability to site B: curl -ksSf https://<site-b-protector-api>:8788/ should return the version document.
2. Repeat the test in the opposite direction from site B to site A.
3. Update network policies, firewall rules, or OpenShift Routes to allow traffic in both directions.
4. Force a re-sync after restoring connectivity: openstack dr metadata sync <pg-id>

Issue: openstack dr commands not found after pip install triliositerecovery

Symptom: openstack dr site list returns openstack: 'dr' is not an openstack command.

Cause: Either the openstackclient snap is in use (incompatible with the plugin), or the plugin was installed into a different Python environment.

Fix:

1. Confirm you are using pip-installed python-openstackclient, not the snap: which openstack — if the path contains snap, remove it and reinstall via pip.
2. Ensure the plugin is installed in the same environment: pip show triliositerecovery
3. Verify the entry point is registered: openstack --help | grep dr