Configuration

This page describes every configuration option for Trilio Site Recovery for OpenStack, explains how those options are delivered to the service in a Juju charm deployment, and shows realistic examples of the most common configuration tasks. Trilio Site Recovery runs as two systemd services — protector-api and protector-engine — whose behaviour is entirely controlled by /etc/protector/protector.conf. In a Juju charm deployment that file is rendered by the charm from relation data and juju config values; you must never edit it directly because the charm overwrites manual edits on every config-changed event. All configuration changes must be applied with juju config trilio-site-recovery <hyphenated-key>=<value>.

Before configuring Trilio Site Recovery, ensure the following are in place:

• Juju charm deployment completed — the trilio-site-recovery-for-openstack charm must already be deployed with all mandatory relations satisfied (shared-db, amqp, identity-credentials). See the deployment guide for topology options.
• Juju 2.9 or 3.x — commands differ slightly between versions (juju run-action vs juju run).
• OpenStack ≥ Victoria on both sites, with Nova, Cinder, Keystone, and Neutron.
• Cinder volume types with replication enabled — every volume attached to a protected VM must use a type that has the extra-spec replication_enabled='<is> True' set.
• Pure Storage FlashArray (Purity//FA 6.0+ for ActiveDR; any supported version for async) with a replication array connection already established between the two arrays, or storage-driver=mock for lab/evaluation use.
• Keystone roles pre-created on both sites — dr_operator and dr_site_admin must exist before any cross-site operation is attempted; the service looks them up and does not create them.
• Network connectivity — TCP port 8788 must be reachable in both directions between sites; port 5000 (Keystone), 8774 (Nova), and 8776 (Cinder) must also be open.
• python-openstackclient installed via pip — the openstack dr CLI plugin (python3-triliositerecovery) is incompatible with the openstackclient snap.

In a Juju charm deployment, configuration is applied exclusively through juju config. There is no separate installation step for the configuration itself — the charm renders /etc/protector/protector.conf automatically once all mandatory relations are satisfied.

Step 1 — Verify the charm is active

juju status trilio-site-recovery

Wait until the unit reaches active/idle. If it is stuck in waiting, a mandatory relation is not yet satisfied.

Step 2 — Review current configuration

juju config trilio-site-recovery

This lists every option with its current value and whether it is at the default. Use this as a baseline before making changes.

Step 3 — Apply configuration

Set one or more options in a single command. Option names use hyphens, not underscores:

juju config trilio-site-recovery \
  storage-driver=pure \
  pure-primary-management-ip=10.0.1.10 \
  pure-primary-api-token=YOUR_PRIMARY_TOKEN \
  pure-secondary-management-ip=10.0.2.10 \
  pure-secondary-api-token=YOUR_SECONDARY_TOKEN

The charm detects the change, re-renders /etc/protector/protector.conf, and restarts the affected services automatically.

Step 4 — Confirm the rendered configuration (read-only)

# Juju 2.9
juju run-action trilio-site-recovery/0 show-config --wait

# Juju 3.x
juju run trilio-site-recovery/0 show-config

This prints the rendered protector.conf without exposing secret values.

Step 5 — Verify service health

# Juju 2.9
juju run-action trilio-site-recovery/0 health-check --wait

# Juju 3.x
juju run trilio-site-recovery/0 health-check

A healthy response confirms that both protector-api and protector-engine are running and the database is reachable.

Step 6 — Verify Keystone registration

Neither juju status nor health-check confirms that the service catalogue was populated. Run this explicitly:

openstack endpoint list | grep protector

Expect six rows (public, admin, internal for each of the two registered service types: disaster-recovery and protector). If the output is empty, re-run registration:

# Juju 2.9
juju run-action trilio-site-recovery/0 register-service --wait

# Juju 3.x
juju run trilio-site-recovery/0 register-service

Do not edit /etc/protector/protector.conf directly. The charm overwrites it on every config-changed event. Manual edits will be silently lost.

All options are delivered to the service via /etc/protector/protector.conf, which is rendered by the Juju charm from relation data and juju config values. The tables below show every supported configuration option grouped by INI section, with the compiled-in default, valid values, and the effect on service behaviour.

Juju charm note: Charm config option names use hyphens (e.g., pure-primary-management-ip), while the underlying protector.conf keys use underscores (e.g., primary_management_ip). Always use the hyphenated form with juju config. For options not exposed as first-class charm config, use config-flags with section:key=value syntax (colon separator, not dot).

[DEFAULT]

The [DEFAULT] section also carries the oslo.messaging broker URL (transport_url), which the charm renders automatically from the amqp relation. You should not set this manually; if you need to override it for testing purposes, use config-flags.

[api]

[database]

In Juju charm deployments, the database and database user (protector by default) are created automatically via the shared-db relation. No manual pre-creation or admin credentials are required.

[engine]

[storage]

Valid values:

[mock_storage]

Applies only when storage_driver = mock. Ignored in production.

[pure_storage]

Applies only when storage_driver = pure. All options in this section are set via juju config pure-* keys.

Retention options

Snapshot retention is controlled by three options passed through the replication strategy. Set these via config-flags if the defaults do not meet your requirements:

Multi-array configuration

To protect VMs whose volumes span multiple FlashArray pairs, list additional array section names in additional_arrays and create a [pure_array:<name>] subsection for each. Each subsection accepts the same options as [pure_storage] plus cinder_backend_names (a list of Cinder backend names that map to this array pair). Note that cinder_backend_names is only valid inside these subsections — setting it in [DEFAULT] has no effect.

Example via config-flags:

juju config trilio-site-recovery config-flags="pure_storage:additional_arrays=site-b,pure_array:site-b:primary_management_ip=10.0.3.10,pure_array:site-b:primary_api_token=TOKEN_B,pure_array:site-b:secondary_management_ip=10.0.4.10,pure_array:site-b:secondary_api_token=TOKEN_B2,pure_array:site-b:cinder_backend_names=ceph-backend-b"

[service_credentials]

This section holds the service account credentials used for cross-site tenant validation and Keystone trust creation. In a Juju charm deployment these are rendered automatically from the identity-credentials relation.

[keystone_authtoken]

Rendered automatically by the charm from the identity-credentials relation. You do not normally set these manually. The most relevant options for troubleshooting:

• www_authenticate_uri — public Keystone endpoint for token validation
• auth_url — admin Keystone endpoint
• auth_type = password
• service_token_roles_required = true
• cafile — set automatically if a certificates relation or ssl-ca config is present
• insecure = true — set automatically if verify-ssl = false

[drbd_storage]

Note: The DRBD storage driver is an in-development integration and is not a shipped or supported feature. The [drbd_storage] section is listed here for completeness because the oslo.config options are registered in the codebase, but you should not configure these options. Use storage_driver = pure or storage_driver = mock only.

config-flags — arbitrary overrides

The config-flags charm option lets you set any protector.conf key that is not exposed as a first-class charm option. Values are comma-separated section:key=value pairs. The separator between section and key is a colon, not a dot — a dot separator fails silently.

juju config trilio-site-recovery config-flags="engine:cross_site_timeout=60,engine:recovery_point_max_count=30"

These values are appended to the rendered protector.conf after the charm's own sections, so they override any value the charm sets for the same key.

API microversioning

The protector-api service supports microversions 1.0 through 1.1. The OpenStack-API-Version: protector <version> request header is optional — when omitted the service defaults to microversion 1.0 and processes the request normally. A malformed value or a version outside the 1.0–1.1 range returns HTTP 400 Bad Request.

OpenStack-API-Version: protector 1.1

Changing the storage driver

The most common post-deployment configuration change is switching from the mock driver (used in evaluation) to the Pure Storage FlashArray driver for production.

juju config trilio-site-recovery \
  storage-driver=pure \
  use-mock-storage=false \
  use-mock-cinder=false \
  pure-primary-management-ip=10.0.1.10 \
  pure-primary-api-token=YOUR_PRIMARY_TOKEN \
  pure-secondary-management-ip=10.0.2.10 \
  pure-secondary-api-token=YOUR_SECONDARY_TOKEN

The charm re-renders protector.conf and restarts both services. Verify the driver loaded correctly:

# Juju 3.x
juju run trilio-site-recovery/0 health-check

Configuring replication type

The replication type (async or active_dr) is set per protection group at creation time, not globally in protector.conf. Use the CLI:

# Async replication (periodic snapshots, RPO = snapshot interval)
openstack dr protection group create my-pg \
  --primary-site primary-site-id \
  --secondary-site secondary-site-id \
  --replication-type async

# ActiveDR (continuous async, RPO ≈ 0, requires Purity//FA 6.0+)
openstack dr protection group create my-pg \
  --primary-site primary-site-id \
  --secondary-site secondary-site-id \
  --replication-type active_dr

Global defaults that affect both replication types (RPO target, snapshot interval) are set in [pure_storage]:

juju config trilio-site-recovery \
  pure-default-rpo-seconds=600 \
  pure-snapshot-interval-seconds=120

Adjusting API workers

The default of 2 workers is conservative. On a production controller node with sufficient CPU, increase it:

juju config trilio-site-recovery api-workers=4

Enabling debug logging

juju config trilio-site-recovery debug=true

Logs are written to /var/log/protector/ and to stderr (captured by systemd), so you can follow them with:

journalctl -u protector-api -f
journalctl -u protector-engine -f

Disable debug logging before leaving it enabled in production — log volume increases significantly.

Configuring TLS for cross-site calls

If the remote site uses a private CA:

# Provide the CA bundle via the certificates relation (preferred)
juju integrate trilio-site-recovery:certificates vault:certificates

# Or set the CA file path via config-flags if you manage the file manually
juju config trilio-site-recovery config-flags="service_credentials:cafile=/etc/protector/ca.pem"

For lab environments with self-signed certificates only:

juju config trilio-site-recovery verify-ssl=false

This sets insecure = true in both [keystone_authtoken] and [service_credentials].

Setting cross-site timeout for slow links

The default 30-second timeout for cross-site API calls may be too short for high-latency inter-site links. Without a timeout, an unreachable peer can block a gunicorn worker indefinitely. Increase it via config-flags:

juju config trilio-site-recovery config-flags="engine:cross_site_timeout=120"

Configuring recovery point retention

For ActiveDR protection groups where scheduled recovery points are enabled:

juju config trilio-site-recovery \
  recovery-point-interval-minutes=15 \
  recovery-point-max-count=48 \
  recovery-point-ttl-hours=48

Preparing Cinder volume types for replication

Every volume attached to a protected VM must use a Cinder volume type with replication_enabled='<is> True'. Create the volume type before adding VMs to a protection group:

openstack volume type create replicated-volumes
openstack volume type set replicated-volumes \
  --property replication_enabled='<is> True'

# For async replication, also set:
openstack volume type set replicated-volumes \
  --property replication_type='<in> async'

Attempting to add a VM whose volumes use a non-replicated type returns HTTP 422 (VolumeTypeNotReplicated). You can pre-check eligibility without side effects:

openstack dr protection group member validate my-pg --instance INSTANCE_ID

Using extra hosts for cross-site name resolution

If the two sites use separate DNS resolvers and cannot resolve each other's Keystone hostnames, add static entries:

juju config trilio-site-recovery extra-hosts="
10.80.65.60 keystone.site-a.example.com
10.32.65.50 keystone.site-b.example.com"

Prefer fixing name resolution at the DNS layer (MAAS upstream forwarder) for a permanent solution; extra-hosts entries are static and go stale if addresses change.

Example 1 — Production deployment with Pure Storage async replication

Configure the Pure Storage driver for two sites with a 10-minute RPO target:

juju config trilio-site-recovery \
  storage-driver=pure \
  use-mock-storage=false \
  use-mock-cinder=false \
  pure-primary-management-ip=10.0.1.10 \
  pure-primary-api-token=T-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
  pure-secondary-management-ip=10.0.2.10 \
  pure-secondary-api-token=T-yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy \
  pure-default-rpo-seconds=600 \
  pure-snapshot-interval-seconds=120

Verify the configuration rendered correctly:

juju run trilio-site-recovery/0 show-config

Expected: the rendered protector.conf shows storage_driver = pure under [storage] and the FlashArray IPs under [pure_storage].

Example 2 — ActiveDR replication with scheduled recovery points

For near-zero RPO using Pure Storage ActiveDR (requires Purity//FA 6.0+):

juju config trilio-site-recovery \
  storage-driver=pure \
  use-mock-storage=false \
  use-mock-cinder=false \
  pure-primary-management-ip=10.0.1.10 \
  pure-primary-api-token=T-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
  pure-secondary-management-ip=10.0.2.10 \
  pure-secondary-api-token=T-yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy \
  recovery-point-interval-minutes=15 \
  recovery-point-max-count=48 \
  recovery-point-ttl-hours=48

Create a protection group using ActiveDR:

openstack dr protection group create prod-pg \
  --primary-site aaaaaaaa-0000-0000-0000-000000000001 \
  --secondary-site bbbbbbbb-0000-0000-0000-000000000002 \
  --replication-type active_dr

Expected output:

+--------------------+--------------------------------------+
| Field              | Value                                |
+--------------------+--------------------------------------+
| id                 | cccccccc-0000-0000-0000-000000000003 |
| name               | prod-pg                              |
| replication_type   | active_dr                            |
| status             | active                               |
+--------------------+--------------------------------------+

Example 3 — Version discovery

Verify the API is reachable and check the supported microversion range:

curl -s http://PROTECTOR_API_HOST:8788/

Expected response:

{
  "versions": [
    {
      "id": "v1.0",
      "status": "CURRENT",
      "version": "1.1",
      "min_version": "1.0",
      "links": [
        {
          "rel": "self",
          "href": "http://PROTECTOR_API_HOST:8788/v1/"
        }
      ]
    }
  ]
}

This endpoint requires no authentication.

Example 4 — Checking replication health via API

Obtain a Keystone token and query the replication health for a protection group:

TOKEN=$(openstack token issue -f value -c id)
TENANT_ID=$(openstack project show myproject -f value -c id)
PG_ID=cccccccc-0000-0000-0000-000000000003

curl -s \
  -H "X-Auth-Token: $TOKEN" \
  -H "OpenStack-API-Version: protector 1.1" \
  http://PROTECTOR_API_HOST:8788/v1/$TENANT_ID/protection-groups/$PG_ID/replication-health

Expected response structure:

{
  "replication_health": {
    "protection_group_id": "cccccccc-0000-0000-0000-000000000003",
    "protection_group_name": "prod-pg",
    "failover_validation": {
      "can_failover": true,
      "warnings": [],
      "blockers": [],
      "message": "Ready for failover"
    },
    "health_report": {
      "status": "healthy",
      "lag_seconds": 45,
      "rpo_compliant": true
    }
  }
}

Use replication_health.failover_validation.can_failover as the programmatic readiness check.

Example 5 — Triggering failover via API

TOKEN=$(openstack token issue -f value -c id)
TENANT_ID=$(openstack project show myproject -f value -c id)
PG_ID=cccccccc-0000-0000-0000-000000000003

curl -s -X POST \
  -H "X-Auth-Token: $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"failover": {"failover_type": "planned"}}' \
  http://PROTECTOR_API_HOST:8788/v1/$TENANT_ID/protection-groups/$PG_ID/action

Expected response (HTTP 202):

{
  "message": "Failover initiated",
  "status": "pending",
  "protection_group_id": "cccccccc-0000-0000-0000-000000000003",
  "operation_id": "dddddddd-0000-0000-0000-000000000004",
  "operation": { ... }
}

Poll the operation until it completes:

curl -s \
  -H "X-Auth-Token: $TOKEN" \
  http://PROTECTOR_API_HOST:8788/v1/$TENANT_ID/operations/dddddddd-0000-0000-0000-000000000004

Or use the CLI with --wait:

openstack dr failover cccccccc-0000-0000-0000-000000000003 \
  --failover-type planned \
  --wait

Example 6 — Arbitrary config override with config-flags

Increase the cross-site call timeout and enable auto-creation of security groups:

juju config trilio-site-recovery \
  config-flags="engine:cross_site_timeout=120,engine:auto_create_security_groups=true"

Verify the changes appear in the rendered configuration:

juju run trilio-site-recovery/0 show-config | grep -A2 '\[engine\]'

Expected:

[engine]
auto_create_security_groups = True
cross_site_timeout = 120

Example 7 — Pre-checking VM eligibility before adding to a protection group

openstack dr protection group member validate my-pg \
  --instance eeeeeeee-0000-0000-0000-000000000005

Expected response when eligible:

{"errors": []}

Expected response when a volume uses a non-replicated type (HTTP 200 with errors array):

{
  "errors": [
    {
      "type": "VolumeTypeNotReplicated",
      "message": "Volume type standard does not support replication"
    }
  ]
}

Fix: update the volume's type to one with replication_enabled='<is> True' before adding the VM.

Issue: protector.conf changes are not taking effect

Symptom: You edited /etc/protector/protector.conf directly and restarted the services, but after a few minutes the file reverts and your changes are gone.

Cause: The Juju charm renders protector.conf on every config-changed event, overwriting any manual edits.

Fix: Apply all configuration through juju config:

juju config trilio-site-recovery <hyphenated-key>=<value>

For options not exposed as first-class charm config, use config-flags with section:key=value syntax:

juju config trilio-site-recovery config-flags="engine:cross_site_timeout=120"

Issue: openstack endpoint list | grep protector returns no rows

Symptom: juju status shows the unit as active/idle and health-check passes, but no Keystone endpoints exist for the service.

Cause: The identity-credentials relation can fire before Keystone's haproxy starts listening (port 35357), causing the registration step to fail silently. The unit continues reporting active.

Fix: Re-run registration manually:

# Juju 2.9
juju run-action trilio-site-recovery/0 register-service --wait

# Juju 3.x
juju run trilio-site-recovery/0 register-service

Expect keystone catalogue updated in the output. Every openstack dr command fails until the endpoint is in the catalogue.

Issue: VolumeTypeNotReplicated (HTTP 422) when adding a VM to a protection group

Symptom: openstack dr protection group member add returns HTTP 422 with VolumeTypeNotReplicated.

Cause: One or more Cinder volumes attached to the instance use a volume type that does not have replication_enabled='<is> True' as an extra-spec.

Fix: Update the volume type (or re-type the volume before protecting it):

openstack volume type set <volume-type-name> \
  --property replication_enabled='<is> True'

Pre-check eligibility without side effects:

openstack dr protection group member validate <pg> --instance <instance-id>

Issue: InstanceNotCinderBoot (HTTP 422) when adding a VM

Symptom: Member add returns HTTP 422 with InstanceNotCinderBoot.

Cause: The Nova instance boots from ephemeral storage, not a Cinder volume. All protected VMs must have a Cinder-backed boot disk.

Fix: Rebuild or recreate the instance from a Cinder volume. Ephemeral-boot instances cannot be protected by Trilio Site Recovery.

Issue: Trust creation fails with Role <name> does not exist on site <site>

Symptom: The first cross-site operation (e.g., creating a tenant mapping or triggering a failover) fails with a role-not-found error.

Cause: One or more roles listed in default_trust_roles (member and dr_operator by default) do not exist on one or both sites. The service looks up roles and does not create them.

Fix: Create the missing roles on both sites:

# On each site
openstack role create dr_operator
openstack role create dr_site_admin

Then grant the roles to DR users and to the protector service account on each DR-enabled project.

Issue: Cross-site API calls time out or fail with TLS errors on MAAS

Symptom: Cross-site operations fail with connection errors or TLS certificate verification failures, even though intra-site operations work normally.

Cause (TLS): Asymmetric routing on a dual-homed host — the unit reaches Keystone by one path but Keystone replies on another, which presents as a broken TLS certificate. Network space bindings are mismatched between the two sites.

Cause (timeout): The default cross_site_timeout of 30 seconds is too short for the inter-site link latency.

Fix (TLS): Ensure Juju network space bindings are symmetric between sites. Re-deploy with explicit --bind flags if needed:

juju deploy trilio-site-recovery-for-openstack trilio-site-recovery \
  --base ubuntu@22.04 --to lxd:0 \
  --bind "oam-space identity-credentials=internal-space certificates=internal-space shared-db=internal-space"

For self-signed certificates without a shared CA, set:

juju config trilio-site-recovery verify-ssl=false

Fix (timeout):

juju config trilio-site-recovery config-flags="engine:cross_site_timeout=120"

Issue: mysql-router install hook fails with a MarkupSafe or tomli wheel error

Symptom: During deployment, the mysql-router subordinate's install hook fails with:

ERROR: Could not find a version that satisfies the requirement MarkupSafe==2.1.1

or

ERROR: Could not find a version that satisfies the requirement tomli

Cause: You deployed mysql-router at revision 1178, which ships cp310 wheels incompatible with Python 3.12 (Noble). On Jammy, --base ubuntu@22.04 was not specified, so the container inherited the Noble host's series.

Fix: Re-deploy specifying the correct revision and base:

juju deploy mysql-router trilio-site-recovery-mysql-router \
  --channel 8.0/stable --revision 1154

juju deploy trilio-site-recovery-for-openstack trilio-site-recovery \
  --base ubuntu@22.04 --to lxd:0

Issue: openstack dr commands not found

Symptom: Running any openstack dr command returns openstack: 'dr' is not an openstack command.

Cause: The triliositerecovery CLI plugin is not installed, or you are using the openstackclient snap, which is incompatible with the plugin.

Fix: Install python-openstackclient and the CLI plugin via pip (not snap):

pip install python-openstackclient
pip install -e ./client   # from the repository root, or:
apt-get install python3-triliositerecovery

If using the snap, remove it first:

snap remove openstackclient

Issue: Tenant mapping creates locally but fails to sync to the remote site

Symptom: openstack dr tenant mapping create succeeds, but the mapping does not appear on the remote site and subsequent cross-site operations fail with validation errors.

Cause: TCP port 8788 is not open from the remote site back to this site. Cross-site metadata synchronisation requires bidirectional connectivity on port 8788 between both protector-api services.

Fix: Open port 8788 in both directions between the two sites at the network/firewall layer, then re-sync:

openstack dr metadata sync <protection-group-id>

Issue: config-flags changes appear to have no effect

Symptom: You set config-flags with values like engine.cross_site_timeout=120 (dot separator) but the rendered protector.conf does not reflect the change.

Cause: The config-flags option requires a colon separator between section and key, not a dot. A dot separator fails silently with no error.

Fix: Re-set with the correct colon separator:

juju config trilio-site-recovery config-flags="engine:cross_site_timeout=120"