Pure Replication Technologies

Trilio Site Recovery for OpenStack supports two Pure Storage FlashArray replication technologies: async (periodic Protection Group snapshot replication) and ActiveDR (continuous pod replica-link replication). Understanding which technology is in use — and how each behaves during normal operation, failover, and failback — is essential for configuring Protection Groups correctly, interpreting replication health reports, and setting realistic RPO/RTO targets. This page explains how each technology works at the storage layer, how the service interacts with Pure Storage arrays, and which configuration options control the behavior of each mode.

Before reading this guide, ensure you have:

• Two Pure Storage FlashArray arrays, one at each OpenStack site, with a replication array connection already established between them (configured via the Purity GUI or API)
• Async replication: any supported Purity//FA version with Protection Group replication licensed
• ActiveDR replication: Purity//FA 6.0 or later on both arrays (required for pod replica links)
• The py-pure-client Python SDK installed on the node running protector-engine (Debian package: python3-pure-client)
• The Pure Storage driver activated in protector.conf by setting storage_driver = pure in the [storage] section (the compiled-in default is mock, which is for development only)
• API tokens for both the primary and secondary FlashArray management interfaces
• Network reachability from the protector-engine host to both FlashArray management IPs
• dr_operator and dr_site_admin Keystone roles pre-created on both OpenStack sites

The Pure Storage driver ships as part of the trilio-site-recovery Debian package and is activated entirely through configuration — no separate installation step is needed once the charm is deployed.

Step 1 — Install the Pure Storage SDK

The py-pure-client SDK is packaged as python3-pure-client and must be present on every node running protector-engine. In a Juju charm deployment the charm installs it automatically from the configured APT source. Verify it is present:

dpkg -l python3-pure-client

If it is missing, install it from the Trilio APT repository:

apt-get install python3-pure-client

Step 2 — Set the storage driver via Juju config

In a Juju charm deployment you must never edit /etc/protector/protector.conf directly — the charm overwrites it on every config-changed event. Use juju config instead:

juju config trilio-site-recovery storage-driver=pure

Step 3 — Provide FlashArray connection details

Set the management IPs and API tokens for both arrays. Option names use hyphens when passed via juju config:

juju config trilio-site-recovery \
  pure-primary-management-ip=<primary-array-ip> \
  pure-primary-api-token=<primary-api-token> \
  pure-secondary-management-ip=<secondary-array-ip> \
  pure-secondary-api-token=<secondary-api-token>

Step 4 — Verify the driver initialised

After the charm applies the new configuration, check the engine log to confirm the driver connected to both arrays:

tail -f /var/log/protector/protector-engine.log | grep -E 'PureStorageDriver|initialized'

A successful startup produces a line similar to:

INFO protector.engine.storage.drivers.pure_driver PureStorageDriver initialized successfully

All Pure Storage driver settings live in the [pure_storage] section of /etc/protector/protector.conf. In Juju charm deployments, set these via juju config trilio-site-recovery <hyphenated-key>=<value>; the charm renders the file from config values and relation data.

[storage] section

[pure_storage] section

Retention options (async replication)

These control how long replicated snapshots (recovery points) are kept on the secondary array:

ActiveDR-specific database columns

When replication_type is active_dr, two additional fields are stored per Protection Group policy:

• activedr_pod_name — the name of the pod pair used for this Protection Group (derived from the Protection Group name)
• activedr_remote_connection — the name of the replication array connection on the primary FlashArray used to establish the pod replica link

These are populated automatically by the engine when the Protection Group is created; you do not set them directly.

Example protector.conf snippet (non-Juju)

[storage]
storage_driver = pure

[pure_storage]
primary_management_ip = 10.21.67.50
primary_api_token = <primary-token>
secondary_management_ip = 10.21.67.60
secondary_api_token = <secondary-token>
primary_site_name = site-a
secondary_site_name = site-b
verify_ssl = false
default_rpo_seconds = 900
snapshot_interval_seconds = 300
retention_all_for_hours = 24
retention_per_day = 24
retention_days = 7

Note: In Juju charm deployments, do not edit this file directly. Use juju config trilio-site-recovery <key>=<value> with hyphenated option names.

Choosing a replication type

When you create a Protection Group you select the storage replication technology with the --replication-type flag:

openstack dr protection group create my-pg \
  --primary-site site-a \
  --secondary-site site-b \
  --replication-type async          # or active_dr

The two supported values map directly to Pure Storage capabilities:

How async replication works

When you create a Protection Group with --replication-type async, the engine creates a Pure Storage Protection Group on the primary array, sets the secondary array as its replication target, and configures the snapshot schedule and retention policy from your [pure_storage] settings. As you add VMs to the Protection Group, their Cinder volumes are added to the Protection Group on the array.

At each snapshot interval, Pure Storage automatically takes a crash-consistent snapshot of all volumes in the Protection Group and replicates it to the secondary array. These replicated snapshots become the recovery points you can select during failover or test failover.

To check how many recovery points are available and when the most recent one was taken:

openstack dr recovery point list <protection-group-id>

To create an on-demand recovery point (for example, before a planned maintenance window):

openstack dr recovery point create <protection-group-id>

During planned failover, the engine triggers one final snapshot, waits for it to replicate to the secondary array, then creates Cinder volumes from it and boots the VMs on the secondary site — giving you zero data loss. During unplanned failover, the engine uses the most recently replicated snapshot, so data loss is bounded by the replication lag at the time of the outage.

How ActiveDR replication works

When you create a Protection Group with --replication-type active_dr, the engine creates a pair of pods: a source pod on the primary array (promoted — writable) and a target pod on the secondary array (demoted — read-only). It then creates a directional pod replica link from the source pod to the target pod. Because ActiveDR replication is continuous rather than scheduled, data written to the source pod is replicated to the target pod with near-zero lag.

Pure requires the local and remote pod names to differ for replica links, so the engine names them <pg-name>-src (on the primary array) and <pg-name>-tgt (on the secondary array).

Volumes attached to VMs in the Protection Group are moved into the source pod on the primary array. The replication link carries changes continuously to the target pod on the secondary array.

During failover, the engine promotes the target pod on the secondary array (making it writable) and demotes the source pod on the primary array. During failback, the direction reverses: the source pod is promoted again and the target pod is demoted, and replication resumes in the original direction.

Artifact cleanup: Pure Storage creates transient pods named <pod-name>.undo-demote.<n> and <pod-name>.relink during promotion and demotion transitions. The engine cleans these up automatically; if you see them on the array after an operation completes, they are safe to eradicate manually if they have not already been removed.

Monitoring replication health

Regardless of replication type, use the replication health command to see the current state of the replication link, lag, RPO compliance, and failover readiness:

openstack dr replication health <protection-group-id>

The response includes a failover_validation.can_failover boolean you can use programmatically to gate automated failover workflows. For the async replication type, the health report also lists the number of available recovery points and the lag in seconds relative to your default_rpo_seconds target.

You can also query the replication health via the REST API:

curl -s \
  -H "X-Auth-Token: $OS_TOKEN" \
  -H "OpenStack-API-Version: protector 1.1" \
  http://<protector-api-host>:8788/v1/<tenant-id>/protection-groups/<pg-id>/replication-health

Example 1 — Create a Protection Group with async replication

Create a Protection Group that replicates using periodic snapshots every 5 minutes:

openstack dr protection group create webapp-pg \
  --primary-site site-a \
  --secondary-site site-b \
  --replication-type async

Expected output:

+--------------------+--------------------------------------+
| Field              | Value                                |
+--------------------+--------------------------------------+
| id                 | 3f2e1d0c-...                         |
| name               | webapp-pg                            |
| replication_type   | async                                |
| status             | active                               |
| primary_site       | site-a                               |
| secondary_site     | site-b                               |
+--------------------+--------------------------------------+

Example 2 — Create a Protection Group with ActiveDR replication

Create a Protection Group that uses continuous pod-based replication for near-zero RPO (requires Purity//FA 6.0+):

openstack dr protection group create critical-pg \
  --primary-site site-a \
  --secondary-site site-b \
  --replication-type active_dr

The engine creates pods critical-pg-src (promoted) on the primary array and critical-pg-tgt (demoted) on the secondary array, then establishes the replica link and waits for it to reach replicating status before returning.

Example 3 — Add a VM to the Protection Group

All VMs added to a Protection Group must boot from a Cinder volume and must have all attached volumes using a volume type with replication_enabled='<is> True' set as an extra-spec:

openstack dr protection group member add webapp-pg \
  --instance <nova-instance-uuid> \
  --boot-order 1

If the instance uses ephemeral storage, the request is rejected with InstanceNotCinderBoot (HTTP 422). If a volume uses a non-replicated type, the request is rejected with VolumeTypeNotReplicated (HTTP 422).

Example 4 — Pre-check member eligibility without side effects

Before adding a VM, validate it meets all eligibility requirements:

openstack dr protection group member validate webapp-pg \
  --instance <nova-instance-uuid>

This calls POST /v1/{tenant_id}/protection-groups/{pg_id}/members/validate and returns HTTP 200 with a structured errors array. An empty errors array means the instance is eligible.

Example 5 — Check replication health

openstack dr replication health webapp-pg

For async replication, the output reports lag in seconds, RPO compliance against your default_rpo_seconds target, and the number of recovery points available on the secondary array. The equivalent REST call:

curl -s \
  -H "X-Auth-Token: $OS_TOKEN" \
  http://<protector-api-host>:8788/v1/<tenant-id>/protection-groups/<pg-id>/replication-health

Example response structure:

{
  "replication_health": {
    "protection_group_id": "3f2e1d0c-...",
    "protection_group_name": "webapp-pg",
    "health_report": {
      "replication_type": "async",
      "link": { "status": "replicating", "connected": true },
      "lag": { "lag_seconds": 42, "rpo_compliant": true, "rpo_target_seconds": 900 },
      "recovery_points": { "total_count": 12 }
    },
    "failover_validation": {
      "can_failover": true,
      "warnings": [],
      "blockers": [],
      "message": "Protection group is ready for failover"
    }
  }
}

Example 6 — Create an on-demand recovery point (async only)

Before a planned maintenance window, force an immediate snapshot and replicate it:

openstack dr recovery point create webapp-pg

Then list all available recovery points on the secondary array:

openstack dr recovery point list webapp-pg

Example 7 — Planned failover selecting a specific recovery point

openstack dr failover webapp-pg \
  --failover-type planned \
  --recovery-point <snapshot-name> \
  --wait

The --wait flag blocks until the operation completes. Without it, the command returns immediately with an operation_id you can poll:

openstack dr operation show <operation-id>

Example 8 — Run a non-disruptive DR test (test failover)

Validate your DR setup without affecting production by cloning replicated volumes and booting isolated test VMs on the secondary site:

openstack dr test failover webapp-pg \
  --auto-network \
  --recovery-point <snapshot-name>

Either --auto-network or --network-mapping is required. Test VMs are created with a test- prefix and run on an isolated network. When you are done, clean up:

openstack dr test failover cleanup webapp-pg

To target a specific test failover operation (if multiple exist for the same Protection Group):

openstack dr test failover cleanup webapp-pg --operation <operation-id>

Issue: py-pure-client SDK not installed error on engine startup

Symptom: The protector-engine log shows py-pure-client SDK not installed and refuses to initialise the Pure Storage driver.

Cause: The python3-pure-client Debian package is missing from the engine node.

Fix:

apt-get install python3-pure-client

In Juju deployments, the charm normally installs this automatically. If it is missing, check that the Trilio APT repository is correctly configured in /etc/apt/sources.list.d/trilio-site-recovery.list and that apt-get update has been run.

Issue: StorageDriverConfigError: Missing required Pure Storage config

Symptom: The engine fails to start and logs Missing required Pure Storage config: primary_management_ip, primary_api_token, ....

Cause: One or more of the four required [pure_storage] options (primary_management_ip, primary_api_token, secondary_management_ip, secondary_api_token) are not set.

Fix: In Juju deployments:

juju config trilio-site-recovery \
  pure-primary-management-ip=<ip> \
  pure-primary-api-token=<token> \
  pure-secondary-management-ip=<ip> \
  pure-secondary-api-token=<token>

Do not edit /etc/protector/protector.conf directly; the charm overwrites it.

Issue: storage_driver = mock still active after setting pure

Symptom: Replication health commands return mock data; the engine log shows Loading storage driver: mock.

Cause: In Juju deployments, the Juju charm config option for the storage driver may not have been applied, or the charm has re-rendered the config file after a manual edit.

Fix: Confirm the Juju config value and that it was applied:

juju config trilio-site-recovery storage-driver
# Should return: pure
juju status trilio-site-recovery
# Check that the unit is not in a 'waiting' or 'blocked' state

If the value is correct but the service is still loading mock, restart the engine unit:

# Juju 3.x:
juju run trilio-site-recovery/<unit-number> health-check
# Juju 2.9:
juju run-action trilio-site-recovery/<unit-number> health-check

Issue: No replication array-connection found during ActiveDR group creation

Symptom: Creating a Protection Group with --replication-type active_dr fails with an error indicating no replication connection exists between the arrays.

Cause: ActiveDR requires a replication-type array connection (not a sync-replication/ActiveCluster connection) to be established between the two FlashArrays before the service can create pod replica links. This connection must be configured at the array level, outside of Trilio Site Recovery.

Fix: Log in to the primary FlashArray Purity GUI and navigate to Settings → Network → Connected Arrays. Establish a connection to the secondary array using a connection key from the secondary array. Verify the connection type is replication (not sync-replication). Once the array connection exists, retry the Protection Group creation.

Issue: RPO violation reported in replication health

Symptom: openstack dr replication health <pg-id> reports rpo_compliant: false and failover_validation.can_failover is false.

Cause: The replication lag (in seconds) exceeds the default_rpo_seconds value configured in [pure_storage]. This can happen due to network congestion, a large burst of writes, or a temporary loss of the replication link.

Fix:

1. Check the array-to-array replication link status on the primary FlashArray.
2. If the link is healthy but lag is consistently high, consider increasing the replication bandwidth or decreasing snapshot_interval_seconds.
3. If the RPO target is genuinely too tight for the workload, increase default_rpo_seconds to a value that reflects the achievable RPO.

For Juju deployments:

juju config trilio-site-recovery default-rpo-seconds=1800

Issue: VolumeTypeNotReplicated (HTTP 422) when adding a VM

Symptom: openstack dr protection group member add fails with VolumeTypeNotReplicated.

Cause: One or more Cinder volumes attached to the instance use a volume type that does not have replication_enabled='<is> True' set as an extra-spec.

Fix: Identify which volume types are in use by the instance's attached volumes:

openstack volume show <volume-id> -f value -c type
openstack volume type show <type-name> -f value -c extra_specs

If replication_enabled is not present, add it to the volume type (admin operation):

openstack volume type set <type-name> \
  --property replication_enabled='<is> True'

Alternatively, migrate the volume to a replicated type before adding it to the Protection Group. You can pre-check eligibility without side effects using:

openstack dr protection group member validate <protection-group> \
  --instance <instance-id>

Issue: InstanceNotCinderBoot (HTTP 422) when adding a VM

Symptom: openstack dr protection group member add fails with InstanceNotCinderBoot.

Cause: The Nova instance boots from an ephemeral disk rather than a Cinder volume. The service requires all protected VMs to boot from Cinder.

Fix: Rebuild or migrate the instance to use a Cinder-backed boot volume before adding it to a Protection Group. Use openstack server rebuild with a Cinder volume as the boot source, or create a new instance from a Cinder volume image.

Issue: Artifact pods (<name>.undo-demote.*) persist on the array after failover

Symptom: After a failover or failback involving ActiveDR, the Purity GUI shows pods named <pg-name>-src.undo-demote.<n> or <pg-name>-src.relink.

Cause: Pure Storage creates these transient pods automatically during pod promotion/demotion transitions. They normally auto-eradicate within approximately 24 hours, but can block subsequent demote operations if they persist.

Fix: The engine attempts to clean these up automatically after each operation. If they persist and are blocking a subsequent operation, eradicate them manually from the Purity GUI or via the FlashArray API. Do not eradicate the primary <pg-name>-src or <pg-name>-tgt pods.

Issue: Replication health shows status: unknown or status: not_found

Symptom: openstack dr replication health <pg-id> returns link.status: unknown and can_failover: false.

Cause: The engine cannot find the Protection Group (for async) or pod (for ActiveDR) on the Pure array. This can happen if the storage-level construct was deleted outside of the service, or if primary_site_name / secondary_site_name are misconfigured causing the engine to query the wrong array.

Fix:

1. Verify primary_site_name and secondary_site_name in [pure_storage] match the names registered in the service:
   Copy

   openstack dr site list
   

2. Check that the Protection Group (async) or pod pair (ActiveDR) exists on the expected array via the Purity GUI.
3. If the storage construct was deleted externally, use openstack dr protection group reconcile <pg-id> to attempt recovery, or delete and recreate the Protection Group.