Getting Started

This tutorial walks you through deploying TrilioVault for OpenStack from scratch — standing up the TVM appliance, running the configurator, registering the Workload Manager API endpoint in Keystone, installing the datamover and Horizon dashboard plugin, and creating your first workload, snapshot, and restore. Completing this tutorial gives you a fully operational TrilioVault environment and confirms that every layer of the backup pipeline is working correctly. If you are new to TrilioVault, start here before consulting the reference documentation for individual components. For supported OpenStack distributions and storage targets, refer to the TrilioVault Support Matrix linked at the end of this page.

Before you begin, confirm the following are in place:

OpenStack environment

• OpenStack release: Stein or later
• Services running: Keystone, Nova, Cinder (Horizon is optional but required for the dashboard plugin steps)
• Cinder quotas must accommodate two snapshots and one temporary volume per disk being backed up — adjust your quota before the first snapshot run
• The TrilioVault Trustee RBAC role must be available in Keystone; the standard OpenStack admin role cannot substitute for it

Infrastructure

• One or more standalone KVM hosts (bare-metal or separate virtualisation layer) to host the TVM QCOW2 appliance — TVM nodes cannot run as VMs inside the OpenStack cloud they protect
• A cluster VIP (virtual IP address) reserved and routable from your OpenStack control plane — required even for single-node deployments; Pacemaker/HAProxy will bind this VIP and it will be registered as the Keystone service endpoint
• Network connectivity between the TVM nodes, all OpenStack compute nodes, and the backup target

Backup target — one of:

• NFS share (all TrilioVault versions)
• S3-compatible object storage endpoint (TrilioVault 3.0+)
• OpenStack Swift endpoint

User accounts and permissions

• nova OS user with identical POSIX UID and GID on every TVM node and every OpenStack compute node (all backup target I/O runs as nova:nova)
• OpenStack admin credentials to register Keystone endpoints and assign RBAC roles

Tooling (optional, for configuration management)

• Ansible, Salt, or Juju if you plan to use triliovault-cfg-scripts for automated deployment
• Python 3.6 or later on any host where you will use the workloadmgr CLI or workloadmgrclient library

Recommended deployment A three-node cluster is best practice for fault tolerance and load balancing. A single-node deployment is supported but is not recommended for production.

The steps below show the minimal path from zero to a verified restore. Each step links to the detailed procedure that follows.

1. Deploy the TVM appliance — Boot the TrilioVault QCOW2 image on each standalone KVM host.
2. Reserve a cluster VIP — Assign a virtual IP that Pacemaker/HAProxy will manage and that will become the Keystone endpoint address.
3. Run the configurator — Execute the TrilioVault configurator (via triliovault-cfg-scripts Ansible playbooks) to form the cluster, configure the backup target, and start all WLM services.
4. Register the Keystone endpoint — Create the workloadmgr service and its endpoint in Keystone pointing to the cluster VIP.
5. Assign the Trustee role — Grant the TrilioVault_Trustee RBAC role to every tenant user who will manage workloads.
6. Install the datamover on compute nodes — Deploy the tvault-contego datamover extension on every Nova compute node using the triliovault-cfg-scripts playbooks.
7. Install the Horizon plugin — Deploy horizon-tvault-plugin on your Horizon node to add the Workloads panel.
8. Verify services — Confirm wlm-api, wlm-scheduler, wlm-workloads, and wlm-cron are running.
9. Create a workload — Use the workloadmgr CLI or the Horizon Workloads panel to define your first workload.
10. Take a snapshot — Trigger an on-demand snapshot of the workload.
11. Verify a restore — Perform a test restore and confirm the workload is recovered successfully.

Step 1 — Deploy the TVM appliance

On each standalone KVM host that will form your TrilioVault cluster, import and boot the TrilioVault QCOW2 image:

virt-install \
  --name triliovault-node-1 \
  --ram 16384 \
  --vcpus 8 \
  --disk path=/var/lib/libvirt/images/triliovault-node-1.qcow2,format=qcow2 \
  --import \
  --os-variant rhel8 \
  --network bridge=br0

Repeat for each node (recommended: three nodes). The QCOW2 appliance includes all TrilioVault services pre-installed. Do not instantiate TVM nodes as Nova instances inside the OpenStack cloud you intend to protect.

Success: Each appliance boots to a login prompt. The default credentials are admin / password. Change the password immediately on first login.

Step 2 — Reserve and configure the cluster VIP

Choose a static IP address that is routable from your OpenStack control plane and all compute nodes. This address will be used by Pacemaker/HAProxy as the cluster VIP and will later be registered as the Keystone service endpoint. Record it — you will supply it to the configurator in the next step.

A VIP is required even if you are running a single-node deployment.

Success: The IP address is pingable from your OpenStack controller and is not assigned to any host interface.

Step 3 — Run the configurator

The triliovault-cfg-scripts repository contains Ansible playbooks that configure the cluster idempotently. Clone the repository onto a host that can reach all TVM nodes, then edit the inventory and variable files to supply your node IPs, cluster VIP, and backup target details.

git clone https://github.com/triliovault/triliovault-cfg-scripts.git
cd triliovault-cfg-scripts
# Edit ansible/inventory/hosts with your TVM node addresses
# Edit ansible/group_vars/all.yml with your VIP and backup target settings
ansible-playbook -i ansible/inventory/hosts ansible/tvault-config.yml

The configurator is idempotent — you can re-run it at any time to change settings without risk of duplicating resources.

The playbooks will:

• Form the Pacemaker/HAProxy cluster and bind the VIP
• Configure the backup target (NFS, S3, or Swift)
• Start wlm-api, wlm-scheduler, wlm-workloads, and wlm-cron
• Ensure nova:nova UID/GID consistency is validated across nodes

Success: The playbook completes with no failed tasks. wlm-api and wlm-scheduler are running on the master node; wlm-workloads is running on every TVM node.

Step 4 — Register the Keystone service endpoint

Create the workloadmgr service type and register its endpoint in Keystone, pointing to the cluster VIP. Replace <VIP> with your actual cluster virtual IP and <PORT> with the port on which wlm-api listens (confirm in your configurator output).

openstack service create --name workloadmgr \
  --description "TrilioVault Workload Manager" \
  workloadmgr

openstack endpoint create --region RegionOne \
  workloadmgr public http://<VIP>:<PORT>/v1/%(tenant_id)s

openstack endpoint create --region RegionOne \
  workloadmgr internal http://<VIP>:<PORT>/v1/%(tenant_id)s

openstack endpoint create --region RegionOne \
  workloadmgr admin http://<VIP>:<PORT>/v1/%(tenant_id)s

Success: openstack endpoint list --service workloadmgr shows three endpoints (public, internal, admin) with status enabled.

Step 5 — Assign the TrilioVault Trustee role

The TrilioVault_Trustee RBAC role is mandatory for all workload, snapshot, and restore operations. The standard admin role cannot substitute for it. Assign it to every tenant user who will use TrilioVault.

# Create the role if it does not already exist
openstack role create TrilioVault_Trustee

# Assign the role to a user in a project
openstack role add --project <PROJECT_NAME> --user <USERNAME> TrilioVault_Trustee

Repeat the role add command for every user and project that requires backup access.

Success: openstack role assignment list --user <USERNAME> --project <PROJECT_NAME> shows TrilioVault_Trustee.

Step 6 — Install the datamover on compute nodes

The datamover (tvault-contego) must be installed on every Nova compute node. Use the triliovault-cfg-scripts Ansible playbooks targeting your compute nodes:

cd triliovault-cfg-scripts
# Ensure your compute nodes are listed under [compute] in ansible/inventory/hosts
ansible-playbook -i ansible/inventory/hosts ansible/tvault-datamover-config.yml

The datamover integrates with Nova to coordinate volume quiescing and block-level data capture during snapshots.

Success: The datamover service is running on each compute node. Confirm with:

systemctl status tvault-contego

Step 7 — Install the Horizon dashboard plugin

If you use the OpenStack Horizon web dashboard, install horizon-tvault-plugin on your Horizon node to add the Workloads panel. This is optional — all operations can also be performed via the workloadmgr CLI.

cd triliovault-cfg-scripts
# Ensure your Horizon node is listed under [horizon] in ansible/inventory/hosts
ansible-playbook -i ansible/inventory/hosts ansible/tvault-horizon-plugin-config.yml

After installation, restart the Horizon web server so the new panel is loaded.

Success: Log in to the Horizon dashboard and confirm a Workloads panel appears in the Project menu.

Step 8 — Verify all WLM services

Confirm that all four core services are running before creating workloads. On the TVM master node:

# wlm-api and wlm-scheduler are cluster-controlled and run on the master node only
systemctl status wlm-api
systemctl status wlm-scheduler
systemctl status wlm-cron

# wlm-workloads runs on every TVM node
systemctl status wlm-workloads

Also verify the API is reachable:

workloadmgr --os-auth-url <KEYSTONE_URL> \
  --os-username <USERNAME> \
  --os-password <PASSWORD> \
  --os-project-name <PROJECT_NAME> \
  workload-list

Success: All four services show active (running). The workload-list command returns an empty list (or existing workloads) without error.

Step 9 — Create your first workload

A workload groups one or more VMs and their attached Cinder volumes into a single protection unit. Each VM may belong to only one workload at a time.

workloadmgr workload-create \
  --name "my-first-workload" \
  --instance instance-id=<NOVA_INSTANCE_UUID> \
  --description "Getting started workload"

To include multiple VMs, repeat the --instance flag for each one.

Success: The command returns a workload object with a status of available and a unique workload ID. Confirm with:

workloadmgr workload-show <WORKLOAD_ID>

Step 10 — Take an on-demand snapshot

Trigger an immediate point-in-time snapshot of your workload. TrilioVault will capture VM state, all attached volume data (changed blocks for incremental, all committed data for full), and — as of version 3.0 — the full tenant network topology automatically.

workloadmgr snapshot-create \
  --name "first-snapshot" \
  <WORKLOAD_ID>

Monitor the snapshot status:

workloadmgr snapshot-show <SNAPSHOT_ID>

Success: The snapshot status transitions from executing to available. The backup target now contains the snapshot data.

Step 11 — Verify a restore

Perform a test restore to confirm the full pipeline is working. TrilioVault supports restoring to the same project (in-place) or a different project. For a first verification, use a selective restore to a new environment to avoid overwriting running VMs.

workloadmgr restore-create \
  --name "verify-restore" \
  --restore-type selective \
  <SNAPSHOT_ID>

Monitor the restore:

workloadmgr restore-show <RESTORE_ID>

Once the restore completes, verify in Nova that the restored VMs are running:

openstack server list

Important: If the snapshot included network topology data, ensure the tenant's network space is completely empty of existing networking components before restoring — any conflict will cause the restore to fail. After network topology restore, you must manually reconnect restored private networks to the public network; TrilioVault does not perform this final step automatically.

Success: The restore status shows available (complete). The restored VMs appear in Nova and are reachable.

Example 1 — List all workloads for the current project

workloadmgr workload-list

Expected output:

+--------------------------------------+--------------------+-----------+
| ID                                   | Name               | Status    |
+--------------------------------------+--------------------+-----------+
| a1b2c3d4-e5f6-7890-abcd-ef1234567890 | my-first-workload  | available |
+--------------------------------------+--------------------+-----------+

Example 2 — Create a workload with two VMs

workloadmgr workload-create \
  --name "web-tier" \
  --description "Frontend web servers" \
  --instance instance-id=aaaa1111-bbbb-cccc-dddd-eeee00001111 \
  --instance instance-id=aaaa2222-bbbb-cccc-dddd-eeee00002222

Expected output:

+-------------+--------------------------------------+
| Property    | Value                                |
+-------------+--------------------------------------+
| id          | f9e8d7c6-b5a4-3210-fedc-ba9876543210 |
| name        | web-tier                             |
| status      | available                            |
| instances   | aaaa1111-..., aaaa2222-...           |
+-------------+--------------------------------------+

Example 3 — Trigger an on-demand snapshot and poll until complete

# Trigger snapshot
workloadmgr snapshot-create \
  --name "web-tier-snap-$(date +%Y%m%d%H%M%S)" \
  f9e8d7c6-b5a4-3210-fedc-ba9876543210

# Poll status (repeat until status is 'available')
workloadmgr snapshot-show <SNAPSHOT_ID>

Expected output when complete:

+-------------+--------------------------------------+
| Property    | Value                                |
+-------------+--------------------------------------+
| id          | 11223344-5566-7788-99aa-bbccddeeff00 |
| name        | web-tier-snap-20240115103045         |
| status      | available                            |
| workload_id | f9e8d7c6-b5a4-3210-fedc-ba9876543210 |
+-------------+--------------------------------------+

Example 4 — List snapshots for a workload

workloadmgr snapshot-list --workload-id f9e8d7c6-b5a4-3210-fedc-ba9876543210

Expected output:

+--------------------------------------+------------------------------+-----------+
| ID                                   | Name                         | Status    |
+--------------------------------------+------------------------------+-----------+
| 11223344-5566-7788-99aa-bbccddeeff00 | web-tier-snap-20240115103045 | available |
+--------------------------------------+------------------------------+-----------+

Example 5 — Initiate a selective restore from a snapshot

workloadmgr restore-create \
  --name "web-tier-restore-test" \
  --restore-type selective \
  11223344-5566-7788-99aa-bbccddeeff00

Expected output:

+-------------+--------------------------------------+
| Property    | Value                                |
+-------------+--------------------------------------+
| id          | deadbeef-0000-1111-2222-333344445555 |
| name        | web-tier-restore-test                |
| status      | executing                            |
| snapshot_id | 11223344-5566-7788-99aa-bbccddeeff00 |
+-------------+--------------------------------------+

Poll workloadmgr restore-show deadbeef-0000-1111-2222-333344445555 until status is available.

Example 6 — Check WLM service logs after a failed snapshot

If a snapshot fails, inspect the relevant log files on the TVM node:

# Worker service log (most snapshot failures surface here)
tail -n 100 /var/log/workloadmgr/workloadmgr-workloads.log

# API service log
tail -n 100 /var/log/workloadmgr/workloadmgr-api.log

# Datamover log on the compute node where the VM resides
tail -n 100 /var/log/nova/tvault-contego.log

# Nova API extension log
tail -n 100 /var/log/nova/nova-api.log

Use the following table to diagnose common issues encountered during the getting-started workflow.

Issue: workloadmgr CLI returns Unauthorized or 403 Forbidden

• Symptom: Running any workloadmgr command returns an authentication error or HTTP 403.
• Likely cause: The user account does not have the TrilioVault_Trustee RBAC role assigned for the target project. The standard admin role does not grant access to WLM operations.
• Fix: Assign the Trustee role: openstack role add --project <PROJECT_NAME> --user <USERNAME> TrilioVault_Trustee. Verify with openstack role assignment list --user <USERNAME> --project <PROJECT_NAME>.

Issue: workloadmgr workload-list returns EndpointNotFound or connection refused

• Symptom: The CLI cannot reach the WLM API; Keystone returns no endpoint or the connection times out.
• Likely cause 1: The workloadmgr service or its endpoints were not registered in Keystone, or were registered with an incorrect URL.
• Fix 1: Run openstack endpoint list --service workloadmgr and verify all three endpoint URLs point to the cluster VIP and port. Re-register if missing or incorrect (see Step 4).
• Likely cause 2: The cluster VIP is not bound — Pacemaker/HAProxy may not have started correctly.
• Fix 2: On the TVM master node, check pcs status and confirm the VIP resource is started. Review the configurator output for cluster formation errors.

Issue: wlm-api or wlm-scheduler is not running

• Symptom: systemctl status wlm-api or systemctl status wlm-scheduler shows the service as inactive or failed.
• Likely cause: These services are cluster-controlled by Pacemaker and run on the master node only. They may not start via systemctl directly on non-master nodes, or the cluster may not have a master elected.
• Fix: Check Pacemaker cluster status with pcs status. If there is no master, investigate cluster quorum. On the master node, use pcs resource enable wlm-api or pcs resource enable wlm-scheduler if the resources are disabled. Consult /var/log/workloadmgr/workloadmgr-api.log for errors.

Issue: Snapshot remains in executing status indefinitely

• Symptom: workloadmgr snapshot-show <SNAPSHOT_ID> shows status: executing for an extended period with no progress.
• Likely cause 1: The datamover (tvault-contego) is not running on the compute node hosting the VM.
• Fix 1: SSH to the relevant compute node and run systemctl status tvault-contego. If stopped, start it: systemctl start tvault-contego. Check /var/log/nova/tvault-contego.log for errors.
• Likely cause 2: Cinder quotas are exhausted — TrilioVault creates two Cinder snapshots and one temporary volume per disk being backed up.
• Fix 2: Check Cinder quota usage: openstack quota show <PROJECT_NAME>. Increase snapshot and volume quotas as needed: openstack quota set --snapshots <N> --volumes <N> <PROJECT_NAME>.
• Likely cause 3: The nova:nova UID/GID is inconsistent between the TVM nodes and compute nodes, causing backup target write failures.
• Fix 3: Verify UID/GID on all nodes: id nova. Reconcile so the UID and GID are identical across every TVM node and compute node, then retry the snapshot.

Issue: Restore fails with a network conflict error

• Symptom: A restore operation fails with an error indicating existing networking components conflict with the restore.
• Likely cause: The tenant's network space already contains networks, subnets, routers, or ports that conflict with the topology captured in the snapshot. TrilioVault (3.0+) captures the full tenant network topology in every snapshot and requires the network space to be completely empty before restoring it.
• Fix: Delete all conflicting networking components from the tenant project before retrying the restore: openstack router delete, openstack network delete, etc. After the restore completes, manually reconnect the restored private networks to the public network — TrilioVault does not perform this step automatically.

Issue: The Horizon Workloads panel does not appear after plugin installation

• Symptom: After running the horizon-tvault-plugin installation playbook, the Workloads panel is absent from the Horizon Project menu.
• Likely cause: The Horizon web server was not restarted after the plugin was installed, so the new panel module was not loaded.
• Fix: Restart the Horizon web server (e.g., systemctl restart apache2 or systemctl restart httpd depending on your distribution). Clear your browser cache and reload Horizon.

Issue: TVM appliance cannot be reached on the cluster VIP after reboot

• Symptom: After rebooting a TVM node or the entire cluster, the VIP is not responding and WLM services are unreachable.
• Likely cause: Pacemaker did not automatically re-elect a master or re-bind the VIP after the reboot.
• Fix: On each TVM node, check pcs status to identify which node holds the master resource. If no master is elected, check Pacemaker logs (/var/log/pacemaker/pacemaker.log) for fencing or quorum issues. In a single-node deployment, ensure the Pacemaker service itself started: systemctl status pacemaker corosync.