Core Concepts

This page explains the fundamental abstractions you work with in TrilioVault for OpenStack: workloads, snapshots, restores, snapshot policies, and file-level recovery. Understanding these building blocks helps you design backup strategies, interpret the dashboard and CLI output, and make informed decisions about retention, scheduling, and recovery options before you begin protecting your OpenStack environment.

Workloads

A workload is the central unit of protection in TrilioVault. Rather than backing up individual virtual machines in isolation, you group all the VMs that belong to the same application — along with their attached Cinder volumes — into a single named workload. TrilioVault then captures, schedules, and restores that entire group as a consistent unit.

This workload-centric model matters because real applications span multiple VMs: a web tier, an application tier, a database tier. If you back up those VMs independently, their snapshots are taken at different moments in time and cannot be restored to a consistent shared state. When you protect them as one workload, every snapshot captures all VMs simultaneously, so a restore brings the whole application back to the same point in time.

Key constraints to keep in mind:

• Each VM may belong to only one workload at a time. If you attempt to add a VM that is already part of another workload, the operation will be rejected.
• A workload includes both the compute (VM configuration and metadata) and storage (volume data) aspects of every member VM.

You create and manage workloads through the Horizon dashboard (via the horizon-tvault-plugin) or through the workloadmgr CLI (provided by the workloadmanager-client component).

Snapshots

A snapshot is a point-in-time backup of a workload. When you trigger a snapshot — either manually or on a schedule — TrilioVault captures:

• The state and configuration metadata of every VM in the workload
• All data on the Cinder volumes attached to those VMs
• The full tenant network topology: networks, subnets, routers, static routes, ports, and floating IPs

Network topology capture is automatic and requires no user action. Every snapshot includes a complete record of how the workload was networked at the time of capture, enabling full-fidelity recovery even after the original infrastructure has changed.

How snapshots are stored

TrilioVault writes snapshot data to a configured backup target — an NFS share, an S3-compatible object store, or OpenStack Swift. All read and write operations on the backup target are performed as the nova user (nova:nova). The POSIX UID and GID of the nova user must be identical across all TrilioVault cluster nodes and all compute nodes to avoid permission errors during backup and restore.

During the backup process, TrilioVault creates Cinder snapshots and temporary Cinder volumes. For each disk being backed up, two Cinder snapshots and one temporary Cinder volume are consumed. Ensure your OpenStack Cinder quotas accommodate this overhead before you schedule large workloads.

Full vs. incremental snapshots

TrilioVault uses an incremental-forever backup strategy powered by its Virtual Snapshot Technology (VAST):

• Full snapshots capture all data blocks that are actually committed to storage volumes. Empty (zeroed) blocks are skipped, so the initial snapshot is typically much smaller than the raw volume size.
• Incremental snapshots capture only the blocks that changed since the previous snapshot. This keeps ongoing backup windows short and minimizes storage consumption on the backup target.

After the first full snapshot, all subsequent automatic snapshots are incremental. The result is a chain of recovery points that can each be restored independently, without requiring you to manually merge or manage the chain.

Snapshot Policies

A snapshot policy (also called a backup policy or retention policy) is a schedule and retention rule attached to a workload. It tells TrilioVault:

1. When to take automatic snapshots — for example, daily at 02:00 or every six hours
2. How many snapshots to keep before older ones are deleted

The wlm-cron service fires scheduled snapshot jobs according to the policy defined on each workload. When the number of stored snapshots exceeds the retention limit, the oldest snapshots are automatically deleted from the backup target.

Retention policies let you balance recovery granularity against storage cost. A tighter schedule with a shorter retention window gives you more recent recovery points but uses more backup target capacity. A longer retention window preserves more history but requires proportionally more storage.

Snapshot policies are defined per workload and can be updated at any time without interrupting ongoing protection. Because TrilioVault's configuration is idempotent, you can also re-run the Ansible-based configurator to change cluster-wide defaults without risk of duplication.

Restore Types

TrilioVault supports two main restore paths:

One-click (full workload) restore

A full workload restore recreates every VM in the workload, their volumes, and — if the target project's network space is empty — the network topology captured in the snapshot. You can restore into the original OpenStack project or into a different project.

Network topology restore has one strict requirement: the tenant's network space must be completely empty of networking components (networks, subnets, routers, ports, floating IPs) before the restore begins. Any conflict will cause the restore to fail. After the restore completes, TrilioVault does not automatically connect the restored private networks to the public network; you must perform that final step manually.

Selective restore

A selective restore lets you recover a subset of the workload — specific VMs or specific volumes — without recreating the entire application. This is useful when only part of an application is affected by data loss or corruption.

File-Level Recovery

File-level restore (also called granular restore) lets you retrieve individual files from inside a snapshot without recreating any VMs. This is the right tool when a user accidentally deletes a file or when only a small amount of data needs to be recovered from an otherwise healthy system.

File-level recovery is performed through the File Recovery Manager, a Glance image deployed alongside TrilioVault. For the File Recovery Manager image to appear as a valid mount target in the Horizon UI, the Glance image must have the property tvault_recovery_manager=yes set. Without this property, the image will not be selectable during a file-level restore workflow.

File-level restores are initiated through the horizon-tvault-plugin dashboard or the workloadmgr CLI and do not require the target network to be empty, making them faster and less disruptive than full workload restores.

The Role-Based Access Model

All workload, snapshot, and restore operations require the TrilioVault Trustee RBAC role. This role cannot be substituted by the OpenStack admin role — even cloud administrators must be assigned the Trustee role to perform TrilioVault operations. Ensure users and service accounts responsible for backup management are granted this role through Keystone before they attempt to create workloads or trigger snapshots.

How the Components Work Together

Understanding which TrilioVault component is responsible for each function helps you interpret logs and troubleshoot issues:

wlm-api and wlm-scheduler are cluster-controlled by Pacemaker and run on the master node only. wlm-workloads runs on every TrilioVault node, allowing parallel data movement across the cluster.

Example 1 — Create a workload from the CLI

The following command creates a workload named web-app-protection that groups two VMs under a daily snapshot policy. Replace the UUIDs with values from your OpenStack environment.

workloadmgr workload-create \
  --name "web-app-protection" \
  --workload-type-id <workload-type-uuid> \
  --instance instance-id=<vm-uuid-1> \
  --instance instance-id=<vm-uuid-2> \
  --jobschedule '{"start_date": "06/17/2024", "end_date": "12/31/2024", "interval": "24hrs", "start_time": "02:00", "fullbackup_interval": "-1", "retention_policy_type": "Number of Snapshots", "retention_policy_value": "30"}'

Expected output:

+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| id               | a3f8c120-4e2b-11ef-b864-0242ac120002 |
| name             | web-app-protection                   |
| status           | available                            |
| instances        | <vm-uuid-1>, <vm-uuid-2>             |
+------------------+--------------------------------------+

The workload status available confirms it is ready for snapshots. The wlm-cron service will automatically trigger the first snapshot at 02:00 on the configured start date.

Example 2 — Trigger a manual snapshot

Once a workload exists, you can take an on-demand snapshot at any time:

workloadmgr snapshot-create \
  --workload-id a3f8c120-4e2b-11ef-b864-0242ac120002 \
  --name "pre-upgrade-snapshot"

Expected output:

+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| id               | b7d91a44-4e2c-11ef-b864-0242ac120002 |
| name             | pre-upgrade-snapshot                 |
| workload_id      | a3f8c120-4e2b-11ef-b864-0242ac120002 |
| status           | executing                            |
+------------------+--------------------------------------+

The status transitions from executing to available when the snapshot completes. The wlm-workloads agent performs the actual data capture; progress is logged at /var/log/workloadmgr/workloadmgr-workloads.log.

Example 3 — List snapshots for a workload

workloadmgr snapshot-list --workload-id a3f8c120-4e2b-11ef-b864-0242ac120002

Expected output:

+--------------------------------------+----------------------+-----------+---------------------+
| id                                   | name                 | status    | created_at          |
+--------------------------------------+----------------------+-----------+---------------------+
| b7d91a44-4e2c-11ef-b864-0242ac120002 | pre-upgrade-snapshot | available | 2024-06-17T01:52:03 |
+--------------------------------------+----------------------+-----------+---------------------+

Each snapshot in the available state can be used as the source for a full workload restore, a selective restore, or a file-level restore.

Example 4 — Initiate a one-click workload restore

workloadmgr workload-restore \
  --snapshot-id b7d91a44-4e2c-11ef-b864-0242ac120002 \
  --restore-name "web-app-restore-01" \
  --oneclickrestore True

Expected output:

+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| id               | c9e02b55-4e2d-11ef-b864-0242ac120002 |
| name             | web-app-restore-01                   |
| status           | executing                            |
| snapshot_id      | b7d91a44-4e2c-11ef-b864-0242ac120002 |
+------------------+--------------------------------------+

Ensure the target tenant's network space is empty before running a one-click restore that includes network topology. After the restore completes, manually reconnect the restored private networks to your public network.

• Install and Configure TrilioVault — Covers backup target setup, cluster deployment, and the Ansible-based configurator that establishes the environment these concepts operate within.
• Install the Horizon Dashboard Plugin — Explains how to enable the horizon-tvault-plugin so you can manage workloads, snapshots, and restores from the Horizon web interface.
• Create a Workload — Step-by-step instructions for defining your first workload and assigning VMs to it.
• Take a Snapshot — How to trigger on-demand snapshots and verify their status using the workloadmgr CLI or Horizon.
• Restore a Workload or File — Detailed guidance on one-click restore, selective restore, and file-level recovery including network topology prerequisites.
• Monitor and Manage Snapshot Policies — How to create, update, and monitor snapshot policies and interpret retention behavior.
• Authenticate and Call the WorkloadMgr API — Reference for the REST API surface exposed by wlm-api, including how to obtain tokens and construct requests.
• Diagnose and Recover a Degraded Service — Log locations, common failure modes for wlm-workloads, wlm-api, and wlm-cron, and recovery procedures.