--- title: Key Concepts product: trilio-share-protection doc_type: concept version: master source: git2docs (code-derived, validation-filtered) canonical: https://git2docs.com/murali-balcha/docs/trilio-share-protection/trilio-share-protection-/key-concepts --- # Key Concepts _Domain concepts, terminology, and API design patterns used in this service. Include manila concepts, including DHSS, and kopia concepts._ ## Overview This page defines every domain concept, term, and API design pattern used in Trilio Share Protection for OpenStack (codename Abacá). Understanding these concepts is essential before you register your first backup target, write automation against the REST API, or interpret a failed job. The page also explains relevant Manila and Kopia concepts — such as DHSS, snapshot strategies, and repository design — because Abacá's behaviour is shaped directly by them. ## Content ## Core resources ### Backup target A backup target (also called a *target*) is a registered S3-compatible bucket, together with its endpoint URL and references to credentials stored in Barbican, that Abacá uses as the destination for one tenant's encrypted backups. Each target holds exactly one Kopia repository. You can register multiple targets per tenant to separate data classes — for example, an object-locked production bucket from a dev bucket, or per-cost-centre buckets — but the repository is the deduplication domain, so splitting targets means splitting deduplication savings. The recommended default is one target per tenant. A target progresses through the following statuses: | Status | Meaning | |---|---| | `pending` | Registered but preflight has not been run or completed. | | `ready` | All preflight bucket-conformance checks passed. | | `preflight_failed` | Preflight failed; the tenant must remediate the bucket (bad lifecycle rules, access denied, addressing-mode mismatch, and so on). | | `enroll_failed` | Preflight passed but Kopia repository initialisation failed (for example, a Kopia timeout or unexpected operator-side fault). | | `disabled` | Administratively disabled. | --- ### Protection policy A protection policy (also called a *policy* or *backup policy*) is a rule that binds a Manila share to a backup target and specifies when backups run (a cron schedule) and how long they are kept (retention settings). Once you attach a policy to a share, backups run automatically according to the schedule. Because of the `uq_policy_share` uniqueness constraint, each share can have at most one active policy at a time — changing targets means reassigning the policy, which triggers a full re-ingest into the new repository. Policies support optional `include_patterns` and `exclude_patterns` that are translated at dispatch time into Kopia per-source policy rules, letting you exclude temporary files, caches, or build artefacts from backups. --- ### Backup A backup (also called a *share backup* or *snapshot*) is a point-in-time snapshot of a Manila share's file contents stored as an encrypted, deduplicated Kopia snapshot inside your S3 bucket. Each backup record carries: - `size_bytes` — logical size of the data backed up. - `stored_bytes` — bytes actually stored after deduplication and compression. - `files_count` — number of files captured. - `crash_consistent` — whether the backup is only crash-consistent rather than point-in-time (see [Crash-consistent backup](#crash-consistent-backup) below). - `kopia_snapshot_id` — the Kopia snapshot manifest ID that serves as the durable handle into the repository. From Kopia's perspective every snapshot is a "full" — unchanged blocks across snapshots share their underlying content objects, so there is no incremental chain to walk at restore time. --- ### Restore A restore (also called a *share restore*) is the operation of recreating files from a backup onto a new share (the default) or back onto the original share. You can restore a whole share, a sub-path, or a single file by supplying a repository-relative `sub_path` selector. Restore modes: | Mode | Description | |---|---| | `new_share` | Files are restored into a freshly created Manila share. Safe default. | | `in_place` | Files are restored over the existing share. Requires `force=true`. | --- ### Job A job (also called a *backup job*, *restore job*, or *enrollment job*) is the unit of work that tracks every asynchronous operation — backup, restore, enrollment, or maintenance — as it moves through a sequence of states. Every resource-mutating operation in Abacá returns or is associated with a job you can poll to follow progress. Job types: | Type | Description | |---|---| | `backup` | A scheduled or on-demand share backup. | | `restore` | A restore of a backup to a new or existing share. | | `maintenance` | Kopia repository maintenance (enforces retention, reclaims space). | | `verify` | Kopia snapshot verification (aspirational; not yet scheduled by the reconciliation loop). | | `target_enroll` | One-time preflight and Kopia repository initialisation for a new backup target. | --- ### Job state machine Every job passes through a fixed linear sequence of states. Each non-terminal state may additionally transition to `error`, at which point the conductor runs idempotent cleanup hooks. ``` queued → provisioning_network → provisioning_source → connecting_repository → transferring → finalizing → releasing → available (terminal — success) → error (terminal — failure, reachable from any non-terminal state) ``` What each state means: | State | What the service is doing | |---|---| | `queued` | Job accepted; waiting for a worker with a free capacity slot. | | `provisioning_network` | Attaching the worker VM to the share's network (DHSS=true: hot-plugging a Neutron port; DHSS=false: verifying export reachability). | | `provisioning_source` | Obtaining a consistent, read-only view of the share data (creating a Manila snapshot or mounting the live share). | | `connecting_repository` | Fetching credentials from Barbican and connecting the Kopia client to the S3 repository. | | `transferring` | Kopia is actively transferring data to (backup) or from (restore) the repository. | | `finalizing` | Recording the completed Kopia snapshot manifest ID, updating size statistics, and marking the backup or restore record. | | `releasing` | Tearing down the snapshot, unmounting the share, detaching the Neutron port, and releasing the capacity slot. | | `available` | Job completed successfully. | | `error` | Job failed. Inspect `error_code`, `error_detail`, and `error_category` for next steps. | `target_enroll` jobs walk the same state machine with no-op passes on `provisioning_network`, `provisioning_source`, and `releasing` because there is no share involved. --- ### Error category Every failed job carries an `error_category` field that tells you who needs to act: | Value | Meaning | |---|---| | `tenant_action_required` | The tenant must fix a configuration — for example, a bad bucket URL, missing object lock, or a revoked Barbican secret. | | `operator_action_required` | The infrastructure or service needs attention — for example, a dead worker VM or a RabbitMQ connectivity issue. | Use `error_category` to route alerts: tenant-actionable errors should surface to the tenant; operator-actionable errors should page your operations team. --- ## Enrollment and bucket conformance ### Enrollment Enrollment (also called *target enrollment* or `target_enroll`) is the one-time setup process where Abacá verifies your S3 bucket meets all requirements, records the credential references in Barbican, and initialises a Kopia repository so backups can be stored there. Until enrollment succeeds, no backups can run against that target. Enrollment is driven as a job through the standard job state machine. --- ### Preflight checks Preflight checks (also called *conformance checks* or *bucket conformance*) are a set of automated checks Abacá runs against your S3 bucket before accepting it as a backup target. All checks run in order; a failure on any fatal check aborts enrollment and sets the target status to `preflight_failed`. | Check | Fatal | What is verified | |---|---|---| | `reachability` | Yes | The endpoint is reachable and the supplied credentials authenticate. | | `rw_round_trip` | Yes | A test object can be written, read back identically, and deleted. | | `addressing_mode` | No | Path-style vs virtual-hosted addressing is detected and recorded. | | `object_lock` | Yes | Object lock (WORM) is enabled on the bucket. | | `lifecycle_rules` | Yes | No lifecycle expiration rule exists that could delete backup objects. | | `bucket_adoption` | Yes | The bucket is empty, holds an adoptable existing Abacá repository, or is rejected (foreign objects present). | S3-compatible targets validated in production include AWS S3, MinIO, Ceph RGW, ODF, and Wasabi. --- ### Object lock Object lock (also called *WORM*, *S3 object lock*, or *immutability*) is an S3 bucket feature that prevents backup objects from being deleted or overwritten for a specified retention period, providing ransomware-resistant immutable storage. Object lock must be enabled at bucket creation time — it cannot be added to an existing bucket. Abacá's preflight check verifies this before accepting a bucket as a backup target, and Kopia's write path relies on it to enforce your retention window even if credentials are compromised. --- ## Security and secrets ### Barbican Barbican (also called the *secret store* or *key manager*) is the OpenStack secret-storage service that holds your S3 access key, secret key, and Kopia repository password as tenant-owned secrets. Abacá never stores the secret payloads itself — it holds only Barbican hrefs (opaque URI references). Secrets travel from Barbican to the worker VM exclusively via environment variables on the Kopia subprocess; they never appear in argv, on disk, or in log files. --- ### Keystone trust A Keystone trust (also called a *trust*, `trust_id`, or *delegation*) is a delegated authorisation token that lets Abacá act on a tenant's behalf — fetching secrets from Barbican and performing Manila operations — without the tenant being logged in. The trust is created once during enrollment and scoped to exactly the permissions Abacá needs. The `trust_id` is stored in the `backup_targets` table; the trust credential itself is never persisted. --- ### FIPS FIPS (also called *FIPS 140*, *FIPS mode*, or *FIPS profile*) refers to the US government cryptographic standard (FIPS 140) that specifies approved algorithms. FIPS mode is a supported, validated configuration that operators can rely on in production. Abacá: - Configures every Kopia repository with `--encryption=AES256-GCM-HMAC-SHA256` and `--block-hash=HMAC-SHA256-128` (overriding Kopia's non-approved BLAKE2 default). - Runs worker VMs on RHEL in FIPS mode. - Uses a FIPS-capable Go toolchain for the baked-in Kopia binary. The cryptographic profile is immutable at repository creation time — you cannot change it after the fact. --- ## Kopia concepts ### Kopia repository A Kopia repository (also called a *repository* or *repo*) is an encrypted, deduplicated, content-addressed archive stored in your S3 bucket that holds all the backup snapshots for one backup target. The repository is the authoritative record of your backup data — the Abacá database holds only a summary index (the Kopia snapshot manifest ID per backup record). This means state can be rebuilt from the repository if the database is lost. One bucket maps to exactly one repository; deduplication is scoped to the repository, so data is never deduplicated across tenants. --- ### Retention Retention (also called *retention policy* or *backup retention*) determines how many backups are kept and for how long — for example, 7 daily, 4 weekly, and 12 monthly backups. Retention rules are defined on the protection policy and enforced at the Kopia level (per snapshot source, identified by the share UUID). Expired backups are reclaimed by a scheduled `maintenance` job that runs Kopia's cleanup against the repository. Retention is not enforced until maintenance runs — overdue maintenance is surfaced prominently by the conductor. --- ### Crash-consistent backup A crash-consistent backup (also called *crash consistency* or *point-in-time consistency*) is a backup that captures the on-disk state of a share at a single point in time, equivalent to what you would see after a sudden power loss. Filesystem structures are intact and recoverable, but in-flight application writes that had not yet been flushed to disk may be missing. Abacá uses snapshot-based strategies by preference to avoid this; the `crash_consistent` flag on a backup record tells you whether the backup was point-in-time or merely crash-consistent. Application-consistent backups (tenant quiesce hooks) are out of scope for Phase 1. --- ### Fidelity Fidelity (also called *restore fidelity* or *data fidelity*) describes how faithfully a restored share reproduces the original — including file contents, ownership, permissions, timestamps, symlinks, and sparse-file layout. Abacá documents both what it preserves and its current known limitations. The `root_read` and `chown` capability probes run against each share at job time to determine what fidelity the backup and restore can promise for that specific share and backend. --- ## Worker fleet concepts ### Worker VM A worker VM (also called a *worker*, `abaca-worker`, or *data-plane worker*) is a short-lived virtual machine that Abacá boots inside the tenant's own network to mount the Manila share and run the actual data transfer. Workers are RHEL VMs with Kopia, NFS utilities, and the `abaca-worker-agent` baked in — no runtime package installation occurs, satisfying the airgap invariant of RHOSO deployments. Plaintext file data and encryption keys exist in worker memory only for the duration of a job; nothing sensitive is written to the worker's disk. Workers are identified in Glance by the `abaca_worker_image=1` property. The `abaca-dev` CLI refuses to boot any image without this property to prevent accidentally using an unprepared image in an airgapped environment. --- ### Capacity slots Capacity slots (also called *slots* or *worker slots*) are the number of backup or restore jobs a single worker VM can run concurrently. The conductor assigns a new job to a worker only if its current running job count is strictly fewer than its slot count. Terminal-state jobs (`available` or `error`) drop out of the count automatically, so slot release is implicit. The conductor picks workers by counting non-terminal jobs associated with each worker in the database. --- ### Reconciliation Reconciliation (also called the *reconciliation loop* or *reconcile*) is a background process in the conductor that periodically checks for stuck or orphaned jobs, dead workers, and backups whose Kopia snapshots have been pruned, keeping the database consistent with the actual state of your repositories. Reconciliation is how Abacá self-heals after partial failures without requiring operator intervention for transient faults. --- ## Manila concepts ### DHSS — Driver Handles Share Servers DHSS (*driver-handles-share-servers*) is a Manila share-type capability that determines whether Manila itself provisions and manages the network infrastructure (share server, IP address, export) for each share. It has two modes: - **DHSS=true**: Manila creates a dedicated share server in a tenant-specified share network. Each share has a `share_network_id` pointing to a Neutron network. Abacá must hot-plug a Neutron port into that network to reach the share export. - **DHSS=false**: The backend exposes shares on a provider or storage network that the worker fleet can reach permanently. No per-job network provisioning is needed. Abacá supports both modes simultaneously; the correct network attachment strategy is resolved per share at job time based on the `dhss` attribute detected from the share type. --- ### Network attachment strategy The network attachment strategy (also called *network strategy*, `AttachToShareNetwork`, or `StaticReachability`) is the method Abacá uses to connect a worker VM to the tenant's share network so it can mount the share. | Strategy | Selected when | What happens | |---|---|---| | `attach-to-share-network` | Share has `share_network_id` (DHSS=true) | A Neutron port is created on the tenant share network and hot-plugged into the worker VM. The worker's new IP is used for the Manila `access_type=ip` access rule. The port is deleted idempotently on cleanup. | | `static-reachability` | DHSS=false backend | The share export is already reachable from the worker fleet on a provider/storage network. No port is provisioned; the job verifies at least one export location is present and fails fast if not. | --- ### Snapshot access strategy The snapshot access strategy (also called *snapshot strategy*, `snapdir`, `clone-from-snapshot`, or `live backup`) is the method Abacá uses to get a consistent, read-only view of a share's data during backup. Selection is data-driven from share-type capabilities and runtime probes; nothing is hardcoded per vendor. | Strategy | Selected when | Consistency guarantee | |---|---|---| | `snapdir` | Backend exposes a `.snapshot` directory on the live mount (e.g., NetApp ONTAP) | Point-in-time. Cheapest: one Manila snapshot, one live mount, one path traversal. | | `mountable-snapshot` | `mount_snapshot_support=true` on the share type | Point-in-time. Manila snapshot is mounted via its own read-only export. | | `clone-from-snapshot` | `create_share_from_snapshot_support=true` | Point-in-time. A temporary Manila share is cloned from a snapshot, mounted, read, then deleted. Slower; consumes quota. | | `live` | No snapshot capability available | Crash-consistent only. The live share is mounted read-only and read directly. Sets `crash_consistent=true` on the resulting backup. | All live-application backups are crash-consistent (power-loss-equivalent). Application-consistent backups are out of scope for Phase 1. --- ## Metering and operations ### Usage meters Usage meters (also called *usage*, *metering*, or *chargeback metrics*) are per-tenant metrics that track how much data is protected (protected capacity in GB), how many bytes were transferred per job, and restore counts. The conductor's reconciliation loop samples usage on a deployment-level cadence and writes it into the `usage_meters` table. These metrics are the basis for chargeback and billing integration. --- ### Enrollment (target enrollment) See [Enrollment](#enrollment) above. The `target_enroll` job type is the concrete job that drives the enrollment process through the standard state machine, with no-op passes on the network, source, and release phases. --- ## API design patterns ### Asynchronous operations Every resource-mutating operation in Abacá — creating a backup, requesting a restore, registering a target — is asynchronous. The API returns immediately with a reference to a **job** resource. You poll the job's `state` field to follow progress through the state machine. This design lets the conductor manage worker fleet scheduling, retries, and cleanup without blocking your HTTP client. ### Secret references, not secret values Abacá never accepts or stores raw credentials. When registering a backup target, you supply Barbican hrefs (URI references to secrets you have already stored in Barbican). The API layer stores only these hrefs; the conductor fetches the actual secret payloads from Barbican at job dispatch time and passes them to the worker exclusively via environment variables. ### Keystone authentication All API endpoints are authenticated via Keystone. The `abaca-api` service registers under the Keystone service type `share-backup`. You obtain a Keystone token and pass it as a bearer token on every request. The `python-abacaclient` library handles token acquisition and renewal for you when you use the `openstack share protection …` CLI commands. ### RBAC and multi-tenancy Abacá enforces oslo.policy RBAC. Tenant resources (targets, policies, backups, restores, jobs) are scoped to the Keystone project. Operators have access to admin-scoped fleet and coverage surfaces. A tenant cannot read or enumerate another tenant's resources, and — by design — operators cannot read the encrypted contents of any tenant's backups. ### Repository as the source of truth The Abacá database is bookkeeping only — it holds no secret material, no file paths, and no repository internals. The Kopia repository in your S3 bucket is the authoritative record. If the database is lost, state can be rebuilt from the repository using the `abaca-manage` recovery tooling. This means your backup data is durable independently of the Abacá control plane. ## Examples ### Inspect a job's state machine progression After you trigger any asynchronous operation, the API returns a job ID. Poll it to watch the job advance through states. ```bash # Using python-abacaclient (openstack share protection sub-commands) openstack share protection job show ``` Expected output when the job is transferring data: ``` +------------------+-----------------------------------------------+ | Field | Value | +------------------+-----------------------------------------------+ | id | a3f1c2d4-... | | job_type | backup | | state | transferring | | progress | 42 | | error_category | None | | worker_id | w7e2b9f1-... | +------------------+-----------------------------------------------+ ``` Expected output when the job completes: ``` +------------------+-----------------------------------------------+ | Field | Value | +------------------+-----------------------------------------------+ | id | a3f1c2d4-... | | job_type | backup | | state | available | | progress | 100 | | error_category | None | +------------------+-----------------------------------------------+ ``` --- ### Interpret a failed job When a job reaches `error`, check `error_category` to determine who needs to act. ```bash openstack share protection job show ``` ``` +------------------+-----------------------------------------------+ | Field | Value | +------------------+-----------------------------------------------+ | state | error | | error_code | BUCKET_OBJECT_LOCK_MISSING | | error_detail | Bucket does not have object lock enabled. | | error_category | tenant_action_required | +------------------+-----------------------------------------------+ ``` A `tenant_action_required` error means you must fix your bucket or credentials — in this case, object lock was not enabled at bucket creation time and the bucket must be recreated. --- ### Check which snapshot strategy was selected for a share Abacá resolves the snapshot strategy at job time from the share's capabilities. The `share_capabilities` probe results are visible on the share's capability record. ```bash openstack share protection share-capability show ``` ``` +----------------------+--------+ | Field | Value | +----------------------+--------+ | share_proto | NFS | | dhss | True | | root_read | pass | | chown | pass | | snapdir | pass | | snapshot_support | True | +----------------------+--------+ ``` With `snapdir=pass`, the `snapdir` strategy is selected — the cheapest and most point-in-time-consistent option. The resulting backup's `crash_consistent` field will be `false`. --- ### List backup targets and their enrollment status ```bash openstack share protection target list ``` ``` +--------------------------------------+-----------+--------+----------+ | ID | Name | Status | Bucket | +--------------------------------------+-----------+--------+----------+ | 9f3a1b2c-... | prod-s3 | ready | my-bkp | | 2d4e5f6a-... | dev-s3 | preflight_failed | dev-bkp | +--------------------------------------+-----------+--------+----------+ ``` A `preflight_failed` target cannot accept backups. Show the detail to find out which check failed: ```bash openstack share protection target show 2d4e5f6a-... ``` The `preflight_detail` field contains per-check results including the `lifecycle_rules` or `object_lock` check that caused the failure. --- ### Restore a single file from a backup To restore only a sub-path (a directory or single file) rather than the whole share, supply the `sub_path` parameter. This is a repository-relative path — it is the path as Kopia recorded it, not a filesystem path on any host. ```bash openstack share protection restore create \ --backup-id \ --mode new_share \ --sub-path /reports/2024/january.csv ``` The restore job walks the same state machine as a full restore; poll the returned job ID to follow progress. ## Related concepts - **Authenticate and call the REST API** — how to obtain a Keystone token, locate the `share-backup` endpoint in the service catalog, and structure authenticated requests. - **Register a backup target** — step-by-step guide to creating Barbican secrets, running preflight checks, and completing enrollment before your first backup. - **Create a protection policy** — how to bind a share to a target, set a cron schedule, and configure retention rules. - **Run an on-demand backup** — how to trigger a backup outside the policy schedule and poll the job to completion. - **Restore a share from backup** — how to choose a restore mode, supply a sub-path, and verify fidelity of the restored data. - **Monitor fleet health and coverage** — how to use the admin fleet surface to inspect worker states, capacity slots, and reconciliation status. - **Recover bookkeeping from repositories after database loss** — how the Kopia repository serves as the source of truth and how to use `abaca-manage` to rebuild the database index. - **Deploy the control plane on OpenShift** — how `abaca-api` and `abaca-conductor` are deployed in the `abaca` namespace and registered in the Keystone catalog. - **Build and register worker VM images** — the airgap invariant, the `abaca_worker_image=1` Glance property gate, and the `abaca-dev` CLI workflow for image preparation.