--- title: REST Endpoints product: trilio-share-protection doc_type: guide version: master source: git2docs (code-derived, validation-filtered) canonical: https://git2docs.com/murali-balcha/docs/trilio-share-protection/trilio-share-protection-/rest-endpoints --- # REST Endpoints _All HTTP endpoints with method, path, request schema, and response schema_ ## Overview This page is the complete HTTP API reference for Trilio Share Protection (Abacá). It covers every endpoint exposed by `abaca-api` — the single entry point for all product operations — organized by resource type: targets, policies, backups, restores, jobs, usage, and admin operations. All paths are relative to the `/v1` base path. Use this reference to understand the exact HTTP method, path, request body fields, response shapes, and error codes for each operation before writing client code or automation scripts. ## Prerequisites Before calling the API you need: - A running Abacá control plane (`abaca-api` reachable, `abaca-conductor` running) - OpenStack ≥ 2023.1 (Antelope) or RHOSO 18 with Keystone, Manila, and Barbican available - A valid Keystone token scoped to your project (all tenant endpoints require it; admin endpoints additionally require the `admin` role) - An S3-compatible bucket (AWS S3, MinIO, Ceph RGW, ODF, or Wasabi) with object lock enabled at creation time — required before you can register a backup target - Barbican secrets for your S3 access key and secret key, and a Kopia repository password — Abacá only stores Barbican hrefs, never the raw credentials - A Keystone trust delegating the necessary permissions to the Abacá service account — obtain the service user ID from `GET /v1/service_info` before creating the trust - `python-abacaclient` if you prefer the `openstack share protection …` CLI instead of raw HTTP ## Installation The API is served by `abaca-api`. Installation of the control plane is covered in the OpenShift deployment guide. To verify the API is reachable and confirm the service identity before enrollment: 1. **Discover the Abacá service user** (unauthenticated — no token required): ```bash curl -s https:///v1/service_info | python3 -m json.tool ``` Expected response: ```json { "service_user": { "id": "a1b2c3d4e5f6...", "name": "abaca" }, "worker_project": { "id": "f6e5d4c3b2a1...", "name": "abaca-workers" } } ``` Record `service_user.id` — you need it when creating the Keystone trust during enrollment. 2. **Obtain a Keystone token** scoped to your project: ```bash export OS_TOKEN=$(openstack token issue -f value -c id) ``` 3. **Confirm authenticated access** by listing your backup targets (empty list is expected on a fresh deployment): ```bash curl -s -H "X-Auth-Token: $OS_TOKEN" \ https:///v1/targets | python3 -m json.tool ``` Expected response: ```json { "targets": [], "pagination": { "limit": 100, "count": 0, "next_marker": null } } ``` ## Configuration ### Authentication All tenant endpoints require a Keystone token passed as `X-Auth-Token`. Admin endpoints (under `/v1/admin/…`) additionally require the caller to hold the `admin` role in Keystone. The `GET /v1/service_info` endpoint is unauthenticated — it exists specifically so clients can self-bootstrap without needing user-list privileges. ### Base URL All paths below are relative to `/v1`. The API does not currently version-negotiate; pin your client to `/v1`. ### Pagination All list endpoints use keyset (cursor) pagination. The parameters are the same across every collection: | Parameter | Type | Default | Maximum | Description | |-----------|------|---------|---------|-------------| | `limit` | integer | `100` | `1000` | Maximum items to return in one page | | `marker` | string | — | — | Return only items whose `id` sorts after this value; use `pagination.next_marker` from the previous response | Every list response body wraps the collection in a `pagination` object: ```json { "": [ … ], "pagination": { "limit": 100, "count": 12, "next_marker": null } } ``` When `next_marker` is non-null, pass it as `?marker=` on the next request to fetch the following page. ### Expired backups `GET /v1/backups` hides backups with `status=expired` by default. A backup becomes `expired` when its Kopia manifest has aged out under the policy's retention rules — it is a tombstone and is no longer restorable. To include expired records, pass `?include_expired=true`. To scope to a specific status exactly, pass `?status=`. ### Retention keys The `retention` object on a policy accepts only the following keys (all map directly to Kopia `--keep-*` flags); values must be non-negative integers: | Key | Effect | |-----|--------| | `keep_latest` | Keep the N most recent snapshots regardless of schedule | | `keep_hourly` | Keep N hourly snapshots | | `keep_daily` | Keep N daily snapshots | | `keep_weekly` | Keep N weekly snapshots | | `keep_monthly` | Keep N monthly snapshots | | `keep_annual` | Keep N annual snapshots | An empty `retention` object (`{}`) or omitting the field entirely means Kopia keeps every snapshot. ### Error shape Every error response uses a consistent JSON body: ```json { "code": "abaca.not_found", "title": "Not Found", "detail": "target abc123 not found", "category": "tenant_action_required" } ``` `category` is either `tenant_action_required` (you need to fix your configuration, credentials, or request) or `operator_action_required` (the infrastructure or service needs attention from an operator). ## Usage ### Tenant endpoints The following sections describe the primary workflows you perform as a tenant. All examples assume `$OS_TOKEN` holds a valid Keystone token and `$API` is set to the base URL, for example `https://abaca.example.com/v1`. --- #### Backup Targets A backup target represents one S3-compatible bucket together with the Barbican secret references needed to access it. Each target holds exactly one Kopia repository. **Register a backup target** `POST /v1/targets` Required fields: `name`, `endpoint`, `bucket`. Include `trust_id`, `barbican_secret_map`, and optionally `repository_password_ref` and `ca_cert_ref` to allow the enrollment job to run immediately. Without `trust_id` and `barbican_secret_map` the target is created with `status: pending` and no enrollment job is dispatched. Request body fields: | Field | Type | Required | Description | |-------|------|----------|-------------| | `name` | string | yes | Human-readable name | | `endpoint` | string | yes | S3 endpoint URL | | `bucket` | string | yes | Bucket name | | `trust_id` | string | no | Keystone trust ID delegating access to Abacá | | `barbican_secret_map` | object | no | `{"access_key": "", "secret_key": ""}` | | `barbican_secret_refs` | array of string | no | Additional Barbican secret hrefs | | `ca_cert_ref` | string | no | Barbican href for a custom CA certificate | | `addressing_mode` | string | no | S3 addressing style (e.g. `path` or `virtual-hosted`) | | `region` | string | no | S3 region hint stored in capabilities | Response: `202 Accepted` with `{"target": Target, "job": Job}` when enrollment is dispatched, or `{"target": Target}` when inputs are insufficient to enroll. **List targets** `GET /v1/targets[?limit=N&marker=M]` Response: `200 OK` with paginated `targets` collection. **Show a target** `GET /v1/targets/{target_id}` Response: `200 OK` with a single `Target` object. **Check preflight status** `GET /v1/targets/{target_id}/preflight` Returns `{target_id, status, detail}`. Preflight checks verify bucket reachability, read/write access, addressing style, object lock configuration, and the absence of expiration lifecycle rules. **Update a target** `PUT /v1/targets/{target_id}` or `PATCH /v1/targets/{target_id}` Updatable fields: `name`, `endpoint`, `bucket`, `trust_id`, `ca_cert_ref`, `barbican_secret_refs`. Response: `200 OK` with the updated `Target` object. **Delete a target** `DELETE /v1/targets/{target_id}` Fails with `409 Conflict` if any policies or backups reference the target — detach or delete those first. Returns `204 No Content` on success. --- #### Protection Policies A protection policy binds a Manila share to a backup target with a cron schedule and retention rules. One share can have at most one policy. **Create a policy** `POST /v1/policies` Required fields: `name`, `share_id`, `target_id`, `schedule`. | Field | Type | Required | Description | |-------|------|----------|-------------| | `name` | string | yes | Human-readable name | | `share_id` | string | yes | Manila share UUID | | `target_id` | string | yes | Registered backup target UUID | | `schedule` | string | yes | Cron expression (e.g. `"0 2 * * *"`) | | `retention` | object | no | `keep_*` keys — see Configuration | | `enabled` | boolean | no | Default `true`; set `false` to pause scheduling | | `include_patterns` | array of string | no | Kopia glob patterns for files to include | | `exclude_patterns` | array of string | no | Kopia glob patterns for files to exclude | Response: `201 Created` with the `Policy` object. Returns `409 Conflict` (`abaca.conflict`) when the share already has a policy. **List policies** `GET /v1/policies[?target_id=T&limit=N&marker=M]` Pass `?target_id=` to filter to policies using a specific target. **Show a policy** `GET /v1/policies/{policy_id}` **Update a policy** (partial update) `PATCH /v1/policies/{policy_id}` Editable fields: `name`, `schedule`, `retention`, `enabled`, `include_patterns`, `exclude_patterns`. Identity fields (`share_id`, `target_id`, `id`, `project_id`) are immutable — attempting to change them returns `400 abaca.field_immutable`. **Delete a policy** `DELETE /v1/policies/{policy_id}` Fails with `409 abaca.policy_has_inflight_backup` if a backup triggered by this policy is still in `creating` state. Historical backup records are preserved (their `policy_id` becomes `null`). Returns `204 No Content` on success. --- #### Backups A backup is a point-in-time snapshot of a Manila share stored as an encrypted, deduplicated Kopia snapshot in your S3 bucket. Creating a backup is asynchronous — poll the returned job for progress. **Create an on-demand backup** `POST /v1/backups` Required fields: `share_id`, `target_id`. | Field | Type | Required | Description | |-------|------|----------|-------------| | `share_id` | string | yes | Manila share UUID to back up | | `target_id` | string | yes | Registered backup target UUID | | `policy_id` | string | no | Associate this backup with a policy (set automatically for scheduled backups) | Response: `202 Accepted` with `{"backup": Backup, "job": Job}`. **List backups** `GET /v1/backups[?share_id=S&target_id=T&policy_id=P&status=X&include_expired=true&limit=N&marker=M]` All filter parameters compose with AND. Expired backups (`status=expired`) are hidden by default. **Show a backup** `GET /v1/backups/{backup_id}` **Delete a backup** `DELETE /v1/backups/{backup_id}` Deletes the backup record. Returns `204 No Content`. --- #### Restores A restore recreates files from a backup onto a new share (the default) or back onto the original share. Restoring to the original share is destructive and requires `force: true`. **Create a restore** `POST /v1/restores` Required field: `backup_id`. | Field | Type | Required | Description | |-------|------|----------|-------------| | `backup_id` | string | yes | UUID of the backup to restore | | `mode` | string | no | `"new_share"` (default) or `"in_place"` | | `force` | boolean | no | Required `true` when `mode` is `"in_place"` | | `sub_path` | string | no | Restore only this path within the backup | | `target_share_id` | string | no | Destination share UUID for `new_share` mode | Returns `400 abaca.force_required` if `mode=in_place` and `force` is not `true`. Response: `202 Accepted` with `{"restore": Restore, "job": Job}`. **List restores** `GET /v1/restores[?limit=N&marker=M]` **Show a restore** `GET /v1/restores/{restore_id}` **Delete a restore** `DELETE /v1/restores/{restore_id}` Deletes the restore record (does NOT undo the data written to the share). Fails with `409 abaca.restore_in_progress` if the restore is still pending. Returns `204 No Content` on success. --- #### Jobs Jobs are the unit of work for every asynchronous operation (backup, restore, enrollment, maintenance). They are read-only for tenants. **List jobs** `GET /v1/jobs[?target_id=T&resource_id=R&limit=N&marker=M]` Pass `?resource_id=` or `?resource_id=` to fetch jobs for a specific resource. **Show a job** `GET /v1/jobs/{job_id}` Response fields: | Field | Type | Description | |-------|------|-------------| | `id` | string | Job UUID | | `job_type` | string | `backup`, `restore`, `target_enroll`, or maintenance type | | `state` | string | Current state (see job state machine below) | | `progress` | integer | Percent complete (0–100) | | `resource_id` | string | UUID of the associated backup, restore, or target | | `share_id` | string | Associated Manila share UUID | | `target_id` | string | Associated backup target UUID | | `worker_id` | string | Worker VM executing the job | | `attempts` | integer | Number of execution attempts | | `executed_by` | string | `conductor:` in production | | `error` | object\|null | `{code, detail, category}` on failure | | `created_at` | string (ISO 8601) | Creation timestamp | **Job state machine** Every job moves through these states in order: ``` queued → provisioning_network → provisioning_source → connecting_repository → transferring → finalizing → releasing → available ↘ error ``` --- #### Usage **Tenant usage summary** `GET /v1/usage` Returns per-tenant metrics for chargeback and capacity planning: ```json { "project_id": "", "protected_shares": 4, "backup_count": 28, "protected_bytes": 107374182400, "stored_bytes": 21474836480 } ``` --- #### Service Info (unauthenticated) **Get service identity** `GET /v1/service_info` No authentication required. Returns the Keystone user ID and project of the Abacá service account. Use `service_user.id` as the trustee when creating the Keystone trust during target enrollment. --- ### Admin endpoints All `/v1/admin/…` endpoints require the `admin` Keystone role. **Fleet workers** | Method | Path | Description | |--------|------|-------------| | `GET` | `/v1/admin/workers` | List all worker VMs in the fleet | | `POST` | `/v1/admin/workers/{worker_id}/drain` | Stop assigning new jobs to the worker; let running jobs finish | | `POST` | `/v1/admin/workers/{worker_id}/retire` | Mark the worker as retired; the conductor will not dispatch to it | **All-tenant jobs** | Method | Path | Description | |--------|------|-------------| | `GET` | `/v1/admin/jobs` | List jobs across all tenants | | `POST` | `/v1/admin/jobs/{job_id}/retry` | Re-queue a job that is in `error` state | | `POST` | `/v1/admin/jobs/{job_id}/cancel` | Cancel an in-progress job | **Coverage** | Method | Path | Description | |--------|------|-------------| | `GET` | `/v1/admin/coverage` | Per-share protection coverage across all tenants | **All-tenant targets** | Method | Path | Description | |--------|------|-------------| | `GET` | `/v1/admin/targets` | List backup targets across all tenants | **Scheduling** | Method | Path | Description | |--------|------|-------------| | `GET` | `/v1/admin/scheduling` | Get scheduling status (`{"paused": false}`) | | `POST` | `/v1/admin/scheduling/pause` | Pause policy-triggered scheduled backups | | `POST` | `/v1/admin/scheduling/resume` | Resume scheduled backups | **Reconciliation** | Method | Path | Description | |--------|------|-------------| | `GET` | `/v1/admin/reconciliation` | Get reconciliation loop status | | `POST` | `/v1/admin/reconciliation/trigger` | Manually trigger a reconciliation run | ## Examples ### Register a backup target and poll enrollment This is the first step in any Abacá workflow. You register the bucket, supply your Barbican secret references and Keystone trust, then poll the enrollment job until it reaches `available`. ```bash # Step 1 — Register the target curl -s -X POST "$API/targets" \ -H "X-Auth-Token: $OS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "my-production-target", "endpoint": "https://s3.us-east-1.amazonaws.com", "bucket": "my-backup-bucket", "trust_id": "e3b0c44298fc...", "barbican_secret_map": { "access_key": "https://barbican.example.com/v1/secrets/aaaa-1111", "secret_key": "https://barbican.example.com/v1/secrets/bbbb-2222" } }' | python3 -m json.tool ``` Expected response (`202 Accepted`): ```json { "target": { "id": "tgt-0001-uuid", "name": "my-production-target", "endpoint": "https://s3.us-east-1.amazonaws.com", "bucket": "my-backup-bucket", "status": "pending", "trust_id": "e3b0c44298fc...", "preflight_at": null, "created_at": "2024-06-01T10:00:00" }, "job": { "id": "job-enroll-uuid", "job_type": "target_enroll", "state": "queued", "progress": 0 } } ``` ```bash # Step 2 — Poll the enrollment job JOB_ID="job-enroll-uuid" until [ "$(curl -s -H "X-Auth-Token: $OS_TOKEN" "$API/jobs/$JOB_ID" | python3 -c "import sys,json; print(json.load(sys.stdin)['state'])" 2>/dev/null)" = "available" ]; do echo "Waiting…" sleep 5 done echo "Enrollment complete" ``` --- ### Create a protection policy with retention Once the target is enrolled, bind a Manila share to it with a daily backup schedule keeping 7 daily and 4 weekly snapshots. ```bash curl -s -X POST "$API/policies" \ -H "X-Auth-Token: $OS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "daily-7d4w", "share_id": "manila-share-uuid", "target_id": "tgt-0001-uuid", "schedule": "0 2 * * *", "retention": { "keep_daily": 7, "keep_weekly": 4 }, "enabled": true }' | python3 -m json.tool ``` Expected response (`201 Created`): ```json { "id": "pol-0001-uuid", "name": "daily-7d4w", "share_id": "manila-share-uuid", "target_id": "tgt-0001-uuid", "schedule": "0 2 * * *", "retention": { "keep_daily": 7, "keep_weekly": 4 }, "enabled": true, "include_patterns": [], "exclude_patterns": [], "last_run": null, "created_at": "2024-06-01T10:05:00" } ``` --- ### Run an on-demand backup Trigger a backup immediately outside the scheduled window. ```bash curl -s -X POST "$API/backups" \ -H "X-Auth-Token: $OS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "share_id": "manila-share-uuid", "target_id": "tgt-0001-uuid" }' | python3 -m json.tool ``` Expected response (`202 Accepted`): ```json { "backup": { "id": "bak-0001-uuid", "share_id": "manila-share-uuid", "target_id": "tgt-0001-uuid", "status": "creating", "kopia_snapshot_id": null, "crash_consistent": false, "size_bytes": null, "created_at": "2024-06-01T10:10:00" }, "job": { "id": "job-backup-uuid", "job_type": "backup", "state": "queued", "progress": 0 } } ``` Poll progress by watching `job.state` advance through the state machine: ```bash curl -s -H "X-Auth-Token: $OS_TOKEN" "$API/jobs/job-backup-uuid" | python3 -m json.tool ``` Example in-progress response: ```json { "id": "job-backup-uuid", "job_type": "backup", "state": "transferring", "progress": 42, "error": null } ``` --- ### Restore a backup to a new share ```bash curl -s -X POST "$API/restores" \ -H "X-Auth-Token: $OS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "backup_id": "bak-0001-uuid", "mode": "new_share" }' | python3 -m json.tool ``` Expected response (`202 Accepted`): ```json { "restore": { "id": "rst-0001-uuid", "backup_id": "bak-0001-uuid", "mode": "new_share", "force": false, "sub_path": null, "status": "pending", "created_at": "2024-06-01T11:00:00" }, "job": { "id": "job-restore-uuid", "job_type": "restore", "state": "queued", "progress": 0 } } ``` ### Restore a single subdirectory in-place In-place restores are destructive and require `force: true`. ```bash curl -s -X POST "$API/restores" \ -H "X-Auth-Token: $OS_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "backup_id": "bak-0001-uuid", "mode": "in_place", "force": true, "sub_path": "data/reports/2024" }' | python3 -m json.tool ``` --- ### List backups for a specific share ```bash curl -s -H "X-Auth-Token: $OS_TOKEN" \ "$API/backups?share_id=manila-share-uuid&limit=5" | python3 -m json.tool ``` Expected response: ```json { "backups": [ { "id": "bak-0001-uuid", "share_id": "manila-share-uuid", "target_id": "tgt-0001-uuid", "status": "available", "kopia_snapshot_id": "kc9a3f2e1b...", "crash_consistent": true, "size_bytes": 2147483648, "stored_bytes": 536870912, "files_count": 14023, "created_at": "2024-06-01T10:15:32" } ], "pagination": { "limit": 5, "count": 1, "next_marker": null } } ``` --- ### Pause and resume scheduled backups (admin) ```bash # Pause all scheduled backups across all tenants curl -s -X POST "$API/admin/scheduling/pause" \ -H "X-Auth-Token: $ADMIN_TOKEN" | python3 -m json.tool # {"paused": true} # Resume curl -s -X POST "$API/admin/scheduling/resume" \ -H "X-Auth-Token: $ADMIN_TOKEN" | python3 -m json.tool # {"paused": false} ``` --- ### Paginating through all jobs (admin) ```bash MARKER="" while true; do URL="$API/admin/jobs?limit=100" [ -n "$MARKER" ] && URL="${URL}&marker=${MARKER}" RESP=$(curl -s -H "X-Auth-Token: $ADMIN_TOKEN" "$URL") MARKER=$(echo "$RESP" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['pagination']['next_marker'] or '')") echo "$RESP" | python3 -c "import sys,json; [print(j['id'], j['state']) for j in json.load(sys.stdin)['jobs']]" [ -z "$MARKER" ] && break done ``` ## Troubleshooting ### `422 Preflight failed` when registering a target **Symptom:** `POST /v1/targets` returns HTTP 422 with `code: abaca.preflight_failed`. **Likely causes:** - The S3 bucket is not reachable from the Abacá control plane - Object lock is not enabled on the bucket (it must be enabled at bucket creation; it cannot be added later) - The bucket has expiration lifecycle rules that would delete backup objects within the retention window - The Barbican secret hrefs are invalid or the Keystone trust lacks the necessary permissions to read them - Incorrect addressing mode (path vs virtual-hosted) for the bucket's S3 provider **Fix:** Review `preflight_detail` in the error response or call `GET /v1/targets/{target_id}/preflight` after the target is created. Correct the bucket configuration or credentials, then re-run enrollment (update the target via `PATCH` and re-POST or trigger re-enrollment through `abaca-dev`). --- ### `409 Conflict` when creating a policy **Symptom:** `POST /v1/policies` returns `409` with `code: abaca.conflict` and the message references `share already has a protection policy`. **Likely cause:** The Manila share UUID you supplied already has a policy in this project. One share can bind to at most one policy. **Fix:** Retrieve the existing policy with `GET /v1/policies?share_id=` (once filtering by `share_id` is available) or list all policies and search for the share ID. Update the existing policy via `PATCH`, or delete it first if you want to change the target. --- ### `400 abaca.field_immutable` when patching a policy **Symptom:** `PATCH /v1/policies/{policy_id}` returns `400` with `code: abaca.field_immutable`. **Likely cause:** The request body includes `share_id`, `target_id`, `id`, `project_id`, or `created_at`, or an unknown field. These fields are immutable once the policy is created because they are baked into every historical backup the policy has created. **Fix:** Remove the immutable or unknown fields from the request body. Only `name`, `schedule`, `retention`, `enabled`, `include_patterns`, and `exclude_patterns` can be changed. --- ### `400 abaca.force_required` when creating an in-place restore **Symptom:** `POST /v1/restores` returns `400` with `code: abaca.force_required`. **Likely cause:** You specified `mode: in_place` without setting `force: true`. In-place restore overwrites the existing share and requires an explicit acknowledgement. **Fix:** Add `"force": true` to the request body. Ensure you understand that this operation is destructive and irreversible. --- ### `409 abaca.restore_in_progress` when deleting a restore record **Symptom:** `DELETE /v1/restores/{restore_id}` returns `409`. **Likely cause:** The restore is still in a non-terminal state (`pending`). A running restore holds a live worker reservation and its record cannot be deleted while the job is active. **Fix:** Wait for the restore job to reach `available` or `error` (poll `GET /v1/jobs/{job_id}`), then retry the delete. If the job appears stuck, an operator can cancel it via `POST /v1/admin/jobs/{job_id}/cancel` or trigger reconciliation via `POST /v1/admin/reconciliation/trigger`. --- ### `409 abaca.policy_has_inflight_backup` when deleting a policy **Symptom:** `DELETE /v1/policies/{policy_id}` returns `409`. **Likely cause:** A backup triggered by this policy is still in `creating` state. **Fix:** Wait for the in-flight backup job to reach a terminal state (`available` or `error`), then retry the delete. Reconciliation's stuck-job sweep will walk hung jobs to `error` on its own cadence if the job is genuinely stuck. --- ### Job stuck in `queued` indefinitely **Symptom:** A backup or restore job stays in `state: queued` and never advances. **Likely causes:** 1. The `abaca-conductor` service is not running or cannot reach the message broker 2. The target associated with the job is not fully enrolled (`trust_id` or `barbican_secret_map` was never set — check target `status`) 3. All worker VMs are at capacity (all slots occupied) **Fix:** 1. Verify `abaca-conductor` health and RabbitMQ connectivity with your operator 2. Check `GET /v1/targets/{target_id}` — if `status` is `pending`, complete enrollment by patching the target with `trust_id` and `barbican_secret_map` 3. An operator can inspect fleet capacity via `GET /v1/admin/workers` and retire or drain workers that are stuck --- ### `error_category: operator_action_required` on a failed job **Symptom:** `GET /v1/jobs/{job_id}` returns `state: error` and `error.category: operator_action_required`. **Likely cause:** The infrastructure or the Abacá service itself encountered a problem — for example, Nova could not boot a worker VM, a network port could not be attached, or the conductor lost its database connection. **Fix:** You do not need to change your configuration. Share the job ID and `error.detail` with your operator. An operator can retry the job via `POST /v1/admin/jobs/{job_id}/retry` once the underlying infrastructure issue is resolved. --- ### `401 Authentication required` **Symptom:** Any endpoint returns `401`. **Likely cause:** The `X-Auth-Token` header is missing, the token has expired, or the token is not scoped to the correct project. **Fix:** Re-issue a token with `openstack token issue` scoped to your project and retry. Keystone tokens have a finite lifetime; ensure your automation refreshes the token before it expires.