Getting Started

This tutorial walks you through installing Trilio Share Protection for OpenStack (Abacá), registering it with Keystone, and making your first API call to create a backup target. By the end you will have a running control plane on RHOSO 18, a Keystone service catalog entry, and a verified round-trip from the OpenStack CLI through the API to the conductor. If you want to understand the architecture before installing, see the Architecture guide; if you are looking for a specific API reference, see the API Reference.

Before you begin, confirm that each of the following is in place:

OpenStack control plane

• Red Hat OpenStack Services on OpenShift (RHOSO) ≥ 18
• OpenShift ≥ 4.14 (hosts the Abacá control plane pods)
• OpenStack Keystone ≥ 2023.1, with trusts enabled and domain-scoped tokens supported
• OpenStack Manila (Shared File System service) — the service Abacá protects
• OpenStack Barbican (Key Manager) — stores S3 credentials and Kopia repository passwords
• OpenStack Nova (Compute) — provisions worker VMs
• OpenStack Neutron (Networking) — manages worker VM network attachment
• OpenStack Glance (Image Service) — stores the worker VM image
• Horizon (optional, for the dashboard plugin)

Infrastructure

• A dedicated MySQL or MariaDB (Galera) instance for Abacá — do not share OpenStack's cell database
• A dedicated RabbitMQ instance for Abacá — do not share the platform's shared broker
• An S3-compatible object storage bucket (Wasabi, MinIO, AWS S3, or equivalent) with versioning enabled and Object Lock enabled at bucket creation time

Build and tooling

• A RHEL ≥ 9 base cloud image and libguestfs on the build host (for worker VM image builds)
• Python ≥ 3.11 on your workstation
• Kopia binary ≥ 0.17.0, FIPS-built and version-pinned
• kubectl / oc with a valid KUBECONFIG pointing at the RHOSO 18 cluster
• Admin credentials for the OpenStack cluster (sourced openrc or clouds.yaml)
• tox ≥ 4.0 (optional, for running tests locally)
• Docker or Podman with Compose (optional, for the local dev stack only)

Access

• KUBECONFIG set to your RHOSO 18 cluster — this is the only credential the install scripts require for the OpenShift side; the admin OpenStack password is read from the osp-secret Secret automatically.

The steps below take you from a bare RHOSO 18 cluster to a verified first API call. Each step maps to a script in deploy/rhoso/; the scripts are idempotent — safe to re-run.

1. Export your kubeconfig

   Copy

   export KUBECONFIG=~/.kube/config-rhoso18
   

2. Run the preflight check

   Copy

   bash deploy/rhoso/00-preflight.sh
   

   Confirms that CRDs, Galera, RabbitMQ, and cert-manager are present and that the admin credentials work. Review any warnings before continuing.

3. Run the one-shot installer

   Copy

   bash deploy/rhoso/install.sh
   

   This runs steps 01–07 in order: Keystone identity and catalog registration, dedicated database and message-bus provisioning, container image build, config rendering, schema migration (abaca-manage db_sync), and pod startup. The script waits for /healthcheck to return 200 before exiting.

4. Build and upload the worker VM image

   Copy

   bash deploy/rhoso/worker/20-worker-image.sh
   

   Installs Kopia, nfs-common, and abaca-worker-agent into a base RHEL cloud image using virt-customize/guestfish, then uploads the result to Glance with the abaca_worker_image=1 property.

5. Install the Abacá CLI on your workstation

   Copy

   pip install python-abacaclient
   

6. Verify the service catalog entry

   Copy

   openstack catalog show share-protection
   

   You should see public, internal, and admin endpoints listed under the share-protection service type.

7. Make your first API call

   Copy

   openstack share protection target list
   

   An empty list (no error) confirms that the API is reachable, Keystone authentication succeeded, and the database schema is in place.

The following procedure expands each quickstart step with the context you need to understand what is happening and how to confirm success.

Step 1 — Run the preflight check

bash deploy/rhoso/00-preflight.sh

The script verifies that the required OpenShift CRDs (KeystoneService, KeystoneEndpoint, MariaDBDatabase, MariaDBAccount, TransportURL) are registered, that a Galera and RabbitMQ instance are reachable, that cert-manager is installed, and that your oc session can authenticate. It also reports whether Manila is present — Abacá can install without Manila, but you will not be able to create backup policies until it is enabled.

✅ Success: The script exits 0 and prints Preflight: all checks passed.

Step 2 — Register the Keystone identity

bash deploy/rhoso/01-identity.sh

Creates a KeystoneService custom resource in the control-plane namespace. The keystone-operator reconciles this CR and creates:

• An abaca Keystone service user with the service and admin roles
• A share-protection service catalog entry
• A dedicated worker project for Abacá's per-Domain service VMs
• A generated service-account password stored in a Kubernetes Secret

Abacá follows standard OpenStack conventions: every API call from tenants and operators must carry a Keystone token, and the service account is used for trust-delegation operations (Barbican secret reads, Manila access-rule management) on behalf of tenants.

✅ Success: openstack service show share-protection returns a record with type: share-protection.

Step 3 — Register the service catalog endpoints

bash deploy/rhoso/02-catalog.sh

Applies a KeystoneEndpoint CR that registers the public (OpenShift Route) and internal (Service) endpoints for share-protection. RHOSO 18 does not use an admin interface for application services, so only two endpoint interfaces are created.

✅ Success: openstack endpoint list --service share-protection shows entries for public and internal.

Step 4 — Provision the dedicated database

bash deploy/rhoso/03-database.sh

Applies MariaDBDatabase and MariaDBAccount CRs against Abacá's own Galera instance (not the OpenStack cell database). Using a dedicated database is required: a backup service must not be able to fill or destabilize the database that core OpenStack services depend on. The script defaults to Abacá's own Galera; to share infrastructure with another Trilio service, pass explicit overrides:

ABACA_GALERA_NAMESPACE=trilio-openstack \
ABACA_GALERA_INSTANCE=trilio-galera-cluster \
    bash deploy/rhoso/03-database.sh

✅ Success: The MariaDBDatabase CR reaches Ready state.

Step 5 — Provision the dedicated message bus

bash deploy/rhoso/04-messaging.sh

Applies a TransportURL CR against Abacá's own RabbitMQ instance. A dedicated broker is required for the same isolation reasons as the database, and because the RHOSO shared broker exposes a single administrator user on vhost / — Abacá passes its transport URL to worker VMs on tenant-reachable networks, which must not carry platform-wide credentials. The TransportURL CR causes the operator to publish a ready-made transport_url string into a Secret.

The broker carries only Barbican secret hrefs and trust IDs — never plaintext secret material.

✅ Success: The TransportURL CR reaches Ready state and the Secret it produces contains a transport_url key.

Step 6 — Build the container images

bash deploy/rhoso/05-build.sh

Triggers an in-cluster OpenShift BuildConfig that builds the abaca-api and abaca-conductor images from source and pushes them into an ImageStream. No local container runtime or registry credentials are required — everything runs inside the cluster.

✅ Success: Both BuildConfig runs complete with Complete phase and the ImageStream shows a new tag.

Step 7 — Render configuration

bash deploy/rhoso/06-config.sh

Renders abaca.conf (oslo.config INI format) and database TLS settings into secret/abaca-config-data in the abaca namespace. The config references the database and RabbitMQ credentials by Secret projection — no passwords appear on the command line. The rendered file includes TLS settings for the database connection (?read_default_file=/etc/my.cnf, ssl-ca) and an explicit cafile for keystone_authtoken, because the UBI9 Python image trusts certifi rather than the system store.

Step 8 — Apply manifests, run migrations, and start pods

bash deploy/rhoso/07-deploy.sh

This step applies Kustomize manifests in two phases to ensure the database schema exists before the API and conductor pods start:

1. All manifests except the abaca-api and abaca-conductor Deployments are applied first (ConfigMaps, Secrets, Services, Route, RBAC, and the db-sync Job).
2. The script waits for the db-sync Job to complete. The Job runs abaca-manage db_sync, which applies all pending Alembic migrations. Migrations are additive-only — they never drop columns or tables.
3. Once db_sync succeeds, the Deployments are applied and the script waits for /healthcheck on the Route to return HTTP 200.

The two-phase approach is necessary because the ImageStream trigger on the Deployments causes OpenShift to start pods immediately when a new image is pushed — before db_sync has run. Phase 1 + explicit wait prevents the app from starting against an un-migrated schema.

✅ Success: oc -n abaca get pods shows abaca-api-* and abaca-conductor-* pods in Running state. curl -sk https://<route>/healthcheck returns {"status": "ok"}.

Step 9 — Build and upload the worker VM image

bash deploy/rhoso/worker/20-worker-image.sh

Uses virt-customize and guestfish on the build host to customize a RHEL ≥ 9 base cloud image: installs Kopia (version-pinned, FIPS-built), nfs-common, and abaca-worker-agent, then uploads the image to Glance with the abaca_worker_image=1 property. Worker VMs must not fetch packages from the internet at boot — everything is baked in at this step. The abaca_worker_image=1 property is required; abaca-dev worker-boot refuses images that lack it.

The script is idempotent: if an image with the same name and Kopia version already exists in Glance, it skips the upload unless you pass --force.

✅ Success: openstack image show abaca-worker-<version> returns the image record and properties includes abaca_worker_image=1.

Step 10 — Register the bootstrap Domain's service project

The installer registers the bootstrap Domain automatically: 01-identity.sh resolves ABACA_BOOTSTRAP_DOMAIN to a Domain ID and ABACA_WORKER_PROJECT to a project ID and records them in configmap/abaca-domain-map. The 07-deploy.sh step writes that mapping into the database once the schema exists.

Verify what was recorded:

oc -n abaca get cm abaca-domain-map -o jsonpath='{.data}'

For every additional Keystone Domain that will use Abacá, you must register its service project manually. Each Domain needs a row that maps it to the project where its worker VMs, S3 credentials, and Barbican secrets will reside:

POD=$(oc -n abaca get pods -l app.kubernetes.io/component=conductor \
    --field-selector=status.phase=Running -o name | head -1 | cut -d/ -f2)

oc -n abaca exec "$POD" -- abaca-manage \
    --config-file /etc/abaca/abaca.conf \
    domain_register \
    --domain-id <domain-id> \
    --service-project-id <project-id>

Confirm the registration:

oc -n abaca exec "$POD" -- abaca-manage \
    --config-file /etc/abaca/abaca.conf \
    domain_list

Important: domain_register is idempotent for correcting the service project ID, but use caution with --key-custody-project-id and --clear-key-custody — changing the key custody project orphans existing Kopia repository passwords already stored in Barbican, which you will only discover at the next restore attempt.

✅ Success: domain_list shows your Domain with a non-null service_project_id.

Step 11 — (Optional) Install the Horizon dashboard plugin

The dashboard plugin is a gated step because it modifies the running Horizon pod:

bash deploy/rhoso/08-dashboard.sh --build-only   # inspect the build first
bash deploy/rhoso/08-dashboard.sh                # repoint Horizon to the new image

Before running 08-dashboard.sh, you must have a trusted TLS certificate on the Abacá public Route — Horizon resolves share-protection at the public endpoint and will render empty panels if the certificate is not trusted inside the cluster:

bash deploy/rhoso/10-public-cert.sh --dry-run    # review the change
bash deploy/rhoso/10-public-cert.sh              # apply the cert

The previous Horizon image is recorded in a ConfigMap so you can revert:

bash deploy/rhoso/08-dashboard.sh --revert

✅ Success: Logging in to Horizon as a tenant user shows a Share Protection panel in the Project menu.

The examples below assume:

• You have completed all install steps above.
• Your shell has valid OpenStack credentials (OS_CLOUD or a sourced openrc).
• python-abacaclient is installed (pip install python-abacaclient).
• The share-protection endpoint is in your service catalog.

Example 1 — Verify the service catalog with the OpenStack CLI

openstack catalog show share-protection

Expected output (URLs will reflect your cluster's Route and Service addresses):

+-----------+--------------------------------------------------------------+
| Field     | Value                                                        |
+-----------+--------------------------------------------------------------+
| endpoints | public: https://abaca.apps.cluster.example.com               |
|           | internal: http://abaca.abaca.svc.cluster.local:9797          |
| id        | <service-uuid>                                               |
| name      | abaca                                                        |
| type      | share-protection                                             |
+-----------+--------------------------------------------------------------+

Example 2 — List backup targets (empty on first install)

This is the minimal smoke test: a successful response confirms Keystone authentication, service catalog discovery, and database connectivity all work.

openstack share protection target list

Expected output on a fresh install:

+----+------+--------+
| ID | Name | Status |
+----+------+--------+
+----+------+--------+

Example 3 — Call the REST API directly with a Keystone token

This example shows how to obtain a token and call the API, which is useful for scripting and for verifying the endpoint independently of the CLI plugin.

# Obtain a Keystone token
TOKEN=$(openstack token issue -f value -c id)

# Discover the public endpoint from the catalog
ABACA_URL=$(openstack catalog show share-protection -f json \
    | python3 -c "
import json, sys
data = json.load(sys.stdin)
for ep in data['endpoints']:
    if ep['interface'] == 'public':
        print(ep['url'])
        break
")

# Call the versioned API root
curl -s -H "X-Auth-Token: $TOKEN" "${ABACA_URL}/v1/targets"

Expected output:

{"targets": [], "links": {"next": null, "previous": null}}

Example 4 — Check control plane pod health

Use this to confirm that both the API and conductor pods are running and healthy after installation.

oc -n abaca get pods -l app.kubernetes.io/part-of=abaca

Expected output:

NAME                              READY   STATUS      RESTARTS   AGE
abaca-api-<hash>                  1/1     Running     0          5m
abaca-conductor-<hash>            1/1     Running     0          5m
db-sync-<hash>                    0/1     Completed   0          6m

Example 5 — Verify the database migration succeeded

POD=$(oc -n abaca get pods -l app.kubernetes.io/component=conductor \
    --field-selector=status.phase=Running -o name | head -1 | cut -d/ -f2)

oc -n abaca exec "$POD" -- abaca-manage \
    --config-file /etc/abaca/abaca.conf \
    db_sync --check

Expected output:

Database schema is up to date (head revision).

Example 6 — Verify the worker image in Glance

openstack image list --property abaca_worker_image=1

Expected output:

+--------------------------------------+------------------------------+--------+
| ID                                   | Name                         | Status |
+--------------------------------------+------------------------------+--------+
| <image-uuid>                         | abaca-worker-<version>       | active |
+--------------------------------------+------------------------------+--------+

Example 7 — Local developer stack (no OpenStack required)

If you want to explore the API locally without a real OpenStack cluster, bring up the Docker Compose dev stack. The API starts in --noauth mode with SQLite and in-memory fakes for all OpenStack clients.

cd deploy
docker compose -f docker-compose.dev.yml up --build

Once running (API on http://localhost:9797), drive a fake end-to-end flow:

# Register a backup target
curl -s -XPOST localhost:9797/v1/targets \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "primary",
    "endpoint": "https://s3.example.com",
    "bucket": "tenant-abaca",
    "barbican_secret_refs": ["https://barbican/secrets/pw"],
    "trust_id": "trust-1"
  }'

# Create a protection policy for a share
curl -s -XPOST localhost:9797/v1/policies \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "nightly",
    "share_id": "share-uuid-0001",
    "target_id": "<TARGET_ID>",
    "schedule": "0 2 * * *",
    "retention": {"daily": 7}
  }'

# Request an on-demand backup
curl -s -XPOST localhost:9797/v1/backups \
  -H 'Content-Type: application/json' \
  -d '{"share_id": "share-uuid-0001", "target_id": "<TARGET_ID>"}'

To run the functional smoke test that exercises the full state machine with in-memory fakes and FakeKopia:

python -m venv .venv && . .venv/bin/activate
pip install -e ".[test]" -e ./python-abacaclient -e ./abaca-dashboard
pytest abaca/tests/functional/test_smoke.py -q

Use the following reference to diagnose the most common failures during initial installation and first API calls. Each entry follows the pattern: symptom → likely cause → fix.

openstack catalog show share-protection returns No endpoint found

Likely cause: Step 02 (02-catalog.sh) did not complete, or the KeystoneEndpoint CR has not been reconciled yet.

Fix:

1. Check the CR status: oc -n openstack get keystoneendpoint -l app.kubernetes.io/part-of=abaca -o yaml and look for error conditions.
2. Re-run bash deploy/rhoso/02-catalog.sh. The script is idempotent.
3. If the CR is present but the endpoint is still missing, check the keystone-operator pod logs: oc -n openstack logs -l app=keystone-operator --tail=50.

abaca-api-* pod is in CrashLoopBackOff

Likely cause: Database migration has not run (schema is missing or at the wrong revision), or the abaca-config-data Secret is malformed.

Fix:

1. Inspect the pod logs: oc -n abaca logs -l app.kubernetes.io/component=api --tail=100.
2. Check whether db_sync completed: oc -n abaca get jobs db-sync. If it failed, re-run bash deploy/rhoso/07-deploy.sh. The Job has backoffLimit: 0, so a single failure is visible immediately without retry noise.
3. If the error is a config syntax problem, re-run bash deploy/rhoso/06-config.sh to re-render and re-apply the Secret, then delete the failing pod to force a restart.

db_sync Job fails with MySQL error 1050 (Table already exists)

Likely cause: An application pod started before db_sync ran (triggered by the ImageStream push) and called create_all(), creating tables without an Alembic version stamp. Subsequent Alembic runs see the tables but no stamp and attempt to create them again.

Fix:

1. Do not drop the database if it contains real data. create_all() does not alter existing tables, so typically only the newest table exists out of schema.
2. Identify which columns are missing by comparing information_schema.columns against the migration output.
3. Add the missing columns by hand to match what the migration produces.
4. Stamp the version manually: UPDATE alembic_version SET version_num='<head-revision>';
5. Re-run bash deploy/rhoso/07-deploy.sh.

/healthcheck returns 502 or 500

Likely cause: The pod started but cannot reach the database or RabbitMQ.

Fix:

1. Check pod logs: oc -n abaca logs -l app.kubernetes.io/component=api --tail=100.
2. Verify the MariaDBDatabase CR is Ready: oc -n abaca get mariadatabase abaca.
3. Verify the TransportURL CR is Ready: oc -n abaca get transporturl abaca.
4. If credentials are wrong, re-run bash deploy/rhoso/06-config.sh and delete the pods to force a config reload.

openstack share protection target list returns an SSL or certificate error

Likely cause: The Abacá public Route is serving a certificate that the client or Horizon cannot verify. The UBI9 Python image trusts certifi, not the system store; the API pod must have REQUESTS_CA_BUNDLE / SSL_CERT_FILE set to abaca-ca-bundle.

Fix:

1. For CLI calls from your workstation, add --insecure temporarily to confirm it is a certificate issue, then fix the certificate properly.
2. Run bash deploy/rhoso/10-public-cert.sh to provision a trusted certificate on the Route. This is a prerequisite for the Horizon plugin and is required before the public endpoint is usable from inside the cluster.
3. Verify the pods export the correct CA bundle: oc -n abaca exec <pod> -- env | grep REQUESTS_CA_BUNDLE.

Horizon shows empty Share Protection panels after 08-dashboard.sh

Likely cause: The public Route does not have a trusted certificate, so Horizon's internal requests to the share-protection public endpoint fail silently.

Fix:

1. Run bash deploy/rhoso/10-public-cert.sh before 08-dashboard.sh — this is a prerequisite, not optional polish.
2. If 10-public-cert.sh has already run, check the Horizon pod logs for SSL errors related to the share-protection endpoint.
3. If the Horizon image was rebuilt by a platform reconfigure and wiped the plugin, re-run bash deploy/rhoso/08-dashboard.sh.

abaca-manage domain_list shows no rows, and GET /v1/domain reports is_service_project: false for every project

Likely cause: The abaca-domain-map ConfigMap was not applied, or 07-deploy.sh ran before 01-identity.sh resolved the Domain and project IDs.

Fix:

1. Check the ConfigMap: oc -n abaca get cm abaca-domain-map -o jsonpath='{.data}'.
2. If it is empty or missing, re-run bash deploy/rhoso/01-identity.sh (which resolves the IDs) followed by bash deploy/rhoso/07-deploy.sh (which writes the row to the database).
3. For additional Domains, register them manually with abaca-manage domain_register as shown in Step 10 of the procedure above.

Worker VM boots but never registers with the conductor

Likely cause: The worker VM cannot reach the RabbitMQ port on the conductor's network, or the abaca.conf baked into the image does not match the conductor's transport URL.

Fix:

1. SSH into the worker VM and check the agent logs: journalctl -u abaca-worker-agent.
2. Verify that /etc/abaca/abaca.conf on the VM contains the same transport_url as the conductor's config: oc -n abaca exec <conductor-pod> -- cat /etc/abaca/abaca.conf | grep transport_url.
3. Verify the worker VM can reach the RabbitMQ port from its network (the tenant network must have egress to the RabbitMQ service, either via a Neutron router or a host-NAT path).

Worker boot fails with an abaca_worker_image=1 gate error

Likely cause: The Glance image you are using does not have the abaca_worker_image=1 property set, which means it was not built by 20-worker-image.sh or it predates the property requirement.

Fix: Rebuild the worker image: bash deploy/rhoso/worker/20-worker-image.sh --force. Verify the property after upload: openstack image show <image-id> -f json | python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('properties',{}).get('abaca_worker_image'))".