Prerequisites

This page describes the environment Abacá requires before you run the install scripts. Abacá drives Manila, Nova, Neutron, Barbican, and an S3-compatible object store on your behalf — and most of these dependencies fail quietly when unmet, producing a share that never snapshots, a worker VM that never registers, or a dashboard panel that renders an empty table rather than a clear error. Reading this page before you deploy saves significant debugging time. The requirements are grouped by service area so you can verify them independently and understand the symptom each unmet requirement produces.

The following services and versions must be present and functional before you install Abacá. "Functional" means the service can actually perform its core operation — a pod reporting Ready proves very little, since Keystone's readiness probe passes without a working database.

Platform

• Red Hat OpenStack Services on OpenShift (RHOSO) 18 or later
• OpenShift 4.14 or later (hosts the Abacá control plane)
• Python 3.11 or later (3.12 recommended)

Required OpenStack services

• Keystone — with trusts enabled and domain-scoped tokens supported (2023.1 or later)
• Barbican — OpenStack Key Manager; a hard dependency, not optional
• Manila — Shared File System service, with snapshot support and IP-based access rules on the backend
• Nova — Compute, for booting worker VMs
• Neutron — Networking, for binding worker VM ports
• Glance — Image Service, for storing the worker VM image

Infrastructure

• MySQL or MariaDB (Galera) — a dedicated instance for Abacá; never share the control plane's database
• RabbitMQ — a dedicated instance for Abacá; never share the platform's shared broker
• S3-compatible object storage (AWS S3, Wasabi, MinIO, or compatible) with bucket versioning enabled and Object Lock enabled at bucket creation time
• RHEL 9 or later base cloud image (for worker VM builds), with libguestfs available on the build host
• Kopia binary (FIPS-built, version-pinned) 0.17.0 or later — baked into the worker image, not fetched at boot

Operator tooling

• oc authenticated to the target OpenShift cluster
• openstack CLI (python-abacaclient installed) with admin credentials
• kustomize (for manifests)
• bash (for deploy/rhoso/ install scripts)

Optional

• Horizon — for the abaca-dashboard plugin
• tox 4.0 or later — for local development and testing
• Docker or Podman with Compose — for the local developer stack only

Keystone role

• The abaca_domain_owner role must exist in Keystone before any target template can be created. deploy/rhoso/01-identity.sh creates it. Without it, no principal can register a backup target and the install is functionally incomplete even if all services are up.

Run the preflight script first. It is read-only and exits non-zero on any condition that would cause the later install steps to fail.

bash deploy/rhoso/00-preflight.sh

The script checks cluster access, required CRDs, the platform CA bundle, and Keystone reachability. Review every FAIL and WARN line before continuing. Manila absence is reported as a warning, not a failure — the control plane installs without it, but end-to-end share protection requires it.

Step 1 — Verify cluster access and required CRDs

The preflight script confirms that the following CRDs are present. If any are missing, you are not running a RHOSO 18 control plane:

mariadbdatabases.mariadb.openstack.org
mariadbaccounts.mariadb.openstack.org
transporturls.rabbitmq.openstack.org
keystoneservices.keystone.openstack.org
keystoneendpoints.keystone.openstack.org
galeras.mariadb.openstack.org
rabbitmqs.rabbitmq.openstack.org

It also checks that cert-manager CRDs are present (required for the API service's internal TLS certificate) and that build.openshift.io BuildConfig is available.

Step 2 — Verify Keystone is functional

Check that tokens can actually be issued — both project-scoped and domain-scoped — and that the abaca_domain_owner role exists:

# Project-scoped token
openstack token issue

# Domain-scoped token — required for target template management
openstack --os-domain-name <your-domain> token issue

# The role must exist before any domain owner can be assigned
openstack role show abaca_domain_owner

If the domain-scoped token command fails, Keystone is not configured to issue domain-scoped tokens. This is the requirement that most often surprises operators: a project-scoped admin token is refused by Abacá's domain-owner endpoints because holding admin on a project says nothing about authority over a domain.

Step 3 — Verify Barbican is reachable

openstack secret list

Barbican is a hard dependency. Abacá stores no secrets in its own database — only Barbican hrefs and trust IDs. A Barbican outage makes enrollment, backup, and restore impossible.

Step 4 — Verify Manila backend health

openstack share service list

Every backend you intend to protect must show State: up. Backends in down state cause backup jobs to fail at provisioning_source with snapshots stuck in creating.

Additionally confirm the backend supports snapshots and IP-based access rules — backends without snapshot support cannot be protected by Abacá.

Step 5 — Verify Nova and Neutron port binding

Port binding failure is the dependency most likely to take the product down without implicating it: if Neutron cannot bind a port, no worker VM boots and every backup fails. Verify by booting a minimal test VM on the worker network:

openstack server create \
  --image cirros \
  --flavor m1.tiny \
  --network <worker-network> \
  --wait preflight-test-vm
openstack server delete preflight-test-vm

If this fails, the fault is in the cloud infrastructure, not in Abacá.

Step 6 — Verify worker VM network reachability

From a VM on the worker network (or using the test VM above), confirm all three paths are reachable. Workers must reach all three from a single network attachment:

# Abacá worker listener (control plane)
curl -sk https://<abaca-worker-listener>/

# S3 endpoint (object storage)
curl -sI https://<s3-endpoint>/

# Manila share export (NFS)
nc -z <manila-export-ip> 2049

A worker subnet with no egress (no router or floating IP) cannot host workers. If egress is unavailable via a Neutron router, attach workers to a provider network that reaches all three destinations directly.

Step 7 — Verify the Abacá schema is at head after db-sync

After running the install scripts, confirm the schema was built by alembic and not by some other mechanism:

# The alembic_version table must contain the current head revision.
# Tables present with an empty stamp means something other than alembic built them.
oc -n abaca exec deploy/abaca-api -- abaca-manage db_sync --check

Never delete a running db-sync Job. MySQL DDL does not roll back; killing alembic mid-upgrade leaves a half-built schema that poisons every subsequent migration.

Abacá uses oslo.config with an INI-style .conf file. Every option belongs to a named section. The table below covers the options most relevant to environment prerequisites and initial deployment. For the full surface, see the annotated sample configuration.

[DEFAULT] — Service catalog and endpoint resolution

[api] — API service binding

[conductor] — Orchestration and worker fleet

[database] — Relational store

[enrollment] — Target registration dispatch

[kopia] — Backup engine

[worker] — Worker VM agent

[abaca] — Service identity

The sections below describe the verification steps you will perform most often during bring-up and ongoing operations. All openstack commands use the python-abacaclient OSC plugin, which adds openstack share protection … subcommands.

Check that the service is registered and reachable

After running the install scripts, confirm the Abacá service endpoint appears in the catalog under the share-protection type:

openstack endpoint list --service share-protection

If this returns no rows, Horizon and the OSC plugin cannot locate the service. The endpoint must appear with the publicURL, internalURL, and adminURL variants.

Verify identity prerequisites before creating your first target

Target templates are a domain-level operation. Confirm your user holds the abaca_domain_owner role on the domain before attempting to create one:

# Confirm the role exists
openstack role show abaca_domain_owner

# Confirm your user has it on the target domain
openstack role assignment list \
  --role abaca_domain_owner \
  --domain <your-domain> \
  --user <your-user>

# Confirm you can obtain a domain-scoped token (required for template operations)
openstack --os-domain-name <your-domain> token issue

Verify the worker fleet is healthy

After the control plane is up, confirm the conductor has booted at least one worker VM and that it is registered:

openstack share protection worker list

A healthy fleet shows at least one worker with a recent heartbeat timestamp. If the list is empty or all workers show an expired heartbeat, check Nova port binding (see Troubleshooting) and verify conductor.worker_boot_network points at a network with egress to the worker listener.

Check the Abacá schema version

After the db-sync Job completes, the alembic_version table must contain the current head revision. Use the management CLI to verify:

oc -n abaca exec deploy/abaca-api -- abaca-manage db_sync --check

Verify Barbican custody separation

Abacá uses two separate Barbican custody paths that must never be merged. S3 credentials (s3_access_key_ref / s3_secret_key_ref) live in the Domain's service project; Kopia repository passwords (kopia_key_ref) live in each project's own Barbican namespace. Merging these grants whoever controls storage the ability to read every project's backups. Verify your Barbican policy enforces this separation after enrollment.

Enable Horizon multi-domain login

If you are deploying the abaca-dashboard plugin, Horizon must have multi-domain login enabled. Without it, any user outside the Default domain cannot authenticate at all — the login form re-renders with no error. Set this on the OpenStackControlPlane CR (this is a cluster-wide change, not part of any Abacá install script):

# OpenStackControlPlane .spec.horizon.template.customServiceConfig
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
OPENSTACK_KEYSTONE_DEFAULT_DOMAIN = 'Default'

Existing Default-domain users continue to work by entering Default in the new Domain field.

Example 1 — Run the preflight check before deploying

Always run this first. It is read-only and exits non-zero on any blocking condition.

bash deploy/rhoso/00-preflight.sh

Expected output (all lines should read ok):

--- cluster access ---
  ok    oc authenticated as system:admin on https://api.cluster.example.com:6443
  ok    namespace abaca present
--- required CRDs (operator-backed resources we rely on) ---
  ok    mariadbdatabases.mariadb.openstack.org
  ok    mariadbaccounts.mariadb.openstack.org
  ok    transporturls.rabbitmq.openstack.org
  ok    keystoneservices.keystone.openstack.org
  ok    keystoneendpoints.keystone.openstack.org
  ok    build.openshift.io (in-cluster image builds)
  ok    cert-manager (internal TLS for the API service)
--- control-plane infrastructure ---
  ok    database + message bus: Abaca's own (abaca-galera / abaca-rabbitmq)
  ok    galeras.mariadb.openstack.org (Abaca creates its own instance)
  ok    rabbitmqs.rabbitmq.openstack.org (Abaca creates its own instance)
  ok    secret/combined-ca-bundle in openstack (platform CA trust)
preflight OK.

If any line reads FAIL, resolve that condition before running install.sh.

Example 2 — Verify Keystone identity prerequisites

Confirm project-scoped and domain-scoped token issuance, and that the abaca_domain_owner role exists:

# Project-scoped token
openstack token issue -c id -f value

# Domain-scoped token (required for target template operations)
openstack --os-domain-name acme token issue -c id -f value

# The role must exist
openstack role show abaca_domain_owner -c name -f value

Expected output for each:

gAAAAAB...         # a Keystone token
gAAAAAB...         # a second, domain-scoped token
abaca_domain_owner

If the domain-scoped command fails with You are not authorized to perform the requested action, Keystone cannot issue domain-scoped tokens for this user. Check that the user holds the abaca_domain_owner role on the domain (not merely on a project within it).

Example 3 — Verify the service catalog entry

After running deploy/rhoso/01-identity.sh, confirm the service and its three endpoints are registered:

openstack endpoint list --service share-protection -c "Service Name" -c Interface -c URL

Expected output:

+-----------------+-----------+------------------------------------------+
| Service Name    | Interface | URL                                      |
+-----------------+-----------+------------------------------------------+
| share-protection| public    | https://abaca.apps.cluster.example.com/v1|
| share-protection| internal  | http://abaca-api.abaca.svc:9797/v1       |
| share-protection| admin     | http://abaca-api.abaca.svc:9797/v1       |
+-----------------+-----------+------------------------------------------+

If this returns no rows, Horizon and the OSC plugin cannot discover Abacá and all subsequent operations will fail.

Example 4 — Verify worker network reachability from a test VM

Boot a minimal VM on the worker network and confirm all three required paths are reachable from it:

# Boot a test VM on the worker network
openstack server create \
  --image cirros \
  --flavor m1.tiny \
  --network <worker-network> \
  --wait \
  reachability-probe

# Obtain its floating IP or use console access, then from inside the VM:
curl -sk https://<abaca-worker-listener>/          # must return HTTP response
curl -sI https://<s3-endpoint>/                    # must return HTTP 200 or 403
nc -z <manila-export-ip> 2049 && echo "NFS reachable"

# Clean up
openstack server delete reachability-probe

All three must succeed from the same network attachment. If any fail, the worker fleet will be unable to complete backup jobs even if Nova successfully boots the worker VM.

Example 5 — Confirm the database schema is at head

After the db-sync Kubernetes Job completes, verify alembic stamped the schema correctly:

# Check the Job completed successfully
oc -n abaca get job abaca-db-sync

# Verify schema version via the management CLI
oc -n abaca exec deploy/abaca-api -- abaca-manage db_sync --check

The db_sync --check command exits zero only when alembic_version contains the current head revision. A non-zero exit or an empty alembic_version table means the schema was not built by alembic — do not start the API or conductor until this is resolved.

Most of these present as "Abacá is broken" while the actual fault is in the underlying infrastructure. Use the symptom as a starting point, then follow the diagnostic steps.

Every backup fails; Nova reports "Exhausted all hosts available for retrying build failures"

• Cause: Neutron cannot bind a port on the worker network. This is the dependency most likely to take the product down without implicating Abacá directly.
• Fix: Boot a plain Cirros VM on the same network. If that also fails, the fault is in the cloud infrastructure, not in Abacá. Check the Nova scheduler logs and Neutron ML2 agent logs on the hypervisor. On VMware ESXi with a Standard vSwitch, the default port-group security (Promiscuous = Reject, MAC address changes = Reject) causes unicast replies to never reach the Neutron router, which looks like intermittent port binding failure from Nova's perspective.

Worker VM is ACTIVE in Nova but never registers with the conductor

• Cause 1: DNS on the worker subnet cannot resolve the hostname in conductor.worker_api_url. The agent starts, tries to connect to the listener, and silently fails — indistinguishable from a stale image without checking worker console output.
• Fix 1: Set conductor.worker_api_host_aliases to IP=HOSTNAME pairs for the listener. Do not substitute a bare IP in worker_api_url — that breaks TLS hostname verification even with the correct CA.
• Cause 2: The worker image was built before the current version of abaca-worker-agent. A stale image runs old code; the symptom is silence rather than an error.
• Fix 2: Rebuild the worker image (deploy/rhoso/worker/20-worker-image.sh) and update conductor.worker_boot_image to the new Glance image name. Worker VMs must not fetch packages from the internet at boot — all components must be baked in.

Backup jobs fail at provisioning_source; Manila snapshots stay in creating

• Cause: The Manila share backend is down or the driver does not support snapshots.
• Fix: Run openstack share service list and confirm every backend you intend to protect shows State: up. If a backend is down, resolve the Manila-side issue. If the backend does not support snapshots, it cannot be protected — Abacá never backs up a live share as the primary strategy.

Dashboard panels render "No items to display" for a tenant that has backups

• Cause: The Abacá publicURL endpoint's TLS certificate is not trusted by the Horizon pod. A failed fetch degrades silently to an empty list.
• Fix: Confirm the public endpoint serves a certificate signed by the CA bundle that OpenStack pods mount (secret/combined-ca-bundle in the control plane namespace). Check Horizon pod logs for TLS verification errors against the share-protection endpoint.

Horizon panels vanish, or another product's Horizon panels vanish after deploying the dashboard plugin

• Cause: The Horizon image was replaced rather than layered. The abaca-dashboard plugin must be layered onto the existing Horizon image — replacing it removes any panels from other products already present in the base.
• Fix: Re-apply the layering process per deploy/rhoso/08-dashboard.sh. If you manage your own Horizon build pipeline, consume abaca-dashboard and python-abacaclient as packages with --no-deps and --ignore-requires-python (the RHOSO Horizon image ships Python 3.9). The layer must also be rebuilt and re-applied whenever the base Horizon image is updated, or the vendor's update silently reverts the panels.

A redeployed dashboard image is "deployed" but the panels are unchanged; two browser sessions behave differently

• Cause: The Horizon image is referenced by :latest tag. With imagePullPolicy: IfNotPresent, pods keep cached layers and the operator sees no diff in the spec — so no rollout occurs.
• Fix: Pin the image reference to a digest (@sha256:…) so every change produces a spec diff that triggers a rollout.

A tenant outside the Default domain cannot log in to Horizon; the login form re-renders with no error

• Cause: Horizon multi-domain login is disabled. There is no Domain field on the login form, so users in any domain other than Default cannot authenticate.
• Fix: Set the following in the OpenStackControlPlane CR (this is a cluster-wide change — it adds a Domain field for all users):

# .spec.horizon.template.customServiceConfig
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
OPENSTACK_KEYSTONE_DEFAULT_DOMAIN = 'Default'

Existing Default-domain users keep working by entering Default in the new field.

Target stuck in registering state indefinitely

• Cause 1: No worker VM is available to run the enrollment job. Check openstack share protection worker list for a healthy, registered worker.
• Cause 2: The enrollment job was never dispatched — confirm enrollment.dispatch = rpc in the conductor's configuration and that the conductor is running.
• Fix: Resolve the worker fleet issue (see the first entry above) or correct the dispatch configuration. The reconciliation loop (conductor.reconciliation_interval, default 300 seconds) will eventually walk the orphaned job to an error state with operator_action_required, at which point you can re-enroll.

Enrollment fails with an error about a missing repository password

• Cause: You are attempting to adopt an existing Kopia repository (the bucket already contains a repository from a previous enrollment) without supplying its password. Kopia repositories are client-side encrypted and Abacá has no way to recover the password — it is a tenant-held secret.
• Fix: Supply the repository_password_ref (a Barbican href pointing to the original repository password) when re-enrolling. This is classified as tenant_action_required.

The db-sync Job completes but alembic_version is empty; API startup fails

• Cause: Something other than alembic created the database tables — for example, a create_all() call from an older version of the code that does not stamp the alembic revision. A half-built schema poisons every subsequent migration.
• Fix: Drop and recreate the Abacá database, then re-run the db-sync Job. Do not attempt to manually stamp the revision — the migration history will be inconsistent. Never delete a running db-sync Job; MySQL DDL does not roll back.

Workers on the tenant network cannot reach the S3 endpoint or the Abacá worker listener

• Cause: The Neutron router is not forwarding traffic. On certain hypervisors (VMware ESXi Standard vSwitch, VMware Distributed vSwitch, or nested-virt IaaS environments), unicast replies destined for the router's fa:16:3e:* MAC are dropped by the underlay because the port-group security policy rejects non-native MACs.
• Diagnosis: Run tcpdump on the hypervisor's external NIC while pinging the router's external IP from another host on the LAN. If you see outbound ARP but no reply, you are in the filtered-underlay case.
• Fix: For development environments, use deploy/kolla/setup-host-nat-egress.sh to plumb a host-side NAT gateway. For production, resolve the vSwitch security policy (set Promiscuous mode or MAC learning at the port group), or attach workers directly to a provider network that has unfiltered layer-3 reachability to all three required destinations.