Guide

Sample Configuration File

An annotated, working example .conf

Overview

This page provides a fully annotated abaca.conf sample that you can copy, adapt, and deploy for either the abaca-api or abaca-conductor process. Every configuration key Abacá recognises is shown in its correct INI section with its type, default value, and a plain-English explanation of what changing it does and why you might want to. The file follows oslo.config conventions: commented-out lines show the compiled-in default; uncomment and edit only the options you need to override. If a key is not shown in this file it does not exist — do not add arbitrary keys.

Prerequisites

Before editing or deploying this file you need:

A running RHOSO 18+ control plane with OpenShift ≥ 4.14
abaca-api and abaca-conductor pods deployed in the abaca OpenShift namespace (via the kustomize manifests in deploy/rhoso/)
A dedicated MySQL/MariaDB (Galera) database for Abacá and its connection string
A dedicated RabbitMQ instance for Abacá and its AMQP URL
The Keystone service user (abaca) created and the service_user_id known
At least one worker VM image uploaded to Glance with the abaca_worker_image=1 property
A Neutron network name or UUID that worker VMs will boot onto
The public URL of the abaca-api OpenShift Route (used as worker_api_url)
Optionally: a PEM CA bundle if the Route uses a private CA

Installation

The configuration file is mounted into the abaca-api and abaca-conductor pods as a Kubernetes Secret or ConfigMap by the kustomize manifests. The typical workflow is:

Copy the sample from the repository:

cp etc/abaca.conf.sample etc/abaca.conf

Edit the file following the annotated reference in the Configuration section below. At minimum you must set:
- [database] connection
- [DEFAULT] transport_url
- [abaca] service_user_id
- [conductor] worker_api_url
- [conductor] worker_boot_network
Create a Kubernetes Secret in the abaca namespace:

kubectl create secret generic abaca-conf \
  --from-file=abaca.conf=etc/abaca.conf \
  -n abaca

Apply the kustomize manifests (the manifests reference the secret by name and mount it at /etc/abaca/abaca.conf):

kubectl apply -k deploy/rhoso/

Run the database migration Job before starting the service pods. The kustomize overlay triggers this automatically, but you can verify it completed:

kubectl wait job/abaca-db-sync --for=condition=complete -n abaca --timeout=120s

Verify the pods are running:

kubectl get pods -n abaca

Expected output:

NAME                               READY   STATUS    RESTARTS   AGE
abaca-api-7d9f8b6c4-xqkpl          1/1     Running   0          2m
abaca-conductor-5c6b7d8e9-mnpqr    1/1     Running   0          2m

After a configuration change, update the Secret and perform a rolling restart:

kubectl create secret generic abaca-conf \
  --from-file=abaca.conf=etc/abaca.conf \
  -n abaca --dry-run=client -o yaml | kubectl apply -f -

kubectl rollout restart deployment/abaca-api deployment/abaca-conductor -n abaca

Configuration

All options use oslo.config INI syntax. Lines beginning with # are commented out and take the compiled-in default. Remove the # and set a value to override.

`[DEFAULT]` — oslo.messaging transport and logging

These keys come from oslo.messaging and oslo.log and are documented in the upstream OpenStack oslo libraries. The most important ones for Abacá are:

Key	Type	Default	Purpose
`transport_url`	string	`rabbit://`	Full AMQP URL for Abacá's dedicated RabbitMQ broker. Must not point at the platform's shared broker.
`debug`	boolean	`false`	Set `true` to emit DEBUG-level logs. Hot-reloadable — no restart needed.
`log_file`	string	(stderr)	Absolute path for a log file. Leave unset in containerised deployments; logs go to stdout/stderr and are captured by the OpenShift log driver.
`use_json`	boolean	`false`	Emit logs as JSON objects. Recommended when a log aggregator (e.g. Splunk, Loki) consumes pod stdout.

transport_url carries only Barbican hrefs and trust IDs — never raw secret material. The format follows oslo.messaging conventions:

transport_url = rabbit://abaca:CHANGEME@rabbitmq.abaca.svc:5672/abaca_vhost

`[abaca]` — service identity

These options tell the API and conductor who they are in Keystone and where worker VMs live.

Key	Type	Default	Purpose
`service_user_id`	string	(none)	Required. Keystone user UUID of the `abaca` service account. Exposed on the unauthenticated `/v1/service_info` endpoint so tenants can create Keystone trusts to the correct trustee without enumerating users. Set this to the UUID emitted by the identity setup script.
`service_user_name`	string	`abaca`	Human-readable name of the service account. Informational; the UUID in `service_user_id` is what the API uses.
`worker_project_id`	string	(none)	Keystone project UUID of the per-Domain service project where worker VMs run. Advertised on `/v1/service_info`.
`worker_project_name`	string	`abaca`	Human-readable name of the worker VM project.

[abaca]
service_user_id = a1b2c3d4-e5f6-7890-abcd-ef1234567890
service_user_name = abaca
worker_project_id = f0e1d2c3-b4a5-9687-fedc-ba0987654321
worker_project_name = abaca

`[api]` — HTTP listener

These options apply only to the abaca-api process.

Key	Type	Default	Purpose
`bind_host`	string	`0.0.0.0`	IP address on which the API server listens. In a pod, `0.0.0.0` is correct.
`bind_port`	integer	`9797`	TCP port. The OpenShift Service and Route front this port.
`noauth`	boolean	`false`	Development only. Bypasses keystonemiddleware and injects a fake admin context. Never enable in production.
`max_limit`	integer	`1000`	Hard ceiling on the page size returned by list endpoints.
`default_limit`	integer	`100`	Default page size when a client omits the `limit` query parameter.

[api]
bind_host = 0.0.0.0
bind_port = 9797
max_limit = 1000
default_limit = 100
# noauth = false  # NEVER set true in production

`[conductor]` — orchestration, scheduling, and worker fleet

These options apply only to the abaca-conductor process. They govern how often the scheduler runs, how the reconciliation loop behaves, and how the warm worker pool is managed.

Scheduling and reconciliation timing

Key	Type	Default	Purpose
`scheduler_interval`	integer	`60`	Seconds between passes where the conductor checks all enabled Policies for due backups. Lower values give tighter adherence to cron schedules at the cost of more database reads.
`reconciliation_interval`	integer	`300`	Seconds between full reconciliation sweeps. The sweep reaps dead workers, re-queues orphaned jobs, and triggers maintenance and catalogue-sync tasks.
`sweep_command_wait_seconds`	integer	`120`	How long a reconciliation sweep waits for a worker command response before giving up and moving on. Keep this well below `reconciliation_interval`. A sweep that blocks longer than the interval causes every other sweep task (heartbeat reap, orphan deadline, warm-pool boot) to accumulate delays — this caused a measured 12-hour sweep period on a reference cluster when it was left at the Kopia subprocess timeout. Tenant-facing jobs are unaffected; only reconciliation cadence suffers.
`queued_job_deadline_seconds`	integer	`300`	Seconds a job may stay in `queued` state before the reconciliation loop considers it orphaned and re-dispatches it. Should be larger than the time it takes a worker to pick up a lease.

Maintenance and catalogue hygiene

Key	Type	Default	Purpose
`maintenance_interval_seconds`	integer	`604800` (1 week)	How often the conductor schedules a `kopia maintenance run --full` for each registered BackupTarget. Kopia only reclaims S3 storage from objects orphaned by retention drops during maintenance — so the tenant's retention policy only frees space after this runs. Safe to run concurrently with backups; Kopia enforces per-repo exclusivity.
`catalogue_sync_interval_seconds`	integer	`3600`	How often the reconciliation loop reconciles each target's Backup rows against the snapshots Kopia actually holds. This is what makes a rebuilt-from-repository target's history visible and what picks up snapshots written by another control plane.
`usage_sample_interval_seconds`	integer	`21600` (6 h)	How often the conductor runs `kopia content stats` per BackupTarget and persists the result as post-dedup, post-compression bytes-on-S3. This number is more accurate than summing `Backup.size_bytes` (which double-counts cross-snapshot deduplication). The operation is read-only and safe to run concurrently.

Worker heartbeat and token lifetime

Key	Type	Default	Purpose
`worker_heartbeat_interval_seconds`	integer	`10`	How often a running worker agent sends a heartbeat RPC cast to the conductor. Must be significantly less than `worker_heartbeat_deadline_seconds`.
`worker_heartbeat_deadline_seconds`	integer	`60`	Seconds since the last heartbeat before the fleet manager marks a worker DEAD and reassigns its in-flight job. Increase this only if your network is lossy enough to swallow a heartbeat cycle.
`worker_token_ttl_seconds`	integer	`1800`	Lifetime of a worker's bearer token (the credential it presents to `/v1/workers/` endpoints — not* a Keystone token). This is a sliding window: every heartbeat pushes the expiry forward, so a healthy worker never loses its credential. A worker that has vanished stops being able to authenticate this long after its last heartbeat, bounding how long a token from a destroyed VM's disk stays usable. The default of 1800 s (30× the heartbeat deadline) covers the measured 15–19 minute cold-boot time of a hard-rebooted worker VM, ensuring `openstack server reboot --hard` remains a valid recovery operation. Set `0` to disable expiry entirely.

Worker fleet boot configuration

Key	Type	Default	Purpose
`min_workers`	integer	`1`	Warm-worker floor per project that has an enabled Policy or a non-terminal Job. The reconciliation loop boots one worker per tick until every active project is at this floor. Setting `0` disables automatic booting; you must boot workers manually. Requires `worker_api_url`, `worker_boot_network`, and `worker_boot_key_name` to be set — without them the sweep logs a warning and stays idle.
`worker_boot_cooldown_seconds`	integer	`180`	Minimum interval between warm-pool boot attempts. Cloud-init typically registers a new worker within 30–90 s; 180 s prevents double-booting when the sweep ticks before the previous boot's agent has registered.
`worker_boot_os_cloud`	string	`abaca-service`	The `clouds.yaml` cloud profile the conductor uses when calling Nova to boot worker VMs.
`worker_boot_image`	string	`abaca-worker-0.23.1`	Name or UUID of the Glance image to boot worker VMs from. Must have the `abaca_worker_image=1` property. Update this when rolling out a new worker image.
`worker_boot_flavor`	string	`m1.small`	Nova flavor for worker VMs. Size according to the number of concurrent Kopia streams you expect per worker.
`worker_boot_network`	string	(none, required)	Required for auto-boot. Neutron network name or UUID that worker VMs attach to at boot.
`worker_boot_key_name`	string	(none)	Nova keypair name injected into worker VMs for emergency SSH access. Optional but recommended for operations.
`worker_boot_max_failures`	integer	`3`	Maximum consecutive boot failures before the conductor pauses auto-booting for a project and emits an `operator_action_required` error.
`worker_boot_failure_window_seconds`	integer	`7200`	The rolling window over which `worker_boot_max_failures` is counted. Failures older than this window do not count against the limit.
`worker_boot_grace_seconds`	integer	`1200`	How long after a boot the conductor waits for the worker agent to register before counting the boot as a failure.
`worker_api_url`	string	(none, required)	Required for auto-boot. The HTTPS URL of the `abaca-api` OpenShift Route as seen from inside the worker VM network. Worker agents use this URL to register and claim leased commands.
`worker_api_ca_file`	string	(none)	Path to a PEM CA bundle that worker VMs use to verify the TLS certificate of `worker_api_url`. Required if the Route uses a private CA. Baked into the worker image at build time or injected via cloud-init.
`worker_api_host_aliases`	list	(empty)	One or more `IP=HOSTNAME` entries written into each booted worker's `/etc/hosts`. Required when the worker subnet's DNS resolver cannot resolve the hostname in `worker_api_url` — common on RHOSO where the Route sits on a private wildcard domain but the worker subnet points at a public resolver. Format: `192.0.2.10=abaca.apps.cluster.example.com`. Omitting this when needed produces the same symptom as omitting `worker_api_ca_file`: the agent cannot contact the API at all.

[conductor]
scheduler_interval = 60
reconciliation_interval = 300
sweep_command_wait_seconds = 120
queued_job_deadline_seconds = 300

worker_heartbeat_interval_seconds = 10
worker_heartbeat_deadline_seconds = 60
worker_token_ttl_seconds = 1800

maintenance_interval_seconds = 604800
catalogue_sync_interval_seconds = 3600
usage_sample_interval_seconds = 21600

min_workers = 1
worker_boot_cooldown_seconds = 180
worker_boot_os_cloud = abaca-service
worker_boot_image = abaca-worker-0.23.1
worker_boot_flavor = m1.small
worker_boot_network = abaca-worker-net
worker_boot_key_name = abaca-ops
worker_boot_max_failures = 3
worker_boot_failure_window_seconds = 7200
worker_boot_grace_seconds = 1200

worker_api_url = https://abaca.apps.cluster.example.com
worker_api_ca_file = /etc/abaca/ca-bundle.pem
worker_api_host_aliases = 192.0.2.10=abaca.apps.cluster.example.com

`[database]` — relational store

Key	Type	Default	Purpose
`connection`	string	(none, required)	SQLAlchemy connection URL for Abacá's dedicated MySQL/MariaDB (Galera) database. Abacá stores only metadata here — backup data lives in S3.

[database]
connection = mysql+pymysql://abaca:CHANGEME@mariadb.abaca.svc:3306/abaca

`[enrollment]` — target enrollment dispatch

Key	Type	Valid values	Purpose
`dispatch`	string	`rpc`, `inline`	Controls how enrollment jobs are dispatched. `rpc` sends the job to a worker VM via the conductor (production default). `inline` runs enrollment synchronously in the API process (development/testing only).

[enrollment]
dispatch = rpc

`[kopia]` — backup engine

These options control how the Kopia binary is invoked inside worker VMs.

Key	Type	Default	Purpose
`binary`	string	`kopia`	Path or name of the Kopia binary. Override to an absolute path if Kopia is installed outside `$PATH`.
`require_fips_profile`	boolean	`true`	Require that every Kopia repository is initialised with a FIPS-approved cryptographic profile (AES-256-GCM-HMAC-SHA256 encryption, HMAC-SHA256-128 block hash, PBKDF2 key derivation). This is enforced at repository creation and is immutable afterward. Do not disable in production.
`subprocess_timeout`	integer	`3600`	Seconds before the worker agent kills a Kopia subprocess. This bounds individual `kopia snapshot create` and `kopia restore` operations — it is separate from and unrelated to `sweep_command_wait_seconds`, which governs only reconciliation sweeps.
`executor`	string	`http`, `ephemeral_container`	How Kopia is executed. `http` runs the binary directly inside the worker VM (production). `ephemeral_container` launches Kopia in a container (local dev with the docker-compose stack).
`image`	string	(none)	Container image to use when `executor = ephemeral_container`. Not used in production.
`container_engine`	string	`docker`, `podman`	Container runtime when `executor = ephemeral_container`. Not used in production.
`container_network`	string	(none)	Docker/Podman network when `executor = ephemeral_container`. Not used in production.

[kopia]
binary = kopia
require_fips_profile = true
subprocess_timeout = 3600
executor = http

`[worker]` — worker agent (abaca-worker-agent)

These options are set in the abaca.conf baked into or injected into each worker VM. They are not read by the API or conductor.

Key	Type	Default	Purpose
`id`	string	(none)	UUID that uniquely identifies this worker VM instance. Set by cloud-init at first boot.
`api_url`	string	(none)	URL of the `abaca-api` endpoint the agent registers with and polls for commands. Must match `worker_api_url` in the conductor config.
`token`	string	(none)	Bearer token the agent presents to `/v1/workers/*`. Provisioned at boot and rotated by the conductor within `worker_token_ttl_seconds`.
`api_ca_file`	string	(none)	PEM CA bundle for TLS verification of `api_url`. Must match `worker_api_ca_file`.
`capacity_slots`	integer	(none, required)	Number of concurrent jobs this worker VM can run simultaneously. The conductor's fleet picker will not assign a job to a worker whose running job count equals or exceeds this value.
`mount_base`	string	`/var/lib/abaca/mnt`	Filesystem path under which Manila shares are mounted during backup and restore operations.
`command_timeout_seconds`	integer	`300`	Seconds the agent waits for the conductor to acknowledge a completed command before treating it as lost and retrying.
`command_lease_seconds`	integer	`120`	How long the conductor holds a command lease for this worker. The agent must acknowledge within this window.
`max_command_duration_seconds`	integer	`21600`	Hard upper bound (6 h) on how long any single command (backup or restore job) may run before the agent aborts it.
`claim_poll_interval_seconds`	float	`2.0`	Seconds between polls when the agent is waiting to claim its next command.
`long_poll_seconds`	integer	`20`	Duration of each long-poll HTTP request the agent makes to the API when waiting for a command.
`progress_interval_seconds`	integer	`30`	How often the agent sends progress updates to the conductor during an active Kopia transfer.
`listener_port`	integer	`9798`	TCP port the worker-listener sidecar listens on.
`command_poll_interval_seconds`	float	`2.0`	Seconds between command-status poll cycles inside the agent.

Usage

In a production RHOSO 18 deployment, abaca.conf is consumed by two distinct processes — abaca-api and abaca-conductor — each running as a separate Deployment in the abaca OpenShift namespace. Both processes read the same file, but only the keys belonging to their own sections are meaningful.

A third, distinct copy of abaca.conf is baked into or injected into each worker VM by cloud-init. That copy populates only the [worker] section (plus [kopia] and [DEFAULT] for logging); it does not carry database credentials or RabbitMQ URLs.

Key workflow: setting the minimum required options

Start from the annotated sample and uncomment only the options you need to change. Most defaults are production-appropriate. The options you must set before the service will start are:

[DEFAULT]
transport_url = rabbit://abaca:CHANGEME@rabbitmq.abaca.svc:5672/abaca_vhost

[abaca]
service_user_id = <uuid from identity setup script>

[database]
connection = mysql+pymysql://abaca:CHANGEME@mariadb.abaca.svc:3306/abaca

[conductor]
worker_api_url = https://abaca.apps.cluster.example.com
worker_boot_network = abaca-worker-net

Key workflow: enabling structured logging

For log aggregation pipelines (Loki, Splunk, Elasticsearch) enable JSON output:

[DEFAULT]
use_json = true
debug = false

Key workflow: adjusting the warm-worker floor

If your environment has many projects with enabled policies, increase min_workers to keep more warm workers per project. If you prefer to boot workers manually (or via your own automation), set it to 0:

[conductor]
min_workers = 0   # disable automatic warm-pool booting

Key workflow: updating the worker image after a new build

After uploading a new worker image to Glance, update worker_boot_image and restart the conductor. The change takes effect on the next reconciliation sweep; already-running workers are unaffected until they are retired.

[conductor]
worker_boot_image = abaca-worker-0.24.0

Key workflow: tuning retention and maintenance

If your retention policies are short (e.g. keep the last 3 dailies) and you need S3 storage reclaimed quickly, shorten maintenance_interval_seconds:

[conductor]
maintenance_interval_seconds = 86400   # run Kopia maintenance daily instead of weekly

Examples

Minimal production configuration

This example shows the smallest set of options needed to run abaca-api and abaca-conductor in a production RHOSO 18 environment. All other values take compiled-in defaults.

[DEFAULT]
transport_url = rabbit://abaca:s3cr3t@rabbitmq.abaca.svc:5672/abaca

[abaca]
service_user_id = a1b2c3d4-e5f6-7890-abcd-ef1234567890
service_user_name = abaca
worker_project_id = f0e1d2c3-b4a5-9687-fedc-ba0987654321
worker_project_name = abaca

[api]
bind_host = 0.0.0.0
bind_port = 9797

[conductor]
worker_api_url = https://abaca.apps.ocp.example.com
worker_boot_network = abaca-worker-net
worker_boot_image = abaca-worker-0.23.1
worker_boot_key_name = abaca-ops

[database]
connection = mysql+pymysql://abaca:s3cr3t@mariadb.abaca.svc:3306/abaca

[enrollment]
dispatch = rpc

[kopia]
require_fips_profile = true

Full annotated production configuration

This example shows a hardened production deployment with JSON logging, a private CA, host aliases for DNS resolution, aggressive warm-pool settings, and daily Kopia maintenance.

[DEFAULT]
# Abacá's dedicated RabbitMQ — not the platform shared broker.
transport_url = rabbit://abaca:s3cr3t@rabbitmq.abaca.svc:5672/abaca
# Structured logging for Loki/Splunk.
use_json = true
debug = false

[abaca]
# UUID emitted by deploy/rhoso/01-identity.sh
service_user_id = a1b2c3d4-e5f6-7890-abcd-ef1234567890
service_user_name = abaca
worker_project_id = f0e1d2c3-b4a5-9687-fedc-ba0987654321
worker_project_name = abaca

[api]
bind_host = 0.0.0.0
bind_port = 9797
max_limit = 1000
default_limit = 100
# noauth must remain false in production.

[conductor]
# --- Scheduling ---
scheduler_interval = 60
reconciliation_interval = 300
# Keep well below reconciliation_interval to avoid sweep pile-up.
sweep_command_wait_seconds = 120
queued_job_deadline_seconds = 300

# --- Heartbeat and token ---
worker_heartbeat_interval_seconds = 10
worker_heartbeat_deadline_seconds = 60
# 1800 s (30× deadline) covers a hard-rebooted VM's 15-19 min boot time.
worker_token_ttl_seconds = 1800

# --- Maintenance and catalogue ---
# Run Kopia maintenance daily so retention actually frees S3 storage.
maintenance_interval_seconds = 86400
catalogue_sync_interval_seconds = 3600
usage_sample_interval_seconds = 21600

# --- Warm worker pool ---
min_workers = 1
worker_boot_cooldown_seconds = 180
worker_boot_max_failures = 3
worker_boot_failure_window_seconds = 7200
worker_boot_grace_seconds = 1200

# --- Worker boot parameters ---
worker_boot_os_cloud = abaca-service
worker_boot_image = abaca-worker-0.23.1
worker_boot_flavor = m1.medium
worker_boot_network = abaca-worker-net
worker_boot_key_name = abaca-ops

# --- Worker-to-API connectivity ---
# The HTTPS URL of the abaca-api OpenShift Route as seen from inside the worker network.
worker_api_url = https://abaca.apps.ocp.example.com
# Required when the Route uses a private CA.
worker_api_ca_file = /etc/abaca/ca-bundle.pem
# Required when the worker subnet's DNS cannot resolve the Route hostname.
worker_api_host_aliases = 192.0.2.42=abaca.apps.ocp.example.com

[database]
connection = mysql+pymysql://abaca:s3cr3t@mariadb.abaca.svc:3306/abaca

[enrollment]
dispatch = rpc

[kopia]
binary = kopia
# FIPS profile is required in production; immutable after repository creation.
require_fips_profile = true
subprocess_timeout = 3600
executor = http

Worker VM configuration (`[worker]` section only)

This is the fragment written into /etc/abaca/abaca.conf on each worker VM by cloud-init. It does not contain database or RabbitMQ credentials.

[DEFAULT]
use_json = true

[worker]
# Set by cloud-init at first boot to a unique UUID.
id = 7f3a1c2d-0000-0000-0000-worker000001
api_url = https://abaca.apps.ocp.example.com
# Bearer token provisioned at boot by the conductor.
token = eyJhb...REDACTED
# CA bundle to verify the api_url TLS certificate.
api_ca_file = /etc/abaca/ca-bundle.pem
# Number of concurrent jobs this VM can handle.
capacity_slots = 2
mount_base = /var/lib/abaca/mnt
command_timeout_seconds = 300
command_lease_seconds = 120
max_command_duration_seconds = 21600
claim_poll_interval_seconds = 2
long_poll_seconds = 20
progress_interval_seconds = 30
listener_port = 9798

[kopia]
binary = kopia
require_fips_profile = true
subprocess_timeout = 3600
executor = http

Local developer stack (docker-compose)

This fragment is used with deploy/docker-compose.dev.yml for local development. It disables authentication, uses SQLite, and runs Kopia in an ephemeral container.

[DEFAULT]
debug = true

[api]
bind_host = 0.0.0.0
bind_port = 9797
# noauth=true is only safe for local development.
noauth = true

[database]
connection = sqlite:////tmp/abaca-dev.db

[enrollment]
# inline runs enrollment synchronously in the API process — no worker needed.
dispatch = inline

[kopia]
executor = ephemeral_container
image = kopia/kopia:0.17.0
container_engine = docker
container_network = abaca_default
require_fips_profile = false

Troubleshooting

Use a consistent diagnostic approach: confirm the symptom in pod logs (kubectl logs -n abaca <pod>), cross-reference with the config option described here, and apply the fix.

Symptom: abaca-conductor pod fails to start with oslo_config.cfg.RequiredOptError: value required for option connection in group [database]

Cause: [database] connection is not set in abaca.conf.

Fix: Add the connection string to your config, update the Kubernetes Secret, and restart the pod:

# Edit abaca.conf, then:
kubectl create secret generic abaca-conf \
  --from-file=abaca.conf=etc/abaca.conf \
  -n abaca --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deployment/abaca-conductor -n abaca

Symptom: Worker VMs boot but the agent cannot reach the API; logs show connection refused or name or service not known for worker_api_url.

Cause (DNS): The worker subnet's DNS resolver cannot resolve the OpenShift Route hostname. The symptom is identical whether the problem is DNS or TLS.

Fix: Set worker_api_host_aliases in [conductor] with the Route's IP and hostname:

[conductor]
worker_api_host_aliases = 192.0.2.42=abaca.apps.ocp.example.com

Cause (TLS): The Route uses a private CA that the worker does not trust.

Fix: Set worker_api_ca_file to the path of the CA bundle baked into the worker image:

[conductor]
worker_api_ca_file = /etc/abaca/ca-bundle.pem

Symptom: Reconciliation sweeps that should take 5 minutes are taking hours. Logs show the sweep completing after many multiples of reconciliation_interval.

Cause: sweep_command_wait_seconds is set too high (e.g., equal to kopia subprocess_timeout). Each blocked sweep command holds the sweep thread, causing the 60 s heartbeat reap, 300 s orphan deadline, and warm-pool boot tasks to accumulate corresponding delays. This is a known operational hazard measured at 12+ hours on a reference cluster.

Fix: Ensure sweep_command_wait_seconds is well below reconciliation_interval:

[conductor]
reconciliation_interval = 300
sweep_command_wait_seconds = 120   # must be < reconciliation_interval

Symptom: After a hard reboot of a worker VM, the agent fails to authenticate and the worker stays in an error state. Logs show 401 Unauthorized from the agent.

Cause: worker_token_ttl_seconds is set too low. A hard-rebooted RHEL worker VM can take 15–19 minutes to reach its first heartbeat after reboot. If the token TTL is shorter than that window, the token baked into /etc/abaca/abaca.conf has already expired.

Fix: Restore the default of 1800 seconds (or higher if your VM boot times are longer):

[conductor]
worker_token_ttl_seconds = 1800

Symptom: The conductor is not automatically booting warm workers despite min_workers = 1 being set.

Cause: One or more of the required boot parameters is missing: worker_api_url, worker_boot_network, or worker_boot_key_name. The sweep logs a warning and stays idle when any required parameter is absent.

Fix: Confirm all three are set in [conductor]:

[conductor]
worker_api_url = https://abaca.apps.ocp.example.com
worker_boot_network = abaca-worker-net
worker_boot_key_name = abaca-ops

Then check the conductor logs for operator_action_required entries that describe which parameter is missing.

Symptom: openstack share protection commands fail with EndpointNotFound.

Cause: The catalog_type in the [DEFAULT] section of the python-abacaclient configuration (or the tempest plugin configuration) does not match the service type registered in the Keystone catalog.

Fix: Verify the catalog entry matches the default share-protection:

[DEFAULT]
catalog_type = share-protection
endpoint_type = publicURL

Also confirm the service and endpoints were registered correctly by the identity setup script (deploy/rhoso/01-identity.sh).

Symptom: abaca-api starts but authentication always fails; all requests return 401.

Cause: [api] noauth = true was accidentally left enabled in a production deployment, then set back to false, but the change was not reflected in the mounted Secret.

Fix: Ensure noauth = false (or the line is commented out) in abaca.conf, then update the Secret and restart the pod. Never set noauth = true in production.

Symptom: S3 storage is not being reclaimed even though retention policies have dropped old backups.

Cause: Kopia only reclaims orphaned S3 objects during kopia maintenance run --full. If maintenance_interval_seconds is set to the default of 604800 (weekly), storage may not be freed for up to a week after retention drops a backup.

Fix: Lower maintenance_interval_seconds to match your operational expectations:

[conductor]
maintenance_interval_seconds = 86400   # reclaim storage daily

Sample Configuration File

[DEFAULT] — oslo.messaging transport and logging

[abaca] — service identity

[api] — HTTP listener

[conductor] — orchestration, scheduling, and worker fleet