---
title: Health Checks
product: trilio-share-protection
doc_type: guide
version: master
source: git2docs (code-derived, validation-filtered)
canonical: https://git2docs.com/murali-balcha/docs/trilio-share-protection/trilio-share-protection-/health-checks
---

# Health Checks

_Liveness and readiness probe endpoints_

## Overview

This page describes the health check endpoints exposed by the `abaca-api` service and explains how to use them to verify that the service is alive and ready to accept traffic. Kubernetes (and OpenShift) use these probes to decide whether to route requests to a pod and whether to restart it — getting them right is essential for zero-downtime deployments and reliable operations. Use these endpoints in your liveness and readiness probe configurations, in integration smoke tests, and in monitoring scripts to confirm that `abaca-api` is healthy before running backup or restore workflows.

## Prerequisites

Before working with the health check endpoints you need:

- A running `abaca-api` deployment (containerised on OpenShift ≥ 4.12, or locally via Docker Compose)
- Network access to the `abaca-api` service on port **9797** (the port declared in `deploy/kustomize/base/api-deployment.yaml`)
- `curl` or any HTTP client capable of issuing `GET` requests
- (For Kubernetes/OpenShift probe configuration) `kubectl` or `oc` CLI access to the cluster with permission to edit Deployment specs

> **Note for reviewer:** The source material does not include explicit route definitions for `/health`, `/live`, or `/ready`. The seed table lists these as common endpoints to verify. Confirm the actual registered paths from the Flask route definitions (or an `openapi.yaml` / `swagger.yaml` file) before publishing.

## Installation

The health check endpoints are built into `abaca-api` — no additional installation steps are required. To reach them you only need the service running and exposed.

**1. Identify the API service address.**

On OpenShift, retrieve the service ClusterIP and port:

```bash
oc get svc abaca-api -o jsonpath='{.spec.clusterIP}:{.spec.ports[0].port}'
```

For a local Docker Compose stack, the default address is `http://localhost:9797`.

**2. Verify basic reachability.**

```bash
curl -sf http://<ABACA_API_HOST>:9797/health
```

A healthy service returns HTTP 200 with a JSON body. If `curl` times out or returns a connection-refused error, the pod is not yet running — check pod status:

```bash
oc get pods -l app=abaca-api
```

**3. (Optional) Configure Kubernetes/OpenShift probes.**

Add the following probe stanza to the `abaca-api` container spec in your Kustomize overlay or Helm values. These settings are safe starting points; tune `initialDelaySeconds` and `periodSeconds` for your environment:

```yaml
livenessProbe:
  httpGet:
    path: /live
    port: 9797
  initialDelaySeconds: 10
  periodSeconds: 15
  failureThreshold: 3

readinessProbe:
  httpGet:
    path: /ready
    port: 9797
  initialDelaySeconds: 5
  periodSeconds: 10
  failureThreshold: 3
```

> **Note for reviewer:** The base `api-deployment.yaml` in the source material does not yet include `livenessProbe` or `readinessProbe` stanzas. Confirm whether these should be added to the base manifest or managed only through overlays, and update the example accordingly.

## Configuration

The health check endpoints themselves have no runtime configuration knobs — they are always enabled when `abaca-api` is running. The behaviour of the probes is controlled through your Kubernetes or OpenShift Deployment spec.

| Field | Where it lives | Effect |
|---|---|---|
| `containerPort` | `api-deployment.yaml` | The port `abaca-api` listens on. Currently **9797**. All probe `port` values must match this. |
| `livenessProbe.failureThreshold` | Deployment probe spec | How many consecutive failures before Kubernetes restarts the container. |
| `readinessProbe.failureThreshold` | Deployment probe spec | How many consecutive failures before Kubernetes stops routing traffic to the pod. |
| `initialDelaySeconds` | Deployment probe spec | Seconds to wait after container start before the first probe fires. Increase this if `abaca-api` takes time to connect to the database and message broker on startup. |
| `periodSeconds` | Deployment probe spec | How frequently Kubernetes runs each probe. |

`abaca-api` is configured via `/etc/abaca/abaca.conf` (oslo.config) loaded automatically by the gunicorn entry point (`abaca.api.wsgi:get_application()`). Health check paths are not affected by that configuration file.

> **Note for reviewer:** Confirm whether `/ready` performs a live dependency check (database reachability, RabbitMQ connectivity) or only checks internal process state. If it checks dependencies, document which ones and what the failure response body looks like.

## Usage

Use these three endpoints depending on what you need to verify:

### Overall service health — `GET /health`

Call this endpoint from monitoring scripts or smoke tests to confirm the service is operational. The response body tells you the aggregate status at a glance.

```bash
curl -s http://<ABACA_API_HOST>:9797/health
```

Expected response (HTTP 200):

```json
{"status": "ok"}
```

### Kubernetes liveness probe — `GET /live`

This endpoint tells Kubernetes whether the process is alive. If it stops returning HTTP 200, Kubernetes restarts the container. Do not call it in application logic — it exists exclusively for the Kubernetes control plane.

```bash
curl -o /dev/null -w "%{http_code}" http://<ABACA_API_HOST>:9797/live
```

Expected output: `200`

### Kubernetes readiness probe — `GET /ready`

This endpoint tells Kubernetes whether the pod is ready to receive traffic. It returns HTTP 200 only when the service has completed initialisation and its dependencies are reachable. If you need to gate a workflow on service readiness (for example, in a CI pipeline after deploying a new version), poll this endpoint until it returns 200:

```bash
until curl -sf http://<ABACA_API_HOST>:9797/ready; do
  echo "Waiting for abaca-api to become ready..."
  sleep 5
done
echo "abaca-api is ready."
```

All three endpoints are unauthenticated — no Keystone token is required. This is intentional: probes must work before a tenant session exists, and none of the responses expose sensitive data.

## Examples

### Example 1 — Check overall health and parse the status field

Use `jq` to extract the status value for use in a shell conditional:

```bash
STATUS=$(curl -sf http://<ABACA_API_HOST>:9797/health | jq -r '.status')
if [ "$STATUS" = "ok" ]; then
  echo "abaca-api is healthy."
else
  echo "abaca-api reported status: $STATUS" >&2
  exit 1
fi
```

Expected output when healthy:

```
abaca-api is healthy.
```

---

### Example 2 — Liveness check returning the HTTP status code

```bash
curl -o /dev/null -w "%{http_code}\n" http://<ABACA_API_HOST>:9797/live
```

Expected output:

```
200
```

---

### Example 3 — Readiness poll in a CI/CD pipeline

Wait up to 60 seconds for `abaca-api` to become ready after a rollout, then fail the pipeline step if it does not:

```bash
MAX_ATTEMPTS=12
ATTEMPT=0
until curl -sf http://<ABACA_API_HOST>:9797/ready > /dev/null 2>&1; do
  ATTEMPT=$((ATTEMPT + 1))
  if [ "$ATTEMPT" -ge "$MAX_ATTEMPTS" ]; then
    echo "ERROR: abaca-api did not become ready after $((MAX_ATTEMPTS * 5))s" >&2
    exit 1
  fi
  echo "[${ATTEMPT}/${MAX_ATTEMPTS}] Not ready yet, retrying in 5s..."
  sleep 5
done
echo "abaca-api is ready after $((ATTEMPT * 5))s."
```

Expected output on success:

```
[1/12] Not ready yet, retrying in 5s...
[2/12] Not ready yet, retrying in 5s...
abaca-api is ready after 10s.
```

---

### Example 4 — OpenShift Deployment patch adding both probes

Apply this patch to add liveness and readiness probes to the existing base deployment without modifying the base manifest directly:

```yaml
# overlays/production/api-probe-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: abaca-api
spec:
  template:
    spec:
      containers:
        - name: abaca-api
          livenessProbe:
            httpGet:
              path: /live
              port: 9797
            initialDelaySeconds: 10
            periodSeconds: 15
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /ready
              port: 9797
            initialDelaySeconds: 5
            periodSeconds: 10
            failureThreshold: 3
```

Reference this patch in your `kustomization.yaml`:

```yaml
patchesStrategicMerge:
  - api-probe-patch.yaml
```

Apply with:

```bash
oc apply -k overlays/production/
```

> **Note for reviewer:** Confirm the actual `/live` and `/ready` route paths from Flask route definitions before finalising this example.

## Troubleshooting

### Symptom: `curl` returns `Connection refused` on port 9797

**Likely cause:** The `abaca-api` pod is not running or has not finished starting.

**Fix:**
1. Check pod status: `oc get pods -l app=abaca-api`
2. If the pod is in `CrashLoopBackOff`, inspect logs: `oc logs -l app=abaca-api --previous`
3. If the pod is `Pending`, check node resources and PVC bindings.
4. Ensure the Service object targets port 9797 and selects the `app=abaca-api` label.

---

### Symptom: `/health` returns HTTP 200 but `/ready` returns a non-200 status

**Likely cause:** `abaca-api` is alive but has not yet established connections to its dependencies (database, RabbitMQ, or Keystone).

**Fix:**
1. Inspect `abaca-api` logs for connection errors: `oc logs -l app=abaca-api`
2. Confirm MariaDB and RabbitMQ pods are running and their Services are reachable from the `abaca-api` pod.
3. Verify `/etc/abaca/abaca.conf` contains correct connection strings for the database and message broker.
4. Increase `initialDelaySeconds` on the readiness probe if the service consistently takes longer to connect on startup.

---

### Symptom: Kubernetes is restarting the `abaca-api` pod repeatedly

**Likely cause:** The liveness probe is timing out or returning a non-200 response, causing Kubernetes to treat the container as dead.

**Fix:**
1. Confirm the probe path and port match exactly: path `/live`, port `9797`.
2. Increase `initialDelaySeconds` (try `30`) if the process is still initialising when the first probe fires.
3. Increase `timeoutSeconds` if the service is under load and not responding within the default 1-second window.
4. Check whether a `failureThreshold` too low (for example, `1`) is causing premature restarts under transient load.

---

### Symptom: Health endpoint returns `404 Not Found`

**Likely cause:** The path you are requesting does not match the registered Flask route.

**Fix:**
1. Confirm the exact paths from the Flask route definitions in the `abaca-api` source (see note in Prerequisites).
2. Ensure you are not including a `/v1/` prefix — the health endpoints are not versioned API resources.
3. Check that no ingress or proxy is rewriting the path before it reaches the container.

---

### Symptom: Health checks pass but backup jobs fail immediately

**Likely cause:** The health endpoints confirm process liveness but do not validate the full data path (Manila connectivity, S3 reachability, Barbican access). The service can be "healthy" while dependencies specific to a tenant's workflow are misconfigured.

**Fix:** Use `abaca-dev` to run a live data-path probe against your OpenStack environment to validate end-to-end fidelity and capability before relying solely on HTTP health checks for operational readiness.