Development Guide

This guide explains how to contribute to the Site Recovery operator: how to build the control plane and UI locally, run the unit and webhook test suites, extend the operator with new CRDs or controllers, and keep generated files in sync. Site Recovery is a Kubebuilder-based Go operator targeting OpenShift 4.17+; understanding the project layout and the code-generation pipeline is essential before making changes, because several files are auto-generated and must never be edited by hand.

Before you begin, ensure you have the following tools and accounts:

• Go 1.26 — matches the module version in go.mod
• Docker (or a compatible CONTAINER_TOOL) for building images
• kubectl or oc CLI, configured against an OpenShift 4.17+ cluster (for manual testing)
• Helm ≥ 3.0 — Helm charts are the only supported deployment path
• kubebuilder CLI — required to scaffold new APIs, controllers, and webhooks; do not create scaffold files manually
• controller-gen v0.19.0 — installed automatically by make manifests; do not upgrade without updating the Makefile pin
• golangci-lint — installed automatically by make lint into bin/golangci-lint-custom
• Node.js + npm — for the Next.js frontend under ui/
• Python ≥ 3.10 (3.11 recommended) — for the Flask backend under ui/
• git — the version tag and commit hash are injected into binaries at build time via -ldflags
• Read access to the container registry eu.gcr.io/amazing-chalice-243510 (default for local/dev builds) or quay.io/triliodata (release branches)

All CRD types live under the API group siterecovery.trilio.io/v1alpha1. Familiarize yourself with the Kubebuilder Book (https://book.kubebuilder.io) and controller-runtime FAQ before writing reconcilers.

Follow these steps to set up the project for local development.

1. Clone the repository

git clone https://github.com/trilioData/site-recovery.git
cd site-recovery

2. Install Go toolchain dependencies

go mod download

This fetches all Go module dependencies declared in go.mod, including controller-runtime and controller-tools.

3. Generate CRDs and DeepCopy methods

Run both generation targets before doing anything else. This installs controller-gen into bin/ if it is not already present.

make manifests generate

• make manifests runs controller-gen over api/v1alpha1/ and writes CRD YAML files into deploy/crds/ (the single canonical location), then calls make sync-crds to copy the per-chart CRD subsets into helm/*/crds/.
• make generate regenerates zz_generated.deepcopy.go files.

Never edit deploy/crds/*.yaml, helm/*/crds/*.yaml, or any zz_generated.*.go file by hand.

4. Build the control-plane binaries

make build

This runs manifests, generate, fmt, and vet before compiling. Two binaries are produced:

Both binaries are packaged into the single site-recovery-control-plane container image.

5. Build the diagnostic tool (optional)

make build-tsr-gather

Produces bin/tsr-gather, the must-gather compatible diagnostic collector.

6. Install the UI dependencies

# Next.js frontend
cd ui
npm install --no-audit --no-fund --legacy-peer-deps

# Flask backend
pip install -r requirements.txt

7. Build container images

Export an image tag and build:

export IMG=eu.gcr.io/amazing-chalice-243510/site-recovery-control-plane:dev
make docker-build IMG=$IMG

For the ProtectionZone controller image:

make docker-build-pz

For the UI frontend and backend images:

./scripts/build-images.sh

8. Deploy to a cluster via Helm

Helm is the only supported deployment path. Load the image into your cluster (or push it to a registry your cluster can pull from), then:

make deploy IMG=$IMG

This runs helm upgrade --install using the chart selected by HELM_CHART. See the Helm chart values files under helm/ for tunable parameters.

Site Recovery's build and test pipeline is configured through Makefile variables, Go +kubebuilder markers in api/v1alpha1/*_types.go, and Helm chart values under helm/*/values.yaml.

Makefile variables

Override any variable on the command line:

make test TEST_PKG_PARALLELISM=2
make docker-build CONTAINER_TOOL=podman IMG=myregistry/site-recovery:local

API field markers (api/v1alpha1/*_types.go)

Kubebuilder markers on struct fields control CRD schema generation. Use the following patterns:

// +kubebuilder:validation:Required
// +kubebuilder:validation:Minimum=1
// +kubebuilder:validation:MaxLength=253
// +kubebuilder:validation:Pattern="^[a-z0-9-]+$"
// +kubebuilder:default="Asynchronous"

Always use metav1.Condition for status conditions and metav1.Time for timestamps — not plain strings.

After any marker change, regenerate:

make manifests generate

RBAC markers (pkg/controllers/*_controller.go)

RBAC is generated from markers in controller files and is also hand-maintained in the Helm chart templates (the Kustomize config/ scaffold was removed). Keep both in sync:

// +kubebuilder:rbac:groups=siterecovery.trilio.io,resources=failoverrequests,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=siterecovery.trilio.io,resources=failoverrequests/status,verbs=get;update;patch
// +kubebuilder:rbac:groups=siterecovery.trilio.io,resources=failoverrequests/finalizers,verbs=update

Logging configuration

Log level and encoder are set at runtime via flags, not at compile time. The Helm charts expose these as values:

Never hardcode Development: true in any binary's logger initialization; all binaries use the production preset (JSON encoder, RFC3339 timestamps).

Day-to-day development loop

The typical change cycle for a controller change is:

1. Edit Go files under pkg/controllers/ or api/v1alpha1/.
2. If you changed type definitions or markers: make manifests generate
3. Fix style: make lint-fix
4. Run tests: make test-unit
5. Build binaries: make build
6. Build and load the image: make docker-build IMG=<your-ref> then push or load into your cluster.
7. Deploy: make deploy IMG=<your-ref>

Running the control plane locally

You can run either manager binary against your current kubeconfig context without building an image:

# Quorum control plane (reconcilers: protection, failover, pg-sync, test-failover, replication-monitor)
make run

# Workload control plane (admission webhooks only, no reconcilers)
make run-workload

Running the UI locally

# Terminal 1: Next.js dev server (port 3000)
cd ui
npm run dev

# Terminal 2: Flask API backend
cd ui
python app.py

Scaffolding a new API kind

Always use the kubebuilder CLI; never create files manually.

kubebuilder create api \
  --group siterecovery \
  --version v1alpha1 \
  --kind MyNewKind

This produces:

• api/v1alpha1/myNewKind_types.go — edit this to define your spec and status
• pkg/controllers/mynewkind_controller.go — reconciler skeleton
• Test file stubs

After editing the types file:

make manifests generate

Then assign the new CRD to the correct chart by adding its plural name to either QUORUM_CRDS, WORKLOAD_CRDS, or PROTECTIONZONE_CRDS in the Makefile, and re-running make manifests. The make verify-crds target will fail in CI if the CRD is not assigned.

Scaffolding a webhook

kubebuilder create webhook \
  --group siterecovery \
  --version v1alpha1 \
  --kind MyNewKind \
  --defaulting --programmatic-validation

Webhook logic lives in pkg/webhook/. Do not delete any // +kubebuilder:scaffold:* comments.

Adding a controller for an external type

To watch a KubeVirt resource (for example):

kubebuilder create api \
  --group kubevirt.io \
  --version v1 \
  --kind VirtualMachine \
  --controller=true --resource=false \
  --external-api-path=kubevirt.io/api/core/v1 \
  --external-api-domain=io

Structured logging conventions

All new code must use the context-acquired logger, never a package-level variable:

log := logf.FromContext(ctx)
log.Info("Starting reconciliation", "protectionGroup", req.Name)
log.V(logging.Debug).Info("Checking volume state", "pvc", pvcName, "node", nodeName)
log.Error(err, "Failed to promote DRBD volume", "drbdResource", resourceName, "cluster", clusterName)

Use the named verbosity constants from internal/logging (logging.Debug = 1, logging.Trace = 2). Never use bare V(1) or V(2). Never log CR objects, Secret data, kubeconfigs, or tokens.

Example 1: Full development build cycle

Build both manager binaries with version metadata injected:

export GIT_COMMIT=$(git rev-parse --short HEAD)
export BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ)
export VERSION=dev-$(git rev-parse --abbrev-ref HEAD)
make build

Expected output (abbreviated):

go build -ldflags "-X github.com/trilioData/site-recovery/internal/version.Version=dev-main ..." \
  -o bin/quorum-manager cmd/quorum-control-plane/main.go
go build -ldflags "..." \
  -o bin/workload-manager cmd/workload-control-plane/main.go

Verify the binaries exist:

ls -lh bin/quorum-manager bin/workload-manager

Example 2: Running unit tests with reduced parallelism

On a laptop with limited cores, cap concurrent envtest instances to avoid startup timeouts:

make test-unit TEST_PKG_PARALLELISM=2

Expected output (abbreviated):

KUBEBUILDER_ASSETS="/path/to/bin/k8s/..." go test -p 2 \
  github.com/trilioData/site-recovery/pkg/controllers/... \
  -coverprofile cover.out
ok  github.com/trilioData/site-recovery/pkg/controllers/failoverrequest  12.4s
ok  github.com/trilioData/site-recovery/pkg/controllers/protectionrequest 9.7s
...

Example 3: Running only controller tests

make test-controllers

Produces cover-controllers.out. Useful when iterating on reconciler logic without running webhook tests.

Example 4: Running only webhook tests

make test-webhooks

Produces cover-webhooks.out.

Example 5: Running UI tests

# Both frontend (vitest) and backend (pytest) in one command
make test-ui-unit

# Backend only
make test-ui-backend-unit

# Frontend only
make test-ui-frontend-unit

Expected backend output (abbreviated):

created venv at ui/.venv-test
collected 42 items
...
42 passed in 3.21s

Example 6: Checking CRD drift in CI

After editing api/v1alpha1/*_types.go, verify that generated files are committed:

make verify-crds

If CRDs are out of sync:

CRDs are out of sync with api/v1alpha1. Run 'make manifests' and commit the result:
 deploy/crds/siterecovery.trilio.io_protectionrequests.yaml | 4 ++--
 helm/site-recovery-quorum-control-plane/crds/siterecovery.trilio.io_protectionrequests.yaml | 4 ++--

Fix by running:

make manifests
git add deploy/crds/ helm/
git commit -m "Regenerate CRDs after types change"

Example 7: Building and pushing the control-plane image

export IMG=quay.io/triliodata/site-recovery-control-plane:v1.2.0
make docker-build docker-push IMG=$IMG

The Dockerfile at docker-images/control-plane/Dockerfile receives VERSION, GIT_COMMIT, and BUILD_DATE as build args, ensuring the startup log banner matches the binary version string.

Example 8: Checking linter output and auto-fixing

# Show all lint issues
make lint

# Auto-fix fixable issues
make lint-fix

The linter uses the custom build at bin/golangci-lint-custom. The logcheck plugin enforces that logging keys are string literals, not Go constants.

CRD drift causes make verify-crds to fail in CI

Symptom: CI fails with:

CRDs are out of sync with api/v1alpha1. Run 'make manifests' and commit the result.

Cause: A *_types.go file was edited but make manifests was not run before committing, so the generated CRD YAMLs in deploy/crds/ or the Helm chart crds/ copies are stale.

Fix:

make manifests generate
git add deploy/crds/ helm/
git commit -m "chore: regenerate CRDs"

make manifests fails with "CRD kind not assigned to any Helm chart"

Symptom:

CRD kind(s) generated in deploy/crds but not assigned to any Helm chart: mynewkinds

Cause: You added a new API kind but did not assign its CRD plural name to one of QUORUM_CRDS, WORKLOAD_CRDS, or PROTECTIONZONE_CRDS in the Makefile.

Fix: Open the Makefile and add the plural name (lowercase, matching the filename stem) to the correct variable, then re-run make manifests.

envtest packages time out with "waiting for apiserver"

Symptom: Individual test packages fail with:

waiting for apiserver to come up: context deadline exceeded

Cause: Too many envtest processes starting simultaneously on a CPU-constrained machine. Each package boots its own kube-apiserver + etcd; the default parallelism of 4 may be too high.

Fix: Lower TEST_PKG_PARALLELISM and increase the start timeout:

make test-unit TEST_PKG_PARALLELISM=2 KUBEBUILDER_CONTROLPLANE_START_TIMEOUT=4m

make run fails with "no such file or directory" for a CRD

Symptom: The manager starts but immediately logs errors about missing CRD definitions.

Cause: CRDs have not been applied to the cluster the kubeconfig points to.

Fix: Apply the CRDs before running the manager:

kubectl apply -f deploy/crds/
make run

UI backend fails to start: ModuleNotFoundError

Symptom: python app.py exits immediately with import errors.

Cause: Python dependencies from ui/requirements.txt are not installed, or the active Python version is below 3.10.

Fix:

cd ui
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python app.py

make lint fails with "logcheck: key must be a string literal"

Symptom: The linter reports:

pkg/controllers/failoverrequest/controller.go:87:12: logcheck: key must be a string literal

Cause: A logging call uses a Go constant or variable as a key instead of a string literal, violating the logcheck rule enforced by golangci-lint.

Fix: Replace the constant with its string literal value:

// Wrong
log.Info("Failover started", keyProtectionGroup, pg.Name)

// Correct
log.Info("Failover started", "protectionGroup", pg.Name)

kubebuilder create api scaffold markers disappear after re-running with --force

Symptom: After running kubebuilder create webhook --force, custom webhook logic is overwritten.

Cause: The --force flag regenerates scaffold files, replacing any custom code.

Fix: Before using --force, save your custom logic:

cp pkg/webhook/myresource_webhook.go /tmp/myresource_webhook.go.bak
kubebuilder create webhook --group siterecovery --version v1alpha1 --kind MyResource \
  --defaulting --programmatic-validation --force
# Restore custom logic

Never remove // +kubebuilder:scaffold:* comments; they are injection points for future CLI operations.