Troubleshooting

This runbook helps you diagnose and resolve common errors encountered when operating tlsproxy, including TLS handshake failures, backend connectivity issues, and API request/response problems.

This runbook covers:

• TLS termination errors reported by the proxy
• HTTP API error codes returned by tlsproxy endpoints
• Backend forwarding failures (proxy cannot reach the plain-text backend)
• Certificate validation and configuration errors
• Connectivity verification between client, proxy, and backend

This runbook does not cover:

• Application-level bugs in your backend service
• Network infrastructure issues outside the host running tlsproxy (firewalls, load balancers, DNS)
• Operating system TLS library (OpenSSL/LibreSSL) upgrades or patching
• Performance tuning or capacity planning

Before using this runbook, ensure you have:

• tlsproxy installed and previously running (refer to the Installation guide if the proxy has never started successfully)
• Python 3.8+ and Twisted 22.x+ installed in the environment where tlsproxy runs
• Read access to the tlsproxy configuration file (typically tlsproxy.yaml or equivalent)
• Access to the tlsproxy process logs (stdout/stderr or your log aggregator)
• A terminal with curl (7.x+) or another HTTP client capable of sending HTTPS requests
• Network access to the proxy's listening address and port
• Credentials or API keys if your tlsproxy deployment requires authentication on its management endpoints

Work through the following steps in order. Stop at the step that resolves your issue.

Step 1 — Confirm tlsproxy is running

Verify the process is alive before investigating anything else.

# Check for a running process
ps aux | grep tlsproxy

Success: You see a tlsproxy process in the output with an expected PID.

If the process is absent, start the proxy according to your deployment procedure and confirm it starts cleanly before continuing.

Step 2 — Inspect recent log output

The proxy emits structured log lines for every connection attempt, TLS event, and backend forwarding decision. Reviewing these first saves significant debugging time.

# Tail the last 100 lines; adjust path to your log destination
tail -n 100 /var/log/tlsproxy/tlsproxy.log

Look for lines containing ERROR, CRITICAL, or Unhandled. Note the timestamp, error code, and any stack trace.

Success: You identify a specific error message to investigate in subsequent steps.

Step 3 — Test TLS connectivity directly

Isolate whether the issue is at the TLS layer or deeper by making a raw HTTPS connection to the proxy.

curl -v --cacert /path/to/your-ca.crt \
     https://<proxy-host>:<proxy-port>/

• If you see SSL certificate verify ok and receive any HTTP response (even an error), TLS termination is working.
• If you see SSL handshake failed, certificate verify failed, or connection refused, the problem is at the TLS layer — proceed to Step 4.
• If TLS succeeds but you receive an HTTP 5xx error, skip to Step 6.

Success: curl completes the TLS handshake without certificate errors.

Step 4 — Diagnose TLS certificate errors

TLS handshake failures are usually caused by one of three problems: an expired certificate, a hostname mismatch, or an untrusted CA.

# Inspect the certificate the proxy is presenting
openssl s_client -connect <proxy-host>:<proxy-port> -servername <expected-hostname> 2>&1 \
  | openssl x509 -noout -text | grep -E 'Subject:|Issuer:|Not Before:|Not After:'

Check the output:

Success: The certificate shows a future expiry, matching hostname, and a trusted issuer.

Step 5 — Validate tlsproxy configuration

A misconfigured file silently causes the proxy to forward to the wrong backend or bind to an unexpected address.

# Validate config syntax (adjust command to match your deployment)
python -m tlsproxy --config tlsproxy.yaml --validate

Review these fields in your configuration:

• listen_address / listen_port — must match the address clients connect to
• backend_host / backend_port — must resolve to a reachable plain-text backend
• tls_cert / tls_key — paths must be readable by the user running tlsproxy

Success: The validate command exits with code 0 and prints no warnings.

Step 6 — Test backend reachability from the proxy host

Because tlsproxy forwards decrypted traffic to a plain-text backend, a backend that is down or unreachable causes 502 Bad Gateway or 503 Service Unavailable responses.

# From the host running tlsproxy, test the plain-text backend directly
curl -v http://<backend-host>:<backend-port>/healthz

If this request fails:

• Confirm your backend service is running.
• Check that no firewall rule blocks traffic from the tlsproxy host to the backend host/port.
• Confirm backend_host and backend_port in tlsproxy config exactly match this address.

Success: curl returns a 2xx response from the backend.

Step 7 — Reproduce the failing API request with verbose output

If a specific API endpoint is returning unexpected errors, capture the full request and response cycle.

curl -v \
     --cacert /path/to/your-ca.crt \
     -H "Authorization: Bearer <your-token>" \
     -H "Content-Type: application/json" \
     -X POST https://<proxy-host>:<proxy-port>/your/endpoint \
     -d '{"key": "value"}'

Record:

• The HTTP status code
• The Content-Type and body of the error response
• Any X-Request-Id or correlation header returned by the proxy

Cross-reference the status code with the table below:

Success: You have a precise error code and message to act on.

Step 8 — Restart tlsproxy

If logs show the proxy is in an error state (stuck connections, corrupted internal state), a controlled restart often resolves transient issues.

# Graceful restart — allows in-flight connections to complete
systemctl restart tlsproxy   # systemd environments
# OR
kill -HUP <tlsproxy-pid>     # SIGHUP for graceful reload if supported

Success: The process restarts cleanly, logs show it is listening, and new connections succeed.

After completing the relevant steps above, confirm the runbook has resolved the issue by checking all of the following:

1. Process health — ps aux | grep tlsproxy shows a single healthy process.

2. Clean log stream — Tailing the log shows no new ERROR or CRITICAL lines for at least 60 seconds after your last corrective action:

   Copy

   tail -f /var/log/tlsproxy/tlsproxy.log
   

3. Successful TLS handshake — The following command completes without any SSL error:

   Copy

   curl -sS --cacert /path/to/your-ca.crt \
        -o /dev/null -w "%{http_code}" \
        https://<proxy-host>:<proxy-port>/
   

   Expected output: an HTTP status code (200, 404, etc.) — any value other than a curl SSL error confirms TLS is functioning.

4. End-to-end API call succeeds — Re-run the specific API request that was originally failing and confirm you receive the expected 2xx response.

5. No regression in connected clients — If your environment has monitoring, verify that error-rate metrics for the tlsproxy listener have returned to baseline.

The steps in this runbook are primarily diagnostic. However, if you made configuration changes or restarted the proxy and the situation has worsened, take these actions:

1. Restore the previous configuration file

   If you edited tlsproxy.yaml (or equivalent), revert to the last known-good version:

   Copy

   # If you made a backup before editing
   cp tlsproxy.yaml.bak tlsproxy.yaml
   

   Always create a timestamped backup (cp tlsproxy.yaml tlsproxy.yaml.$(date +%Y%m%d%H%M%S)) before editing configuration files.

2. Restart with the restored configuration

   Copy

   systemctl restart tlsproxy
   

3. Restore the previous TLS certificate

   If you replaced a certificate file, copy the original back to the path specified by tls_cert in your configuration, then restart the proxy.

4. Re-pin to a known-good process state

   If a restart made things worse and you have a process supervisor (e.g., systemd, supervisord), stop the proxy entirely and investigate logs before starting it again:

   Copy

   systemctl stop tlsproxy
   journalctl -u tlsproxy -n 200
   

5. Communicate impact

   If the proxy is unavailable and you cannot restore service within your SLA window, notify stakeholders and proceed to the Escalation section.

If you have completed all steps in this runbook and the issue persists, escalate with the following information pre-gathered to reduce resolution time:

Who to contact

Information to provide

Collect and attach or paste the following when escalating:

1. tlsproxy version — output of python -m tlsproxy --version (or equivalent)
2. Python and Twisted versions — output of python --version and python -c "import twisted; print(twisted.__version__)"
3. Operating system and kernel — output of uname -a
4. Sanitized configuration file — remove private keys, secrets, and internal hostnames before sharing
5. Log excerpt — the 200 lines surrounding the first error occurrence (use grep -n 'ERROR\|CRITICAL' tlsproxy.log to find the line number)
6. Exact curl command and full output — including the -v verbose flag output from Step 7
7. Timeline — when the issue first appeared, any recent changes to configuration, certificates, backend services, or the host environment
8. Steps already attempted — which steps from this runbook you completed and their outcomes

Providing this information upfront prevents back-and-forth and significantly accelerates resolution.