---
title: Troubleshooting
product: csi-driver-host-path
doc_type: runbook
version: master
source: git2docs (code-derived, validation-filtered)
canonical: https://git2docs.com/xing-yang/docs/csi-driver-host-path/csi-hostpath-driver/troubleshooting
---

# Troubleshooting

_Common API errors and debugging_

## Objective

Use this runbook to diagnose and resolve common errors encountered when deploying the CSI Hostpath driver, provisioning volumes, or verifying driver operation in a Kubernetes cluster.

## Scope

This runbook covers troubleshooting for:
- Failed or incomplete driver deployment
- Pods not reaching `Running` status after deployment
- PVCs stuck in `Pending` state
- Volumes not mounting correctly inside application pods
- VolumeAttachment objects missing or in an error state
- Snapshot provisioning failures

This runbook does **not** cover:
- Rebuilding the driver binary from source
- Multi-node cluster networking issues unrelated to CSI
- Kubernetes control plane failures
- Snapshot feature gate configuration beyond enabling `VolumeSnapshotDataSource`

## Prerequisites

Before following this runbook, ensure you have:

- A running Kubernetes cluster at version **1.13 or later**
- `kubectl` installed and configured with access to the cluster
- The CSI Hostpath driver previously deployed via `deploy/kubernetes-<version>/deploy-hostpath.sh`
- Sufficient RBAC permissions to describe pods, inspect events, and exec into containers
- For snapshot-related issues: the `VolumeSnapshotDataSource` feature gate enabled (required for Kubernetes v1.12–v1.16 where snapshots are alpha)

## Steps

Work through these steps in order. Stop at the step that resolves your issue.

1. **Check that all driver pods are running.**

   Run:
   ```shell
   kubectl get pods
   ```
   Expected output shows all four pods with `STATUS: Running` and no unexpected restarts:
   ```
   csi-hostpath-attacher-0      1/1     Running   0
   csi-hostpath-provisioner-0   1/1     Running   0
   csi-hostpath-snapshotter-0   1/1     Running   0
   csi-hostpathplugin-0         2/2     Running   0
   ```
   If any pod is in `Pending`, `CrashLoopBackOff`, or `Error` state, proceed to step 2.

2. **Inspect the failing pod for errors.**

   Replace `<pod-name>` with the name of the unhealthy pod:
   ```shell
   kubectl describe pod <pod-name>
   ```
   Scroll to the **Events** section at the bottom of the output. Common issues to look for:
   - `FailedScheduling` — the node may lack resources or a required label
   - `ImagePullBackOff` — the container image (e.g., `quay.io/k8scsi/hostpathplugin:v1.0.1`) could not be pulled; verify network access to `quay.io`
   - `CrashLoopBackOff` — the container is starting and immediately exiting; retrieve logs in step 3

3. **Retrieve container logs for the crashing component.**

   For the main plugin container:
   ```shell
   kubectl logs csi-hostpathplugin-0 -c hostpath
   ```
   For the node driver registrar sidecar:
   ```shell
   kubectl logs csi-hostpathplugin-0 -c node-driver-registrar
   ```
   For the liveness probe sidecar:
   ```shell
   kubectl logs csi-hostpathplugin-0 -c liveness-probe
   ```
   Look for stack traces, permission errors, or socket-related failures. The liveness probe monitors CSI service health; repeated failures here indicate the driver gRPC server is not responding.

4. **Diagnose a PVC stuck in `Pending` state.**

   Run:
   ```shell
   kubectl get pvc
   kubectl describe pvc csi-pvc
   ```
   In the **Events** section, look for provisioner errors. A healthy bound PVC looks like:
   ```
   NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS
   csi-pvc   Bound    pvc-58d5ec38-03e5-11e9-be51-000c29e88ff1   1Gi        RWO            csi-hostpath-sc
   ```
   If the PVC remains `Pending`:
   - Confirm the `csi-hostpath-sc` StorageClass exists: `kubectl get storageclass`
   - Confirm the provisioner pod `csi-hostpath-provisioner-0` is running (step 1)
   - Check provisioner logs: `kubectl logs csi-hostpath-provisioner-0`

5. **Diagnose an application pod that fails to start or mount the volume.**

   Run:
   ```shell
   kubectl describe pods/my-csi-app
   ```
   Check the **Events** section for `FailedMount` or `FailedAttach` errors. Then inspect the attacher logs:
   ```shell
   kubectl logs csi-hostpath-attacher-0
   ```
   A successfully attached volume produces a `VolumeAttachment` object with `Status.Attached: true`. Verify this in step 6.

6. **Verify the VolumeAttachment object exists and is healthy.**

   Run:
   ```shell
   kubectl describe volumeattachment
   ```
   Look for `Status.Attached: true` in the output:
   ```yaml
   Status:
     Attached: true
   ```
   If `Attached` is `false` or the object is missing entirely, the external attacher is not communicating with the driver. Re-check attacher pod logs (step 5) and confirm the `csi-hostpath-attacher` service is present:
   ```shell
   kubectl get svc csi-hostpath-attacher
   ```

7. **Confirm volume data path is working end-to-end.**

   Exec into the application pod and write a test file:
   ```shell
   kubectl exec -it my-csi-app /bin/sh
   / # touch /data/hello-world
   / # exit
   ```
   Then exec into the hostpath plugin container and search for the file:
   ```shell
   kubectl exec -it $(kubectl get pods --selector app=csi-hostpathplugin -o jsonpath='{.items[*].metadata.name}') -c hostpath /bin/sh
   / # find / -name hello-world
   /tmp/057485ab-c714-11e8-bb16-000c2967769a/hello-world
   / # exit
   ```
   If the file is **not** found under `/tmp` inside the hostpath container, the volume bind-mount is broken. Recheck pod scheduling — both the app pod and the plugin DaemonSet pod must be on the same node.

8. **Diagnose snapshot provisioning failures.**

   Confirm the snapshot class exists:
   ```shell
   kubectl get volumesnapshotclass
   ```
   Expected output:
   ```
   NAME                     AGE
   csi-hostpath-snapclass   11s
   ```
   If the class is missing, re-run the deployment script which creates it. If snapshots fail after the class exists, verify the `VolumeSnapshotDataSource` feature gate is enabled on your cluster's API server and that the snapshotter pod is healthy:
   ```shell
   kubectl logs csi-hostpath-snapshotter-0
   ```
   Inspect the snapshot object for status details:
   ```shell
   kubectl describe volumesnapshot
   ```
   Look for `Status.Ready: true`. A `false` value accompanied by an error message in Events points to a driver-side failure.

## Verification

After completing the relevant steps above, confirm the full workflow is healthy by checking all of the following observable outcomes:

- **All pods running:**
  ```shell
  kubectl get pods
  ```
  All four driver pods report `STATUS: Running` with `RESTARTS: 0` (or a stable low number).

- **PVC bound:**
  ```shell
  kubectl get pvc
  ```
  The `csi-pvc` entry shows `STATUS: Bound`.

- **PV exists:**
  ```shell
  kubectl get pv
  ```
  A corresponding PersistentVolume with `STATUS: Bound` is listed.

- **Application pod running:**
  ```shell
  kubectl describe pods/my-csi-app
  ```
  `Status: Running` and all conditions (`Initialized`, `Ready`, `ContainersReady`, `PodScheduled`) show `True`.

- **VolumeAttachment attached:**
  ```shell
  kubectl describe volumeattachment
  ```
  `Status.Attached: true`.

- **Data path verified:** The `hello-world` file written in the app pod is discoverable under `/tmp/<volume-id>/hello-world` inside the hostpath container.

## Rollback

If you need to remove the CSI Hostpath driver deployment and return the cluster to a clean state:

1. **Delete the example application resources** (pod, PVC, StorageClass):
   ```shell
   kubectl delete -f ./examples
   ```
   Wait until the PVC is fully deleted before proceeding, as the Delete reclaim policy will trigger volume cleanup on the driver side.

2. **Delete snapshot resources** if you created them:
   ```shell
   kubectl delete volumesnapshot new-snapshot-demo
   kubectl delete volumesnapshotclass csi-hostpath-snapclass
   ```

3. **Delete the driver components.** Re-run the deployment manifests with `delete` instead of `apply`:
   ```shell
   kubectl delete -f deploy/kubernetes-<version>/hostpath/
   ```
   This removes the StatefulSets and Services for the attacher, provisioner, snapshotter, and plugin.

4. **Remove RBAC rules** by deleting the service accounts, cluster roles, and bindings created during deployment for `csi-provisioner`, `csi-attacher`, and `csi-snapshotter`.

> **Note for reviewer:** The source material does not include an explicit teardown or uninstall script. The steps above are inferred from the deployment procedure. If an official uninstall script exists, it should be referenced here.

## Escalation

If you have followed all steps in this runbook and the issue is not resolved, escalate through the following channels:

**Kubernetes CSI community (primary channel):**
- **Slack:** [http://slack.k8s.io/](http://slack.k8s.io/) — search for the `#csi` or `#sig-storage` channels
- **Mailing List:** [https://groups.google.com/forum/#!forum/kubernetes-dev](https://groups.google.com/forum/#!forum/kubernetes-dev)

**When escalating, collect and provide the following information:**

1. Kubernetes version: `kubectl version`
2. Full pod listing: `kubectl get pods -o wide`
3. Logs from all four driver pods (attacher, provisioner, snapshotter, plugin — all containers)
4. Output of `kubectl describe pvc`, `kubectl describe volumeattachment`, and `kubectl describe pods/my-csi-app`
5. The exact error message or unexpected behavior you observed
6. The deployment script and Kubernetes version directory used (e.g., `deploy/kubernetes-1.13/`)
7. Any customizations made to the default manifests or example files