# Kube Provider

`--launcher Kube` submits Pods to an existing Kubernetes cluster. Nothing
is provisioned — the cluster is assumed to be there. The default
`dispatch` is `direct`: creating the Pod *is* the launch.

| Phase | What happens |
| ----- | ------------ |
| Provision | None. |
| Launch | One Pod created through `@kubernetes/client-node`, `restartPolicy: Never`. |
| Default dispatch | `direct` |

## Where the launch happens

Kube is the one launcher with **two working paths**, and they differ in who
holds the kubeconfig.

| | Who creates the Pod | Kubeconfig lives |
| --- | --- | --- |
| Control plane (default) | `@kubernetes/client-node`, server-side | in the control plane's Secret store |
| A nymph in the cluster | `kubectl apply`, from `runner/kube.rs` | on the box the nymph runs on |

The second path is the same shape SLURM uses: a nymph that can reach the
scheduler does the submitting itself, so the credential never leaves the
cluster. See [Running a daemon in-cluster](#running-a-daemon-in-cluster) for
when to prefer it — and for the `hostPath` constraint that limits it to
single-node clusters today.

> **Warning:** The spelling is enforced. `k8s`, `K8S`, and `local` are all rejected with a
> 400 — some of those appear in stale comments in the Prisma schema. The launcher
> name is exactly `Kube`.

## Kwargs

Connection (usually on the provider):

| Field | Default | Meaning |
| ----- | ------- | ------- |
| `kubeconfig` | — | A filesystem path, or inline kubeconfig content, or a `{ "$secret": … }` ref. Omit to use the server's `KUBECONFIG` / `~/.kube/config`. |
| `context` | — | Switches the active context after loading. |
| `namespace` | `default` | The Kubernetes namespace the Pod is created in. |

Pod shape (usually on the mode):

| Field | Required | Meaning |
| ----- | -------- | ------- |
| `image` | **yes** | Container image. Missing or empty fails the launch immediately. |
| `resources` | no | Passed straight through as the container's `resources` block — so it takes the real Kubernetes shape, `{ requests: {...}, limits: {...} }`. |
| `env` | no | Object of name → value. Numbers and booleans are stringified; anything else is skipped. |
| `node_selector` | no | → `spec.nodeSelector`. |
| `tolerations` | no | → `spec.tolerations`, verbatim list. |
| `service_account` | no | → `spec.serviceAccountName`. |
| `image_pull_secrets` | no | List of names → `spec.imagePullSecrets`. |

The Pod runs a single container named `user` with
`command: ["/bin/sh", "-c"]`.

> **Warning:** EC2 and GCE take the launch `script` as a bash userdata blob. The Kube
> launcher instead heredocs it into `/tmp/u.py` inside the container and
> runs `python3 /tmp/u.py`, so the image needs a `python3` on `PATH`. This
> is the one launcher where the `script` contract differs.

Labels always include `lakeshore=true`, `lakeshore.namespace=<ns>`, and
`lakeshore.provider=<name>` (slugified). Launch-body `tags` become extra
labels; anything colliding with the three reserved keys is dropped. The
`instances` listing selects on exactly those three.

Pod names are always auto-generated as `lakeshore-<8 hex>` — a `name`
kwarg is not read by this launcher.

## Example

**.dreamrc tab:** The same shape in a local `.dreamrc`:

**CLI**

```bash
lakeshore providers add my-cluster --launcher Kube \
  --kwarg context=prod-east \
  --kwarg namespace=lakeshore

lakeshore modes add k8s-gpu \
  --field provider=my-cluster \
  --field image=ghcr.io/lakeshore-py/cuda12.4-pytorch:2.4 \
  --field resources='{"limits":{"nvidia.com/gpu":1,"cpu":"4"}}' \
  --field node_selector='{"cloud.google.com/gke-accelerator":"nvidia-tesla-a100"}'
```

**.dreamrc**

```yaml file=".dreamrc"
providers:
  my-cluster: !providers.Kube
    context: prod-east
    namespace: lakeshore

modes:
  k8s-gpu:
    provider: my-cluster
    image: ghcr.io/lakeshore-py/cuda12.4-pytorch:2.4
    resources:
      limits: { nvidia.com/gpu: 1, cpu: "4" }
    node_selector: { "cloud.google.com/gke-accelerator": nvidia-tesla-a100 }
```

## Credentials

Same as `kubectl`. With no `kubeconfig` kwarg the server loads its
default config (`KUBECONFIG` or `~/.kube/config`). A `kubeconfig` string
that looks like a path (starts with `/`, `./`, or `~` and has no
newlines) is loaded from disk; anything else is treated as inline
kubeconfig content and staged to a temp file. A `{ "$secret": … }` ref
resolves to plaintext, gets written to a chmod-600 temp file for the
duration of the SDK call, and is cleaned up in a `finally`. See
[Secrets](https://lakeshore.dreamlake.ai/api/auth-and-secrets#secrets).

## Server-side routes

| Verb | Path | Result |
| ---- | ---- | ------ |
| `POST` | `/v1/namespaces/:ns/providers/:name/launch` | 202 `{ instanceId, state, providerName, launchedAt }` |
| `GET` | `/v1/namespaces/:ns/providers/:name/instances` | Pods matching the three reserved labels |
| `DELETE` | `/v1/namespaces/:ns/providers/:name/instances/:id` | 204 — deleting an already-gone Pod is idempotent |

`instanceId` is the Pod name. `state` comes from the Pod at create time:
a `deletionTimestamp` means `stopping` regardless of phase, otherwise
`Pending → starting`, `Running → running`, `Succeeded`/`Failed →
stopped`, anything else `unknown`. At create time it is normally
`starting`; poll `instances` to watch it become `running`.

## Running a daemon in-cluster

Set `dispatch: daemon` when the cluster cannot reach the control plane,
when you want every launch to flow through one in-cluster service you can
audit, or when you would rather not hand the control plane a kubeconfig
with broad permissions.

Note that `nymph` also has its own `kube` runner, which is a different
thing: a daemon *already running* can render a Pod manifest and
`kubectl apply` it per invocation. That runner mounts the daemon's
workdir into the Pod with a `hostPath` volume, so it only works when the
daemon and the Pod land on the same node — a single-node kind / minikube
/ k3d cluster, or a DaemonSet. Multi-node needs an RWX volume, which is
not built.

## Smoke test

```bash
lakeshore providers test my-cluster
lakeshore providers instances my-cluster
kubectl get pods -n lakeshore
```

Streaming stdout from the Pod back to the CLI is not implemented — read
it with `kubectl logs`.