<!-- source: https://docs.dreamlake.ai/examples/lakeshore/kubernetes-job @ 0.1.192 b2f973b1 -->
# Kubernetes jobs and storage

Use an existing cluster and an explicitly configured user launch server. The
self-contained [22-kubernetes-job example](https://github.com/dreamlake-ai/lakeshore-examples/tree/66f622023e648519012a9f60658b4954b75798fe/22-kubernetes-job)
contains native Kubernetes manifests, offline preparation/tests and admin/user
setup guides. These are concrete reference fixtures, not a claim that all
standard DreamLake mount methods have been integrated into the current runner.
No Nautilus access, infrastructure apply or live GPU acceptance was performed
for this source iteration.

## Choose a starting point

The [configuration catalog](https://github.com/dreamlake-ai/lakeshore-examples/blob/66f622023e648519012a9f60658b4954b75798fe/22-kubernetes-job/docs/configurations.md)
covers existing EKS, GKE, AKS, shared research clusters, private APIs,
self-managed clusters and local development. A kubeconfig selects endpoint,
trust, identity and context; it does not supply permissions, GPU capacity or
storage drivers. Read selected credential metadata before requesting secrets.
Use explicit config/context/namespace without changing global settings.

Administrators provide the configured identity/network route, namespace/RBAC,
image access, storage and accelerator prerequisites. Users select workloads and
already-granted resources. The launch server submits to Kubernetes; the scheduled
Pod runs the workload. Kubernetes controls individual container status/logs/exec,
with DreamLake enforcing user authorization. Host Nymph supports admin health and
maintenance; optional job Nymph has job-limited permissions and lifecycle.
Neither is a reason to expose host credentials to every container.

## Examples and verification

The folder provides CPU hello, one-GPU CUDA visibility, private archive download
through an init container, existing-PVC checkpoint save/resume, two indexed Pods
using shared storage, and multiple containers with individually selected logs.
Separate optional PVC templates illustrate block-backed RWO and shared RWX
filesystems. Applying a PVC may allocate billable storage; reuse existing claims
unless creation is separately authorized.

Run the folder's Python 3.12+ unit tests offline, then render a native manifest
with explicit namespace/name and reviewed image digest. Rendering does not
submit work. On the selected launch server, run the read-only preflight, inspect
the manifest and API dry-run, and create only the reviewed diagnostic Job. The
[user guide](https://github.com/dreamlake-ai/lakeshore-examples/blob/66f622023e648519012a9f60658b4954b75798fe/22-kubernetes-job/docs/user-setup.md)
provides commands and exact-resource cleanup. Retain launch identity, source/image
hashes, Job/Pod UIDs, exit codes, bounded logs and checkpoint readback. Local tests,
registration and node readiness do not prove a job executed.

## Mounts and the remaining adapter work

Standard launch definitions remain authoritative. Jaynes' Nautilus example
translated S3Code mounts into init containers, volumes and volume mounts; its
main example used an ephemeral memory volume, while another used a CephFS RWX
claim. The inspected Nymph runner instead assumes a daemon-local workdir can be
mounted at the same hostPath on a remote worker. The missing work is to map the
standard mount intent into Pod preparation and storage, preserving destinations,
workdir, access mode, topology and durable output semantics.

The native download fixture demonstrates that pattern using private scoped
artifact access and digest validation. It does not implement the production
snapshot protocol. Scientific acceptance continues through the canonical launch
interface, with GPU timings and independent metrics/checkpoint readback. See the
[integration contract](https://github.com/dreamlake-ai/lakeshore-examples/blob/66f622023e648519012a9f60658b4954b75798fe/22-kubernetes-job/docs/integration.md)
for capability gates; unsupported methods must fail before submission.

## Terraform and recovery

The existing EKS Terraform remains under
[infra/user-launch/kube-launch-20261015](https://github.com/dreamlake-ai/dreamlake-workspace/tree/setup/kube-terraform-profiles-20261016/infra/user-launch/kube-launch-20261015).
Its topology profiles are partial review inputs, not copies of the live stack or
a provisioning action. Preserve existing HTTP backend/state, resource ownership
and current operational inputs. Offline mock plans validate capacity and network
constraints without contacting AWS. For existing-cluster setup, skip provisioning.

After interruption, inspect the original Job before retrying; preserve durable
checkpoints and delete only the recorded disposable Jobs. Do not delete existing
PVCs, namespaces, agents or clusters. The troubleshooting guide distinguishes
authentication, authorization, scheduling, image, init-download, storage and CUDA
failures, with an explicit administrator handoff for shared prerequisites.
