Lakeshore
Lakeshore runs your Python functions on machines that are not your laptop. You
decorate a function with @udf and call it; Lakeshore queues the invocation,
ships it to a worker, runs it, and hands the result back. You write Python — it
handles the scheduling, the transport, and the worker side.
You reach for Lakeshore when the work does not fit on the machine in front of you: a GPU job, a long sweep, or a fan-out of identical calls you want to run all at once. Nothing about the function has to change to move it off your laptop — the decorator is the only difference between a local call and a remote one.
Machine access, worker runtime and queue access
These are separate layers, not mutually exclusive kinds of machines:
- Fabric — Add Host: enroll a machine to establish its identity, connection and remote-management capability. Use an authorized execution connection; no task queue or queue mount is a prerequisite.
- Runtime — run a queue worker: start a task-consumer process on a machine, associate it with the intended queue and verify task execution. A host can be used for development and can also run workers. Host enrollment alone does not configure a queue consumer.
- Client queue access — queue mount: save a connection for accessing queues and inspecting jobs and workers. This neither enrolls a host nor starts a worker. An online queue mount cannot authorize Add Host.
Start with connection selection and host enrollment for machine access. For task execution, see queues, worker lifecycle and the Nymph daemon reference.
The three layers
Lakeshore ships as three pieces, each deployed and upgraded on its own:
| Layer | What it is | What it does |
|---|---|---|
| Client | DreamLake CLI (dreamlake) and Python SDK (dreamlake-lakeshore) | Register hosts, launch work and manage DreamLake resources; write Python @udf workloads |
| Control plane | A single service that owns the authoritative state | Holds queues, workers, providers, and pending invocations; dispatches work |
| Runtime | The nymph daemon on each compute host | Long-polls the control plane and executes invocations next to your data |
The daemon connects outbound only, so a worker can sit behind a firewall, in a cluster, or on a machine you already own, and still take work. What it does between starting and taking that work — coming up, running its setup, declaring itself ready, and saying so on the way out — is the worker lifecycle.
Use dl for user-facing host registration, tracked runs, providers,
queues, authentication and Vault. Lakeshore names the underlying compute
service and Python SDK. The older lakeshore CLI remains a low-level operator
client in some service setup guides; it is not a second required user login or
an alternative to the DreamLake CLI.
Start here
Run a Python function through a local queue, then point the SDK at a service.
Host the control-plane server and create its first namespace and client login.
Add an existing service and inspect its queues, workers, and jobs.
Enroll an SSH-accessible machine, prepare credentials, and test tracked runs.
The first UDF type — sync, async, generator and async-generator bodies, scopes, key-based returns, and the registry payload each one produces.
The whole stack on one laptop — no cloud account, no release download, and a daemon you built yourself.
A queue is one model carrying both a scheduling surface and a launch policy, because queue depth is the autoscaling signal.
The provider types compute can be launched through, and which of them are finished.
Deep reference
This tab is orientation. The canonical, complete documentation for the SDK, the CLI, the daemon, and the control plane lives on its own site.
The full SDK reference — @udf, queues, invocations, dispatch, and payloads.
The lakeshore command — auth, queues and jobs, providers, daemons, and the
tui dashboard.
How a call becomes a queued invocation and lands on a remote worker.
The worker daemon — lifecycle, wire protocol, identity and key rotation, telemetry.
How it connects to DreamLake
| Piece | Where | Role |
|---|---|---|
| Control plane | api.lakeshore.dreamlake.ai | Queues invocations, tracks workers, brokers results |
| nymph daemon | Compute hosts, outbound-only | Executes invocations next to your data |
| Python SDK | dreamlake-lakeshore (PyPI) | @udf decorator, queues, invocation handles |
| CLI | @dreamlake/lakeshore (npm) | lakeshore tui, jobs, daemons, queue management |