DreamLake

Lakeshore

Lakeshore runs your Python functions on machines that are not your laptop. You decorate a function with @udf and call it; Lakeshore queues the invocation, ships it to a worker, runs it, and hands the result back. You write Python — it handles the scheduling, the transport, and the worker side.

You reach for Lakeshore when the work does not fit on the machine in front of you: a GPU job, a long sweep, or a fan-out of identical calls you want to run all at once. Nothing about the function has to change to move it off your laptop — the decorator is the only difference between a local call and a remote one.

Machine access, worker runtime and queue access

These are separate layers, not mutually exclusive kinds of machines:

  • Fabric — Add Host: enroll a machine to establish its identity, connection and remote-management capability. Use an authorized execution connection; no task queue or queue mount is a prerequisite.
  • Runtime — run a queue worker: start a task-consumer process on a machine, associate it with the intended queue and verify task execution. A host can be used for development and can also run workers. Host enrollment alone does not configure a queue consumer.
  • Client queue access — queue mount: save a connection for accessing queues and inspecting jobs and workers. This neither enrolls a host nor starts a worker. An online queue mount cannot authorize Add Host.

Start with connection selection and host enrollment for machine access. For task execution, see queues, worker lifecycle and the Nymph daemon reference.

The three layers

Lakeshore ships as three pieces, each deployed and upgraded on its own:

LayerWhat it isWhat it does
ClientDreamLake CLI (dreamlake) and Python SDK (dreamlake-lakeshore)Register hosts, launch work and manage DreamLake resources; write Python @udf workloads
Control planeA single service that owns the authoritative stateHolds queues, workers, providers, and pending invocations; dispatches work
RuntimeThe nymph daemon on each compute hostLong-polls the control plane and executes invocations next to your data

The daemon connects outbound only, so a worker can sit behind a firewall, in a cluster, or on a machine you already own, and still take work. What it does between starting and taking that work — coming up, running its setup, declaring itself ready, and saying so on the way out — is the worker lifecycle.

Use dl for user-facing host registration, tracked runs, providers, queues, authentication and Vault. Lakeshore names the underlying compute service and Python SDK. The older lakeshore CLI remains a low-level operator client in some service setup guides; it is not a second required user login or an alternative to the DreamLake CLI.

Start here

Quick start →

Run a Python function through a local queue, then point the SDK at a service.

Setting Up Lakeshore Service →

Host the control-plane server and create its first namespace and client login.

Connect Lakeshore to DreamLake →

Add an existing service and inspect its queues, workers, and jobs.

Use an existing host →

Enroll an SSH-accessible machine, prepare credentials, and test tracked runs.

Simple functions →

The first UDF type — sync, async, generator and async-generator bodies, scopes, key-based returns, and the registry payload each one produces.

Local dev loop →

The whole stack on one laptop — no cloud account, no release download, and a daemon you built yourself.

Queues →

A queue is one model carrying both a scheduling surface and a launch policy, because queue depth is the autoscaling signal.

Providers →

The provider types compute can be launched through, and which of them are finished.

Deep reference

This tab is orientation. The canonical, complete documentation for the SDK, the CLI, the daemon, and the control plane lives on its own site.

Python SDK →

The full SDK reference — @udf, queues, invocations, dispatch, and payloads.

CLI →

The lakeshore command — auth, queues and jobs, providers, daemons, and the tui dashboard.

Architecture →

How a call becomes a queued invocation and lands on a remote worker.

Nymph daemons →

The worker daemon — lifecycle, wire protocol, identity and key rotation, telemetry.

How it connects to DreamLake

PieceWhereRole
Control planeapi.lakeshore.dreamlake.aiQueues invocations, tracks workers, brokers results
nymph daemonCompute hosts, outbound-onlyExecutes invocations next to your data
Python SDKdreamlake-lakeshore (PyPI)@udf decorator, queues, invocation handles
CLI@dreamlake/lakeshore (npm)lakeshore tui, jobs, daemons, queue management