The Registry

The registry is where a piece of code stops being a file on your laptop and becomes a name that other people can find, pin and import. DreamLake owns the declaration half of that: the name, the source, the signature, and the environment the code expects. It owns none of the execution half.

This page is the write direction — declaring something and giving it a version history. The read direction is Importing from the Registry.

Two layers, and the split is the whole design

A registered runnable is two rows, not one.

Runnable is the mutable head. It survives every edit to the function body, and it is what a name resolves to:

interface Runnable {
  id:            string
  namespaceId:   string
  name:          string    // "<module>.<qualname>", unique within the namespace
  module:        string
  qualname:      string
  kind:          string    // "udf" | "session" | "agent"; defaults to "udf"
  description?:  string    // one-line documentation

  markdown?:     string    // agent-only — the agent's INSTRUCTIONS, not its docs
  channel?:      object    // agent-only — { kind, target?, config? }

  latestVersion?: string   // denormalised pointer to the newest version
  metadata?:     object
  createdAt:     Date
  updatedAt:     Date
}

RunnableVersion is immutable. One row per distinct source revision, append-only — note the missing updatedAt and the missing deletedAt:

interface RunnableVersion {
  id:          string
  runnableId:  string
  namespaceId: string
  version:     string    // sha256(source), LOWERCASE HEX — the content key
  source?:     string    // full source captured at decoration time
  signature:   object    // { params, returns, doc } — shape owned by the SDK
  metadata?:   object
  createdAt:   Date
}

The head is keyed (namespaceId, name, deletedAt). The version is keyed (runnableId, version). Everything below follows from those two keys.

kind is a lifetime, not a substance

udf, session and agent answer one question — how long the process lives:

That is the only axis kind encodes. It does not say whether the body is sync, async, a generator or an async generator — see what the registry cannot tell you.

What makes this version control and not version display

Four properties, and all four are load-bearing.

1 · Registration is idempotent by content and never overwrites. Posting a version that already exists returns 200 with the stored row, unchanged. A new one returns 201. That is safe rather than merely convenient, because version is sha256(source) — a row already under that key already holds those exact bytes, so a second caller cannot silently replace what the first one stored. There is no PUT anywhere in this collection and no way to rewrite a version row.

2 · latestVersion is a cache, and only one endpoint may move it. It is deliberately not settable through PATCH; the version-registration endpoint is the only writer. Re-registering an older version hits the 200 path and returns early, so the pointer never moves backwards. RunnableVersion stays the source of truth — if that denormalised write were ever lost, the newest row is still findable by createdAt desc.

3 · Deleting the head is soft, and leaves every version intact. DELETE on a Runnable stamps deletedAt and stops there. The version rows are untouched; they are hidden because their parent is tombstoned, not because they were removed.

4 · Nothing deletes a version. Not soft, not hard. There is no endpoint that deletes a RunnableVersion or a RepoSnapshot anywhere in the collection. DELETE exists only on the three mutable heads (Runnable, RunConfig, Repo). Both children are append-only immutable history keyed by content, and hard-deleting one would orphan every invocation on the control plane that names it.

The routes

Every route is namespace-scoped. :id resolves as a 24-hex ObjectId or as the dotted name — the SDK knows the id, a human knows examples.embed.embed_frames, and both work.

List responses are enveloped under a collection key — runnables, versions, runConfigs, repos, snapshots — alongside the pagination fields. Errors are always { error: string } and nothing else.

Declaring, end to end

The dreamlake CLI wraps each of the routes above one for one.

# 1 — create the head. Create-only; a second run is a 409.
dreamlake declare runnable.json
#   POST /namespaces/<ns>/runnables → 201

# 2 — push the first version. The hash is computed from the file bytes.
dreamlake runnable version push examples.embed.embed_frames \
  --source src/v1_embed.py
#   sha256 = 9f2c…  →  POST …/runnables/<id>/versions → 201

# 3 — edit the source, push again. latestVersion moves.
dreamlake runnable version push examples.embed.embed_frames \
  --source src/v2_embed.py
#   sha256 = 41ab…  →  201

# 4 — push v1 again, unchanged. This is the point of the whole design.
dreamlake runnable version push examples.embed.embed_frames \
  --source src/v1_embed.py
#   sha256 = 9f2c…  →  200 UNCHANGED, and latestVersion stays at 41ab…

# 5 — the history, newest first, without dragging source over the wire.
dreamlake runnable version list examples.embed.embed_frames

# 6 — one version, with its source.
dreamlake runnable version show examples.embed.embed_frames 9f2c…

Step 4 is the sentence worth remembering: the version string is the hash of the source, so a second caller pushing the same bytes cannot silently replace what the first one stored, and pushing an old revision does not rewind the head.

RunConfig — the environment, declared separately

A RunConfig is the named, reusable declaration of the environment a runnable runs inside. It is a sibling of Runnable, not a field on it.

interface RunConfig {
  id:           string
  namespaceId:  string
  name:         string      // label, unique within the namespace
  description?: string

  runtime:      object      // read by the RUNTIME layer
  extras:       object      // forwarded VERBATIM to the launcher
  tags:         string[]
  provider?:    string      // a LakeshoreProvider NAME, not a foreign key
  hostKey?:     string      // derived "hk1_…"; never accepted from a client

  metadata?:    object
  createdAt:    Date
  updatedAt:    Date
}

runtime and extras are two audiences, and they do not feed each other

This is the thing readers get wrong.

• runtime is read by the runtime layer: { backend?, server?, runner?, image?, resources?, env?, timeout_s?, host_setup?, run_setup? }.
• extras is handed to the launcher untouched: { instance_type?, partition?, time_limit?, n_cpu?, n_gpu?, … }.

Setting runtime.resources.gpu does not fill in extras.n_gpu, and setting extras.instance_type does not tell the runtime anything. From the schema comment, which is the clearest statement of why there are two columns rather than one bag:

Keeping them in one bag is exactly what let _RUNTIME_FIELDS rot into a hand-maintained filter with zero readers; two columns make the boundary a schema fact.

tags is a typed column rather than a key in either bag because it is the one field forwarded to both halves — an EC2 instance tag and a job tag are the same string, so it cannot honestly live on one side.

provider is a LakeshoreProvider.name, not a foreign key. The control plane resolves providers by name, late, and an FK here would break the moment it does.

hostKey is derived, and its exclusions are claims

hostKey is recomputed on every write and never accepted from a client — the create body schema simply has no such property, and PATCH recomputes it from the merged value whenever runtime, extras or provider is touched.

subset    = { v: 1, runner, image, resources, host_setup, provider, extras }
canonical = JSON, UTF-8, no whitespace, keys sorted by code point, recursively
digest    = sha256(canonical)
hostKey   = "hk1_" + base32-lower-nopad(digest[0..10])     // 20 chars total

Two configs with the same hostKey can share a worker. What is left out is where the meaning is:

And host_setup is in the key, with its order preserved — the array is never sorted. Running pip install torch before apt-get install ffmpeg is a different host preparation, and pretending otherwise would let a worker prepared one way serve a job that declared the other. Adding anything to the subset bumps the prefix to hk2_.

# Which configs can share a worker?
dreamlake runconfig list --host-key hk1_4d7qz2xk6m3aynbe

# The derived key on one config — you never sent this value.
dreamlake runconfig show gpu-a10g

Agents: markdown and channel

An agent is a Runnable with kind: "agent" and two extra columns:

• markdown — the agent's instructions, authored as Markdown. This is behaviour. description is the one-line documentation, and the two are deliberately distinct fields.
• channel — where the agent's pipe is attached. A stored declaration only; no live connection, socket or session is modelled in DreamLake.

Both are rejected with a 400 on any kind other than agent, so a caller who typed prose into a UDF finds out rather than watching it disappear.

channel is shape-checked and credential-scanned

channel must be exactly { kind, target?, config? }. Any other top-level key is a 400. Then every key inside it is walked — to a depth of 8, through arrays — and rejected if it looks like a credential:

/token|secret|password|passwd|api_?key|private_?key|authorization|credential|bearer/i

Order matters, and it surprises people: the shape check runs first, so { token: "x" } comes back with the shape error, not the credential error — token is not one of the three allowed top-level keys. { config: { auth: { token: "x" } } } is the case that gets the credential error.

One database caveat that will bite you

Try it

The starter kit's 04_version_history directory walks the six steps above against a local control plane — declare, push, edit, push, push the old bytes back, and watch the 200 — and finishes on the versions panel in the dashboard with a ?v=<sha256> deep link.