DreamLake

Architecture

The mental model behind DreamLake: how episodes, files, and collections are organized, how auth works, and what happens to a file between dl upload and playback in the dashboard.

Data model

Everything hangs off a namespace (a user or an org). Projects group episodes; an episode is one recording session and owns its files, tracks, logs, and parameters.

Namespace (user or org)
└── Project
    ├── Episode (a recording session)
    │   ├── Files (video, audio, labels, text tracks)
    │   ├── Tracks (time-series scalars/tensors)
    │   ├── Logs
    │   └── Parameters
    ├── Bindrs (curated file collections)
    ├── Datasets (groups of bindrs)
    └── Node tree (folder hierarchy, materialized paths)

Namespaces also own artifacts — versioned renderable documents with per-artifact visibility and share links, kept in their own catalog (see the CLI Reference).

Node tree

All files and folders are nodes in one unified tree, stored in MongoDB using materialized paths — lookups by path are a single indexed query, no recursion.

Node kindDescription
projectTop-level project node
episodeRecording session
folderOrganizational directory
file, video, audio, image, text, codeLeaf nodes

Path convention: root = ",", depth 1 = ",camera,", depth 2 = ",camera,front,".

Auth

A two-token system separates identity from API access:

  1. Login — OAuth device flow via vuer-auth → short-lived token
  2. Exchange — POST /auth/exchange → long-lived DreamLake JWT (HS256)
  3. API calls — Authorization: Bearer <dreamlake-token>

First login auto-creates the User and Namespace records — there is no separate sign-up step.

Upload pipeline

CLI / SDK → ask DreamLake Server to authorize the upload
          → S3 directly, against the signatures it minted (16 MiB parts)
          → register the node, which records where the bytes went

The server mints the object key and the signed requests; the client only executes them and never builds a key itself. Uploads are resumable — the upload id is persisted to ~/.dreamlake/uploads/, and on re-run the server is asked which parts already landed, so an interrupted transfer sends only the rest.

Planned DreamDB maintenance

During a coordinated storage-format upgrade, the operator can temporarily pause the Artifacts, Annotations, Scenes and Workflows APIs, plus managed DreamDB Source creation and its DreamDB write/credential endpoints. These requests return HTTP 503 with code DREAMDB_MIGRATION_PAUSED, Retry-After: 60 and Cache-Control: no-store. This is a maintenance failure, not empty data: retain the existing dataset and retry after service resumes; do not delete or recreate it to work around the pause. Search and non-DreamDB Source operations are not paused by this switch.

The server switch is DREAMDB_MIGRATION_PAUSED=true (off by default). It does not revoke credentials already issued to clients or stop direct storage writes. Operators must coordinate a scoped storage-side freeze, compatible client versions and rollback records separately before changing dataset refs.

Tech stack

ComponentTechnology
ServerFastify, MongoDB (Prisma), JWT
StorageS3 (server-signed, direct)
SearchQdrant (CLIP embeddings)
CLIPython

Next steps

API Reference →

The REST endpoints these pieces expose.