Architecture

The mental model behind DreamLake: how episodes, files, and collections are organized, how auth works, and what happens to a file between dreamlake upload and playback in the dashboard.

Data model

Everything hangs off a namespace (a user or an org). Projects group episodes; an episode is one recording session and owns its files, tracks, logs, and parameters.

Namespace (user or org)
└── Project
    ├── Episode (a recording session)
    │   ├── Files (video, audio, labels, text tracks)
    │   ├── Tracks (time-series scalars/tensors)
    │   ├── Logs
    │   └── Parameters
    ├── Bindrs (curated file collections)
    ├── Datasets (groups of bindrs)
    └── Node tree (folder hierarchy, materialized paths)

Namespaces also own artifacts — versioned renderable documents with per-artifact visibility and share links, kept in their own catalog (see the CLI Reference).

Node tree

All files and folders are nodes in one unified tree, stored in MongoDB using materialized paths — lookups by path are a single indexed query, no recursion.

Path convention: root = ",", depth 1 = ",camera,", depth 2 = ",camera,front,".

Auth

A two-token system separates identity from API access:

1. Login — OAuth device flow via vuer-auth → short-lived token
2. Exchange — POST /auth/exchange → long-lived DreamLake JWT (HS256)
3. API calls — Authorization: Bearer <dreamlake-token>

First login auto-creates the User and Namespace records — there is no separate sign-up step.

Upload pipeline

CLI → chunk file (10 MB) → S3 multipart upload via BSS
    → register asset in DreamLake Server (creates node hierarchy)
    → trigger Lambda for HLS splitting (video/audio)

Uploads are resumable — progress is persisted to ~/.dreamlake/uploads/, so an interrupted transfer picks up at the last completed part. The HLS split is what powers both instant playback in the dashboard and the 2-second chunks that semantic search embeds.

Planned DreamDB maintenance

During a coordinated storage-format upgrade, the operator can temporarily pause the Artifacts, Annotations, Envs and Workflows APIs, plus managed DreamDB Source creation and its DreamDB write/credential endpoints. These requests return HTTP 503 with code DREAMDB_MIGRATION_PAUSED, Retry-After: 60 and Cache-Control: no-store. This is a maintenance failure, not empty data: retain the existing dataset and retry after service resumes; do not delete or recreate it to work around the pause. Search and non-DreamDB Source operations are not paused by this switch.

The server switch is DREAMDB_MIGRATION_PAUSED=true (off by default). It does not revoke credentials already issued to clients or stop direct storage writes. Operators must coordinate a scoped storage-side freeze, compatible client versions and rollback records separately before changing dataset refs.

Tech stack

Next steps