DreamLake

Architecture

Data Model

Namespace (user or org)
└── Project
    ├── Episode (a recording session)
    │   ├── Files (video, audio, labels, text tracks)
    │   ├── Tracks (time-series scalars/tensors)
    │   ├── Logs
    │   └── Parameters
    ├── Bindrs (curated file collections)
    ├── Datasets (groups of bindrs)
    └── Node tree (folder hierarchy, materialized paths)

Node Tree

Uses MongoDB materialized paths. All files and folders are nodes in a unified tree.

Node KindDescription
projectTop-level project node
episodeRecording session
folderOrganizational directory
file, video, audio, image, text, codeLeaf nodes

Path convention: root = ",", depth 1 = ",camera,", depth 2 = ",camera,front,".

Auth

Two-token system:

  1. Login — OAuth device flow via vuer-auth → short-lived token
  2. ExchangePOST /auth/exchange → long-lived dreamlake JWT (HS256)
  3. API callsAuthorization: Bearer <dreamlake-token>

First login auto-creates User + Namespace records.

Upload Flow

CLI → chunk file (10 MB) → S3 multipart upload via BSS
    → register asset in DreamLake Server (creates node hierarchy)
    → trigger Lambda for HLS splitting (video/audio)

Uploads are resumable. State persisted to ~/.dreamlake/uploads/.

Tech Stack

ComponentTechnology
ServerFastify, MongoDB (Prisma), JWT
StorageS3 via BSS
SearchQdrant (CLIP embeddings)
CLIPython, click