Sources (DreamDB)
A DreamLake-hosted store for your own videos and structured data. Push from the terminal — each video is sliced into streamable fragments automatically.
How this differs from file Sources
DreamDB sources hold structured records in collections — videos are transcoded and sliced, scalars and embeddings are typed by a schema. File Sources hold original bytes: an external, read-only layer (a connected bucket or repo) plus a managed layer you upload into (CLI sources guide). The commands on this page only work on DreamDB sources.
The syntax on this page is the shipped Python CLI. These commands also need
the DreamLake service to provide the DreamDB source endpoints (creating a
dreamdb-kind source and minting its write credentials). Installing the SDK
does not guarantee the service you log in to offers them, and it is not
confirmed here that a service with Source v1 enabled accepts dreamdb-kind
sources. File Source uploads (dl source upload) are a different
path and are not interchangeable with source push.
Install
ffmpeg and ffprobe must be on PATH for any push with video.
The standalone dreamlake binary from
cli.dreamlake.ai also has a source group — for
uploading original bytes into a Source's managed files — and it does not
run the DreamDB commands below. python -m dreamlake.cli always reaches the
Python SDK, whatever dreamlake resolves to on your PATH.
Log in
Model: source → collection → data
Think database → table → rows.
| Layer | What it is | Command |
|---|---|---|
| Source | Namespaced, permissioned store | source create |
| Collection | A dataset with a schema | source collection create |
| Data | Records at time anchors | source push |
One source holds many collections; access is granted per source.
Quick start — a folder of videos
--preset video = two fields (video + path), and is the default when you
pass neither --preset nor --schema. push <dir> uploads every video as one
record. Each is normalised to H.264 854×480 (--width/--height to change)
and sliced into ~2s fragments (fixed). push only accepts a DreamDB source.
Naming a source
Source and collection names must match ^[a-z0-9][a-z0-9._-]{0,63}$. The
namespace is a separate flag, --ns <slug> (alias --namespace). Omit it
and the source lands in your login's namespace — the common case:
Creating requires being the namespace owner — org members are read-only and
get 403. A 409 means the name already exists.
Custom schema
Write a schema JSON, pass it to collection create:
Field types
| Type | Value in a record |
|---|---|
video | file path — needs mime (e.g. "h264") |
image | file path (raw bytes) |
embedding | .npy path or inline list — needs dim |
scalar_string · scalar_categorical | a string |
scalar_int · scalar_timestamp | an integer (timestamp = ns) |
scalar_float | a number |
scalar_bool | true / false |
Push structured data — manifest
A folder can't carry scalar values, so custom schemas push with a manifest:
one JSON listing fields + every record. Its fields block is a schema —
same file works for both steps.
video/image— path relative to the manifest.embedding— a.npypath, or an inline list of exactlydimnumbers.anchor— required per record, integer ns. For unrelated videos useindex × 3_600_000_000_000(a 1-hour slot each).
Lay files out next to the manifest: clips/ thumbs/ embeds/.
Everything is validated before anything uploads — bad manifest fails fast, never a half-written collection.
Command reference
| Flag | Default |
|---|---|
--ns / --namespace | your login's namespace |
--preset / --schema | video preset |
--width × --height | 854 × 480 |
Planned command spelling (not shipped)
This is the target surface, recorded for reference only. No released
dreamlake-py accepts it — use the commands above.
The planned rename makes the groups plural (sources, collections), carries
the namespace as an optional prefix on the name — [<namespace>/]<name>, so a
source is identifiable from a single string in a command, config file or
message — in place of --ns, and adds a configurable fragment duration
(--frag-duration <seconds>) to push. The ~2s fragment length is fixed until
then.