Connections
A Connection is a credential DreamLake holds for a namespace to reach another system: an S3-compatible bucket, a Hugging Face account, Dropbox. A Source with an external layer reads through one. DreamLake stores the credential encrypted and never shows it again — not in the web app, not in the CLI, not to the assistant.
A Connection is not the same thing as:
- a Lakeshore connection — a link to a Lakeshore service (its URL and namespace) used for compute;
- host enrollment — how a computer joins DreamLake with Nymph. Enrolling a machine does not need a Connection or an S3 bucket (Connect to DreamLake).
Who can manage them
Only the namespace's credential managers — the owner of a personal namespace, or an owner of an organization — can see or change its Connections. Anyone else sees nothing: an id they are not allowed to read looks exactly like one that does not exist.
| Who | Discover and list | Create or reconnect with a secret | Credential-free create |
|---|---|---|---|
| You, signed in (web, CLI, SDKs) | Yes | Yes | Yes |
| The DreamLake assistant, in your chat | Yes | You enter the secret in the browser; the assistant never sees it | Yes, with your approval of writes |
| Background or delegated agent runs | No | No | No |
| Service accounts and machine tokens | No | No | No |
Connections cannot be granted to an agent service account; Project and host grants do not include them.
What you can connect
CLI and SDK support depends on the installed release. Check dl connection --help
and the documentation for your SDK version before using the examples below;
a client method alone does not establish support in the connected server.
What a server accepts is declared by that server, and can differ between
deployments. Ask it: the web form reads the list, and so do
dl connection connectors, client.connections.connectors(namespace) in
Python and connections(namespace).connectors() in TypeScript. Each way in has
a mode:
| Mode | You provide | Example |
|---|---|---|
| Typed fields | A credential (some fields secret) and non-secret configuration | S3 access key; S3 role; Hugging Face token |
| Browser sign-in (OAuth) | Nothing typed — you approve at the provider | Hugging Face, Dropbox |
| None | Only configuration, no credential | Public GitHub repositories, public Hugging Face |
For S3, there are two ways in. Both ask for a profile — for example Amazon S3, Backblaze B2 or Cloudflare R2; the profiles offered are the ones this deployment has configured — and, for some profiles, a region or account id.
- Access key — an access key id and secret access key, and an optional session token. Works with every S3 profile the server offers.
- Role — Amazon S3 only. DreamLake assumes an IAM role in your AWS account
instead of holding a long-lived key. You give it the role ARN (stored as
readable configuration) and an external id (stored as a secret). Before
creating the Connection, the role's trust policy must allow DreamLake's AWS
identity to assume it, with an
sts:ExternalIdcondition equal to that external id, and the role needs read access to the buckets you will import. DreamLake does not display its AWS identity in the app; get it from whoever operates your DreamLake deployment. Backblaze B2, Cloudflare R2 and other S3-compatible profiles refuse the role way in.
Create one
In the web app, open Connections in your namespace
(/<namespace>/connections), choose + new connection, pick the service and
the way in, and fill in the form. DreamLake validates the credential first; if it
is refused, no Connection is created. Check on a Connection then reports
available when the provider accepted the credential — which proves the
credential and the Connection's endpoint, not access to any particular bucket —
or unverified when this way in has no check that works without a target.
Whether a bucket and prefix can be read is decided when you import a Source.
From the CLI, pass secrets from a file or stdin, never as an argument:
For the role way in, the credential file holds {"externalId": "..."} and the
configuration file holds {"profileId": "aws", "region": "us-east-1", "roleArn": "arn:aws:iam::123456789012:role/dreamlake-read"} with --auth-kind role.
The CLI Connections guide covers every
command, OAuth, reconnecting and deleting. The Python SDK has the same
operations on client.connections.
By asking the assistant: say which storage you want to connect. The assistant reads what this server accepts, then gives you a link to the Connections page. That page shows the namespace and what will be created, with the service and way in already chosen; you enter the secret there, signed in. The link works only for you, only in that namespace, once, for 30 minutes. When you have finished, the assistant reads back the new Connection's id and status — never the credential. You can decline the request on that page, or ask the assistant to cancel it. Replacing an expired key works the same way, from the Connection's own page.
Never paste a credential into a chat; the assistant refuses tool calls that carry one.
Use it from a Source
Import a Source that reads through the Connection, in the web app's Source import or with:
The Connection decides which endpoint the Source reads from; the target names only the bucket and prefix. A Connection that a Source still uses cannot be deleted — disable it to stop access, or detach the Source first.