DreamLake

Public GitHub repositories

Public GitHub preview. This provider adds public github.com repositories to the existing Connection and Source v1 APIs. It requires the existing Connection and Source v1 runtime surfaces; there are no new credentials, environment variables, schemas, infrastructure resources, or HTTP routes. Release status is tracked in the implementation PR.

Connect and import

Create a Connection in your namespace using POST /namespaces/:ns/connections:

json
{"label":"Public GitHub","connectorKind":"github","authKind":"public"}

This records a public access connection without a token. Connection creation cannot verify an unnamed repository; the subsequent import performs that check. The existing namespace management permissions apply. No OAuth or PAT is accepted.

Import a Source using POST /namespaces/:ns/sources:import-external, replacing the Connection ID with the one returned by the create operation:

json
{
  "name": "repository-reference",
  "external": {
    "connectorKind": "github",
    "connectionId": "<connection-id>",
    "target": {"repoId": "octocat/Hello-World", "revision": "master"}
  }
}

repoId is an owner/repository name, not an arbitrary URL. revision accepts a branch, tag or commit; omit it to use the repository's default branch. Import resolves the revision to a commit and tree. The resulting Source is an immutable snapshot: later branch movement does not alter its content. Import again to select a newer revision. Import does not enumerate or download files.

Use the existing Source browse/stat/read APIs or CLI commands with the returned Source. Reads still require DreamLake Source access, even though the upstream repository is public. A canonical repository link identifies the pinned commit.

bash
dl source browse --source <name>
dl source stat README --source <name>
dl source download README --source <name> -o ./README

The read capability is an anonymous, commit-pinned raw.githubusercontent.com URL, with no authorization headers. Its five-minute refresh timestamp tells the client to renew DreamLake authorization; the underlying public URL is not an expiring secret. Disabling the Connection stops future capability issuance, through the existing broker. It cannot revoke public GitHub access or bytes already downloaded. This connector does not persist a content cache.

Bounds and failure behavior

  • Each operation checks current public visibility and the repository's numeric identity. Private repositories, renamed targets that redirect, and a different repository recreated under the old name fail closed.
  • Browse returns canonically sorted pages from one bounded directory. At most 2,000 entries and 1 MiB of provider response data are accepted; truncated or larger trees are rejected explicitly instead of presenting partial listings.
  • Paths are canonical and limited to 16 segments. Provider requests use fixed HTTPS origins, GET only, refused redirects, and a ten-second request deadline inside the existing Source operation deadline.
  • Ordinary files support public reads and byte ranges. Symlinks and submodules remain visible with reading disabled. Git LFS pointers are ordinary Git files; this connector does not resolve their external payloads.
  • Anonymous GitHub API calls share the server's upstream IP quota: GitHub ordinarily allows 60 requests per hour. Import needs three metadata calls; browse/read require a visibility check plus tree traversal. There is no token fallback, automatic retry loop, background sync, or bulk indexing. Rate limits return the existing retryable SOURCE_UPSTREAM_RATE_LIMITED error; upstream diagnostic bodies are not exposed.

Private repositories, GitHub Apps/OAuth/PAT credentials, provider-wide search, incremental sync and writes are future slices. Agent harness and MCP callers must reuse the authorized Source service; this provider change does not itself register new agent or MCP tools.

Validation and deployment

The provider smoke uses the tiny octocat/Hello-World repository at commit 7fd1a60b01f91b314f59955a4e4d4e80d8edf11d. It checks import metadata, browse, stat, and the 13-byte README through the real unauthenticated provider. This is provider integration evidence, not acceptance of a deployed DreamLake route. Deployment remains coordinated through the existing centralized Terraform stack; this change does not provision or independently deploy a server.

Provider references: Git trees, commits, and REST rate limits.