DreamLake

Standalone catalog transfer

This implementation stage is not advertised as released until its owning API admission flag is enabled after compatible readers replace the old fleet. Standalone Artifact transfers use artifact-metadata-v1; they do not use the legacy Project RTC checkpoint executor.

Qualification on 2026-10-25

The source audit used workspace 1265f6eb and the installed CLI 0.86.1. Artifact, Env/Scene, Annotation and Pipeline have independent metadata adapters; this is not a statement that every resource instance or admission flag is ready. The current regression candidate passes 32 real-Mongo tests, including overlapping catalog edits versus acceptance, transaction interruption, exact request replay, receipt-bound old links, share-token revocation and indexed historical lookup. The three new catalog race tests each exercise four independently created resources. A write either commits before transfer and survives it, or fails with LOST_AUTHORITY; a new write under the current destination then succeeds.

ProfileQualified source scopeStill unsupported
ArtifactPrivate personal source; source owner also destination member; stable ID/storage/share token/visit grantsOrganization source, public visibility, ordinary Project filings
Env/ScenePrivate personal source; source owner also destination member; stable ID/storage/share token and both URL familiesOrganization source, public visibility, ordinary Project filings
AnnotationPrivate personal source; source owner also destination member; stable ID/storage/schema and authenticated old readsOrganization source, public visibility, ordinary Project filings
PipelinePersonal source; source owner remains destination member; code/history/runtime rows retain identityOrganization source and ordinary Project filings
AssetLibraryReader binding only; the generation protocol remains a separate unmerged candidateOwnership admission and efficient immutable delta publication

External grant mounts require their explicit reviewed disposition. Rejecting an unsupported resource leaves ownership unchanged; it must never silently remove filings, widen visibility or drop an audience. Organization-source support must retain live membership and revocation through the shared receipt-bound audience grant model and audited per-type readers and writers before its admission expands.

The production canary for each enabled profile must use a disposable resource: create it privately in the personal source, record its stable catalog ID and storage binding, save a transfer review, overlap an edit with recipient acceptance, then read the old and new links and inspect exact completed replay. Artifact and Scene additionally exercise the same share token before and after transfer and after revocation. Remove the test resource through its current canonical owner and verify it is absent; do not move a real user resource as a qualification test. Annotation currently lacks first-class CLI creation/edit/deletion commands; use the audited HTTP resource API for those steps, rather than inventing commands. All four transfers use dl ownership transfer TYPE ID with --dry-run --out, followed by --confirm --request-file --accept-if-authorized --wait.

Ready Artifact profile

A live private Artifact can move from a personal namespace when its owner is also an active member of the destination namespace. The Artifact must have no Project filings, pending storage mutation, hidden upload reservation, destination name collision, or invalid ownership history. Unsupported cases fail explicitly; there is no destructive detach, clone/delete, or legacy fallback.

Mounts admitted by an external mount grant are not Project filings for this check. Artifact, Scene and Annotation transfers list each such grant in the preview and require an explicit reissue or revoke disposition in policy.externalMounts; without one they fail with EXTERNAL_MOUNT_DISPOSITION_REQUIRED.

The accepted request retains the catalog ID, canonical Artifact name, all content versions, share token, share-visit access, and original object storage bucket/prefix. No object payload or Note/RTC history is read or copied. A new namespace-qualified location points at the same pinned storage location. Old namespace links resolve only through the completed accepted request and its exact ID/history/epoch/storage receipt; the alias grants no access by itself. A replacement Artifact created under the old name takes precedence.

Concurrency and recovery

Acceptance revalidates current authority, the access preview, destination name availability, catalog storage admission, and source-owner destination membership. Membership mutation shares the organization authority write parent. Catalog storage mutation and transfer share the catalog row CAS; an uncertain external storage effect remains blocked. A new share visit participates in that same row fence, preventing it from recreating access under the obsolete owner.

The owner update, pinned storage binding, share-visit relocation, lease epoch, completion event and durable receipt commit in one MongoDB transaction. There is no durable preparing interval or RTC handoff. Interruption before commit rolls everything back; a lost response is handled by replaying the original idempotency key or reading its completed operation. Recipient acceptance is required; cancellation/rejection remain available while pending.

Remaining stages

AssetLibrary, ordinary retained Project filings, public visibility and organization-source catalog audiences require their own proof-aware readers and access guarantees. They are not enabled by adding names to a registry. The shared organization audience grant model is owned by the standalone Memo stage and will be reused rather than creating a competing store.

Prepared Scene/Env stage

The next independent default-off executor is env-metadata-v1. Its initial profile matches the Artifact constraints: private personal source, source owner still a destination member, no external filings or pending storage mutations. It retains the same EnvCatalog ID/name, share token, every content version and the exact original S3 storage binding. Both /scenes and legacy /envs GET families resolve old links through a completed receipt and then check current namespace authority. A replacement source name takes precedence; a revoked share token remains revoked. Writes always use the current canonical namespace.

This reader floor requires the additive nonunique EnvCatalog index {name:1, 'ownershipHistory.sourceNamespaceId':1} before rollout, so a missing canonical name does not introduce an unindexed catalog scan. The schema adds nullable ownership history; the scoped workflow provisions this index; no blob migration, secondary content copy or RTC work is involved. Provision through the existing reviewed infrastructure/deployment process, then prove the compatible fleet drained before enabling new Scene transfers.

Prepared Annotation stage

annotation-metadata-v1 is a separate default-off executor with the same private personal-source, destination-member and unfiled admission profile. It preserves the AnnotationCatalog ID/name, schema metadata and original bucket and storage prefix without reading or copying annotation payloads. Detail and presign-read requests to old namespace URLs require completed receipt proof, then current destination authority. Presign admission performs that lookup and catalog writer fence inside its existing transaction; an alias cannot authorize an obsolete namespace or bypass a pending storage mutation.

This stage adds nullable AnnotationEntry ownership history and the nonunique AnnotationCatalog {name:1, 'ownershipHistory.sourceNamespaceId':1} index. Provision that index before reader rollout, verify with verifyAnnotationTransferReadIndex(db), drain incompatible readers, then enable the separate admission gate. Resource bodies and issued presigned URLs keep their existing storage/TTL behavior; existing URL lifetimes are not silently shortened or described as instant revocation.

The shared organization-audience stage and proof-aware retained Project filings remain separate work. AssetLibrary additionally needs a stable storage binding and durable external storage admission before an ownership executor can safely be enabled; its namespace-derived object paths are not portable today.

Catalog API integration and rollout

The Env and Annotation singleton protocols expose the same ownership endpoints as Artifact: GET /namespaces/:namespace/ownership/resources/<type>/:id/transfer-plan, then the shared preview/create and recipient decision endpoints. <type> is env or annotation. Their plans return the stable resource ID, resourceName, review version and transferMode: metadata. Scene detail/list responses expose that same catalog id; Annotation detail/list already expose it.

Capabilities advertise separate envTransfer and annotationTransfer objects, each {apiVersion: 1, enabled, metadataOnly: true}. New requests and pending acceptance require their own default-false deployment variables: OWNERSHIP_ENV_METADATA_TRANSFER_ENABLED and OWNERSHIP_ANNOTATION_METADATA_TRANSFER_ENABLED. Disabling admission preserves exact same-key request replay, operation history, old-link readers, rejection and cancellation. Startup verifies the required index before allowing each type.

Before deploying the reader floor, run the scoped catalog-metadata-read-indexes.yml workflow from merged main, first with mode=plan. Apply only its fresh source/database-bound plan_hash with confirmation=CREATE_TWO_CATALOG_METADATA_READ_INDEXES_ONLY and mode=apply. It creates only the missing nonunique normal {name:1, 'ownershipHistory.sourceNamespaceId':1} indexes on EnvCatalog and AnnotationCatalog, accepts equivalent indexes, refuses conflicting index options, and never drops indexes or changes resource records. The existing ECS deployment identity and noncancelling deployment concurrency group are reused; there is no new IAM or infrastructure launch. Apply rechecks the production service's database fingerprint and ten-minute plan window.

Keep both admission variables false while the compatible-reader API deploys. After index verification, replacement fleet/runtime proof, old-writer drain and concurrent-work canaries, enable only the independently accepted resource type. The presence of an endpoint or green CI alone is not live transfer acceptance.

AssetLibrary reader compatibility floor

The prepared reader-only stage adds an optional AssetLibraryCatalog storagePrefix. Missing values keep the existing namespace/name object path; there is no object copy, migration, upload protocol change, transfer admission or background job in this stage. A later generation writer can pin a unique libraries/catalog/<catalog-id>/generations/<session-id> binding.

Manifest, file, thumbnail, vector and merged cross-library search readers all honor the pinned bucket/prefix. Manifest/vector cache identities include the catalog ID and physical binding, so a reused display name or revision counter cannot return another catalog's cached content. Existing private/share checks remain in force; a storage locator is never an access grant.

Ship this compatibility floor before the separately gated upload-session and retired-generation cleanup protocol. Library ownership transfers remain unsupported until the storage writer floor, compatible CLI, crash/reclaim tests and access-preserving transfer proof are independently accepted.