Prefix KMS
2026-09-15 — retained-entry policy preview (Unreleased)
affectedEntries.observedAt is the application's UTC wall-clock observation when
that page read completes, not a Mongo cluster timestamp or a guarantee spanning
multiple pages. contextFingerprint identifies the policy/owner/tenant/key/schema
context also bound into continuation cursors. Equal fingerprints do not freeze
entry revisions between observations. All opt-in metadata probes have a 10-second
Mongo operation limit; ordinary preview timeout behavior is preserved.
The existing owner-only preview endpoint accepts optional affectedEntries: {limit, cursor?}. Limits are 1–200. Ordinary requests retain their response shape.
The summary, governing policy and metadata page use one Mongo snapshot transaction;
each subsequent page is a new snapshot. This is an observation, not a reservation
or migration authorization. Cursors bind the owner, tenant, prefix, selected key,
retained-record schema and policy/migration epochs. A changed context refuses the
old cursor; re-inspect before starting a new traversal.
Rows include retained soft-deleted entries, IDs, names, revisions, types and
lifecycle dates. currentPolicy describes the governing policy, not ciphertext
provenance. proposedPolicy is null when an overlapping boundary or active
migration blocks comparison. Receipt/HOTP/password-snapshot counts remain separate
from entry rows. Mongo projects only metadata and reads at most limit + 1 entry
rows; no value read, KMS call, DDL or mutation is performed.
Local replica-set tests inspect real Mongo commands for projections, tenant predicates, limits, index use and one snapshot session. A concurrent retirement remains outside the established snapshot and appears on the next observation. The paired source CLI/Python fixture exercises actual HTTP routes and three pages, including retired entries. Neither this feature nor these tests establish a cross-boundary move contract, hosted acceptance or release.
2026-09-15 — AWS request deadline failure and candidate fix
An owned real-AWS network-stall test failed: the read returned sanitized 503 only after the independent 65,219ms safety guard. Three encrypted response records were stalled across three SDK attempts. The post-failure entry/checkpoint assertions and migration recovery phases were not reached. The three temporary IAM/grant resources were destroyed, and the original key policy/grants were verified unchanged. Failed live receipt.
The installed Smithy 4.9.5 handler only warns for requestTimeout unless throwOnRequestTimeout is enabled. The candidate production factory now enables enforcement while preserving three attempts and a 10-second deadline per attempt. A real verified-TLS SDK regression measured 30,125ms for three stalled attempts, sanitized 503, and recovery in the same client. Retry backoff adds to total latency; this is not a hard 30-second overall deadline. A second owned cloud run then passed: read 30,073ms and migration resume 30,204ms, each exactly three attempts and nine newly stalled encrypted response records. Entry/checkpoint snapshots stayed unchanged, same-process recovery and published CLI 0.20.2/Python 0.16.1 reads passed, and all three temporary permissions were removed with original policy/grants unchanged. Successful receipt. This used the same key with a new policy epoch, local actual API/Mongo and real AWS; it does not prove a distinct-key migration, hosted outage, or deployment.
Run the isolated TLS regression from dreamlake-server (Node, dependencies and OpenSSL required; no AWS credentials/resources):
2026-09-15 — flag snapshot differs from the historical enabled test
A read-only ECS snapshot at 11:04:56 UTC found staging task171 solely completed with one running task and DREAMLAKE_VAULT_KMS_MIGRATION_ENABLED=false. It retained the AWS primary, one AWS prefix reference and additional allowed keys; no Google prefix route was configured. The earlier task154 disabled gate, task155 enabled gate and successful hosted AWS forward/outage/reverse receipts remain valid historical evidence. They do not authorize changing today's flag.
The snapshot was recorded while rollout coordination was tracked in #555. Before any newly approved enablement, repeat the existing writer-inventory/readiness procedure against the actual settled fleet. This adds no new gate and does not erase completed acceptance. No cloud resources or flags were changed by this documentation audit; production promotion and remaining customer/hosted-provider cases remain separate.
2026-09-15 — mixed AWS/Google live acceptance and cleanup
Seven live phases passed in an owned local service fixture; not hosted
acceptance. Examples #40
merged at e89bf7ae8af42cee74655a9c3ef80370ab20967f from reviewed source 5c6b304.
The complete setup and execution guide
and sanitized live receipt
record actual Vault HTTP, isolated Mongo, real AWS/Google KMS and published CLI
0.20.1/Python 0.16.1. The run pinned backend 35416f55 and runner 117d64c;
backend routing #514
merged separately as 70b594db.
Baseline, AWS-to-Google migration, AWS denial/recovery, Google denial/recovery and reverse migration all passed. Both migration directions preserved two entries, two write receipts, one HOTP receipt, logical revisions/counters and exact retained replay. Removing each run-owned permission produced an actual provider permission-denied response and HTTP 503 for the affected entry while the other provider returned 200; restoring that permission recovered access. Authentication used explicit 900-second AssumeRole and service-account impersonation credentials in private SDK transports, not default ADC runtime.
Cleanup destroyed all five reviewed permission resources and verified empty Terraform state, no fixture process/children, absent temporary AWS role and no run-owned Google key/service-account bindings. The original AWS grant and canonical key policy were unchanged; both existing KMS keys remain enabled. The fixture used synthetic accounts and values; hosted backend flags were not changed.
The initial startup attempt exited before readiness with unknown cause and is not acceptance evidence. The successful run required stdin EOF to finish; the later shutdown/helper fix has regression coverage, not a repeated live run. This did not test a second live AWS region, migration resume during a provider outage, hosted deployment or customer IAM onboarding. Those limits remain open; the seven completed phases do not close all KMS delivery work. Tracking: Vault #241 and master #247.
2026-09-15 — provider and region routing candidate
Implemented candidate; not deployed. The server selects an adapter from canonical, operator-allowlisted immutable key IDs. AWS ARNs select the SDK region; Google resource names retain their location. DREAMLAKE_VAULT_KMS_PROVIDER, DREAMLAKE_VAULT_KMS_KEY_ID and the AWS default region still define managed encryption. DREAMLAKE_VAULT_KMS_ALLOWED_KEY_IDS also supplies explicitly retained/customer routes. Unknown keys, provider mismatches and customer outages fail closed; there is no managed-key retry.
Tenant-bound DREAMLAKE_VAULT_KMS_PREFIX_KEYS entries may include provider: "gcp-kms" or "aws-kms". Omission keeps the existing managed-provider interpretation; a mismatched canonical key is rejected at startup. Public CLI/Python APIs still select an opaque keyRef, never submit arbitrary keys or endpoints. Listing reports the selected ref's provider. This addresses the routing portion of vault/kms/customer-bootstrap; customer IAM grants, identity onboarding and live multi-provider service acceptance remain open.
Example operator configuration uses identifiers only; cloud authentication stays in workload identity outside Vault:
These are illustrative identifiers; use reviewed real immutable IDs and tenant ownership. Adding a reference does not grant IAM access or activate a prefix. Existing CLI/Python prefix preview/activation/migration operations keep their paired contracts.
No schema or ciphertext backfill is required: envelopes, boundaries and migration records already store provider and immutable key identity. Reads use the recorded envelope provider/key (including GCP version); writes use the authoritative boundary. Populated transitions use the existing resumable migration and retained-record inventory, preserving logical revisions, receipt replay, binding references and policy epochs. Keep all old keys/routes and permissions while any entry, receipt, password snapshot or backup requires them. Configure router-capable readers/writers across the fleet with migration disabled, verify defaults and historical reads, then explicitly enable bounded migration acceptance. Do not roll back to a reader that cannot open newly written provider envelopes.
Validation: TypeScript and all 191 Vault tests across 29 files pass. Provider routing tests use synthetic KMS transports; integration tests use actual owned Mongo and HTTP, including AWS → GCP → AWS retained receipt replay, tenant context rejection, customer outage refusal and a paused pre-cutover write. Live mixed-provider router acceptance remains required.
The isolated Google federation prerequisite is independently complete: examples39 records real positive STS controls and wrong-audience, wrong-identity and missing-impersonation-permission denials. Sanitized receipt includes UID cleanup, disabled/soft-deleted owned trust and unchanged existing IAM/KMS. This does not prove deployment of this routing candidate.
2026-09-14 — refresh, denial and owned cleanup receipts
The live follow-up examples37 merged at 33690f3 from reviewed head 96c7909, a documentation follow-up to the receipts at 8196b60. Seventeen fixture tests, ten bundle tests and syntax checks passed. Its refresh receipt proves the same process crossed both access-token and projected-token expiry, observed rotated tokens, and completed another real GCP KMS roundtrip. Permission-revocation evidence records successful ADC exchange followed by KMS denial; fresh-process federation evidence records denial after federation was removed. These are separate checks, not wrong-identity or wrong-audience coverage.
The cleanup receipt confirms owned Kubernetes cleanup and absence of the fixture's federation and KMS IAM memberships. The workload-identity pool is disabled and reports DELETED, a soft-deletion state. The service account, encryption key and STS API are retained; physical key erasure is explicitly false. Earlier pending fields in the client-install receipt describe that earlier stage; these later lifecycle receipts record the new results.
Published CLI0.18/Python0.15 core/two-owner, pagination and HOTP checks remain the isolated HTTP/native Mongo acceptance recorded below. Wrong-identity/audience cases, hosted production acceptance and the retained-resource/backup lifecycle remain open. No product deployment or complete Vault signoff follows from this test evidence.
2026-09-14 — default GCP ADC fixture and live acceptance
The owned federation and paired-client fixture examples36 merged at 8f0f39df. Live follow-up examples37, currently fbbea8b, proves an actual default-ADC exchange before the required KMS-denial check. Its initial run without the project setting failed ADC discovery with 403; startup now derives the project from the reviewed key path. No additional IAM permission was needed for that correction. Seventeen fixture tests and JavaScript syntax checks passed.
Published CLI0.18 and Python0.15 passed three groups—core/two-owner isolation, pagination and HOTP—against the isolated loopback HTTP/native Mongo fixture with real GCP KMS and backend 90ef770. The recorded evidence marks all three groups passed; it does not establish hosted production acceptance.
Token refresh, revocation and final owned-fixture cleanup remain open acceptance gates; the PR is not merged. Follow the KMS examples and the candidate's runnable procedure. These results do not establish production GCP activation or authorize deleting historical keys. Shared staging promotion remains held by #387.
2026-09-13 — staged AWS acceptance; production pending
Tracking: Vault #241, master #247, policy plan, migration plan.
The empty-prefix foundation merged in backend353, CLI57 and Python43. Resumable migration merged in backend359 (5c82000e), CLI59 (c37a3932) and Python45 (fbd814e1). Native CLI 0.17.0 and Python 0.14.0 subsequently shipped. The initial staging rollout 34772400456 used migration disabled. The later enabled task155/source 433c4cdd passed the runtime gate and owned AWS forward/outage/reverse acceptance; see the dated evidence and remaining gates.
Migration changes only encrypted envelopes, covering retained entries and write/HOTP recovery receipts, including receipts whose original entry was purged. Epoch markers and Mongo transactions preserve logical revisions, counters, bindings and immutable recovery requests. Explicit resume prefixes are checked before mutation. A tenant guard trades throughput for race safety: high concurrency can return a redacted 503; reconcile and retry the identical write intent.
Final validation: 166 local Vault tests and typecheck passed; exact backend full CI passed. CLI passed 937 tests and exact CI; Python passed 539 with 60 skips. Paired real HTTP/Mongo tests passed lost committed response recovery across fresh CLI processes, bounded cross-client resume, managed-key reversal and tenant denial. Additional Mongo tests cover contention, paused write/HOTP races, purge overlap, preserved binding revisions and query-plan isolation. Provider transports in those local migration tests are synthetic; the later separately recorded staging acceptance used real AWS KMS. Docs builds/index checks passed; Sphinx retained 38 existing warnings.
Migration defaults off behind DREAMLAKE_VAULT_KMS_MIGRATION_ENABLED. The ECS workflow forwards the gate, trusted prefix registry and allowed IDs, and staging task155 was enabled after the recorded historical fleet gate. Verify every writing instance runs epoch-aware code before enabling migration, and never roll back to a pre-epoch writer afterward. Broader customer onboarding, second-tenant/wrong-context live denial, cross-provider routing, GCP acceptance and production promotion remain open. Active-database completion is not backup re-encryption or permission to delete historical keys.
2026-09-15 — tree clients and rollout follow-up
CLI 0.22.0 publication is incomplete; Python 0.17.0 is published on PyPI with exact frozen archive verification. The reviewed artifacts passed actual local
HTTP/Mongo/PTY and staged live pagination checks. See the
paired tree guide
and its frozen artifact receipts. API586
is merged; production task 122/source 1058ef4 passed metadata-tree first-page
and continuation checks after the retired-process stop gate. Both synthetic
entries were retired/read404 with the earlier uncertain cleanup receipt retained;
see the runtime evidence. The initial backend
preparation below is retained as history, not current deployment status.
2026-09-15 — metadata-only prefix tree candidate (initial preparation)
Backend merged in PR576; not released or deployed. GET /v1/vault/tree
requires an explicit authorized personal prefix and owner authentication. It
returns paginated prefix and entry rows, the governing prefix, friendly key
reference and configured canonical key identifier. It does not read encrypted
payloads or call KMS. The policy key is not a claim about each historical
ciphertext's recorded encryption key.
Each page uses a Mongo snapshot. Migration receipts must match the current boundary epoch, provider and key reference. Stale completed receipts cannot establish completion; active ambiguous associations and legacy unconfigured keys remain explicitly unknown. Cursors bind the owner, tenant, prefix and tree schema, but pages do not share a frozen snapshot. Prefix-only boundaries and an entry sharing its path with a prefix remain distinct. Policy paths are paginated independently, governing policies are resolved only for selected-page ancestors, and migration lookup loads current epochs plus a bounded active-overlap summary. Old completed history does not make a small current tree unavailable; 1,105 empty boundaries and 1,105 historical receipts are covered by a real Mongo pagination regression. Each page uses five metadata operations, not one request per entry.
Seven actual HTTP/Mongo tests cover inheritance, empty boundaries, metadata
redaction, no crypto calls, pagination, owner/tenant denial, legacy/stale state
and concurrent atomic policy/migration cutovers. CLI list --tree, Python
vault.tree() and real PTY acceptance are the next slices; these examples are
proposed until those clients ship. Change and cross-boundary move preview remain
separate work. The aggregate TUI/retention issue requirement stays open.