Host password authentication and rotation

Status: In progress — probes merged; reservation backend under local validation, not released. Tracking: Vault #241, master #247. Saved passwords and target/jump bindings exist; saving does not establish SSH authentication. This hidden draft remains publicly reachable.

The first slice verifies a saved password through a fresh password-only SSH connection. Isolated config excludes other identities, agents, keyboard-interactive and connection reuse. A packaged internal askpass helper receives the selected password through private one-shot IPC, never secret arguments, environment values or journal JSON. Parent-client authentication evidence plus a nonce and remote identity distinguish real password login from a command pretending to authenticate. Temporary authentication logs are bounded and deleted, never published. Python does not prompt. Target and jump credentials are selected separately; the first target-through-jump transport uses a separately selected jump key.

# Proposed; requires the reviewed client implementation.
dreamlake vault verify-password --binding-id "$password_binding_id" \
  --ssh 'worker@worker.example' --known-hosts "$HOME/.ssh/known_hosts"

result = client.vault.verify_host_password(
    binding_id=password_binding_id,
    ssh={"host": "worker.example", "user": "worker", "port": 22,
         "knownHostsFile": str(known_hosts)},
)

Rotation contract — next reviewed slice

Passwords normally have one active value; key-style additive overlap is unavailable. A separately verified recovery-key connection must survive through final proof. Save the distinct replacement first, then reserve the exact binding and both live entry revisions before remote mutation. The pending operation retains both encrypted entries and blocks overlapping changes to the same binding. A remote account-scoped guard also prevents aliases from racing. Change only the pinned account through an explicitly selected privileged Linux adapter, with the password on encrypted stdin. No global SSH/PAM policy changes, shadow-hash export or password-derived verification token are permitted.

After an uncertain outcome, probe the new password then the old password once. If the new works and old is denied, finish the same reserved operation; ambiguous results retain recovery and references. Never automatically roll back. Final proof has a discriminated password method, entry IDs, remote account identity and ordered timestamps; key fingerprints are not password evidence. The backend records client attestation. Shared entries are not automatically retired, and password changes do not end existing sessions. Reservation/schema, privileged mutation, explicit recovery and password confirmation remain unimplemented until the next review.

Reservation backend implementation plan

Proposed, pending contract review. Inspection of backend 433c4cd found that entry retention currently protects an entry's identity, not its historical ciphertext revision: normal writes replace the single VaultEntry document. Locking a binding alone cannot preserve the old password after a shared entry is edited. The chosen proposal retains encrypted, operation-owned revision snapshots and freezes only the dedicated replacement entry while pending. The shared old entry remains writable. Snapshots participate in KMS inventory and migration; this is required work, not an optional optimization. Freezing shared source entries is not the selected design.

Model VaultHostPasswordRotation with tenant, immutable operation ID/intent, binding/host/enrollment/role/endpoint, exact old and replacement entry IDs/revisions, an explicit remote account identity and separately verified recovery-key identity. Store the two encrypted snapshots in a modeled private collection using the original cryptographic context. Public operation metadata never contains ciphertext, passwords, password hashes or bearer tokens. No snapshot TTL may remove recovery material while an operation is pending. The recovery key remains client-owned; a public key fingerprint may identify that key, never the password.

Reservation transaction verifies personal ownership, a live password binding, distinct live string entries at the requested revisions, and no other pending operation for the binding. It copies the exact encrypted revisions and records reserved without changing the binding. A binding operation guard serializes reserve with unbind, supersede and other reservation operations. Entry touches serialize snapshot capture with writes/retirement/purge; KMS policy epoch checks serialize capture with migration. Reservation additionally requires the replacement to have no active binding, pending host operation or active access-key scope, and no expiresAt. It acquires a unique tenant/entry reservation guard for that dedicated replacement. Every semantic entry writer, retirement/restore path, new binding, supersession, access-key grant and purge checks this guard transactionally; attempts return CREDENTIAL_RESERVED rather than overwriting or sharing it. Shared old-entry writes remain possible because recovery uses the old snapshot. KMS rewrapping may update encryption only, preserving entry ID, logical revision and reservation guard; its migration transaction and snapshot inventory must include this collection. A delayed precomputed write must recheck the guard, not merely its former entry revision. Replay of the same operation and exact intent returns its receipt; reuse with another intent returns 409.

Proposed owner-only HTTP endpoints under /v1/vault are PUT /host-credentials/:bindingId/password-rotations/:operationId (reserve), GET /host-password-rotations/:operationId (metadata), POST /host-password-rotations/:operationId/read (explicit selected old/new snapshot read), POST .../:operationId/start and POST .../:operationId/confirm. CLI/Python wrappers must share this contract; automatic libraries never prompt. The start transition is durably acknowledged before any remote mutation. Recovery is bounded and explicit; expiry, lockout or lost remote outcomes must not silently release the binding guard or snapshots. The pending-operation read is an explicit personal-owner recovery exception: it may decrypt the pinned snapshot after its original entry expires, is overwritten or is retired. Ordinary entry reads still enforce their normal lifecycle. Access keys and other owners cannot use this exception; operations and reads retain their original tenant and immutable intent scope even if the host disappears.

The paired low-level client contract is merged but unreleased in CLI #64 (b4aefc10) and Python #51 (7167a30e): vault password-rotation reserve|show|read|start|cancel|confirm and Python reserve_host_password_rotation, host_password_rotation, read_host_password_rotation, start_host_password_rotation, cancel_host_password_rotation, confirm_host_password_rotation. These recovery primitives do not replace the future human-facing rotate-password flow. Reserve and confirmation accept metadata only; the explicitly selected read is the sole plaintext-returning call. All calls use the same pinned account/API origin and immutable operation ID; unknown outcomes recover by show or exact replay, never a new operation or automatic SSH retry. CLI request files must reject password/private-key/token fields before network I/O. Python calls never prompt.

# Proposed low-level API wrappers; metadata only in these files.
dreamlake vault password-rotation reserve \
  --binding-id "$binding_id" --operation-id "$operation_id" \
  --input "$HOME/.dreamlake/password-rotation-intent.json"
dreamlake vault password-rotation show --operation-id "$operation_id"
# Start must be durably acknowledged before the privileged client changes anything.
dreamlake vault password-rotation start --operation-id "$operation_id"
# Confirmation follows new/old/new password and recovery-key proof.
dreamlake vault password-rotation confirm --operation-id "$operation_id" \
  --input "$HOME/.dreamlake/password-rotation-proof.json"

# Proposed paired library contract; intent and proof contain no passwords.
operation = client.vault.reserve_host_password_rotation(
    binding_id=binding_id, operation_id=operation_id, intent=intent,
)
operation = client.vault.host_password_rotation(operation_id)
operation = client.vault.start_host_password_rotation(operation_id)
# The separately reviewed privileged adapter runs between start and confirm.
operation = client.vault.confirm_host_password_rotation(operation_id, proof=proof)

States are reserved → mutation_pending → confirmed. Cancellation is allowed only from reserved, transactionally competing with start; after mutation_pending, ambiguous outcomes remain pending until explicit evidence resolves them. No automatic timeout reset or rollback is permitted. Remote adapter journals an account-scoped operation marker under a privileged helper before changing the password, preventing concurrent aliases from bypassing the backend binding guard. It pins account UID, user, home identity and machine identity and checks the separately selected recovery key on every resumed mutation. Neither a matching hostname nor a successful password alone establishes the DreamLake HostId.

Confirmation first verifies the dedicated replacement still exists at its frozen ID/revision, is live and non-expiring, and remains exclusively reserved by this operation. In the same transaction it switches the still-pinned binding to that exact usable replacement and releases both guards only after an exact client attestation: new password accepted, old authentication refused, new password accepted again, and recovery key accepted, with ordered timestamps and the same remote account identity. A discriminated ssh_password proof uses entry IDs and revisions; it cannot enter the existing key-fingerprint cleanup endpoint. Replays preserve the exact first receipt; another proof/intent conflicts. Metadata edits that invalidate the pinned host/enrollment/binding keep the operation unresolved rather than invent success. Confirmation does not kill existing sessions or retire a shared source entry. After verified confirmation, encrypted snapshots get a 30-day retention deadline; a bounded transactional collector purges them after the deadline using the immutable verified terminal receipt as authorization. Later legitimate rotation or unbinding does not extend obsolete ciphertext retention. Only an explicit active recovery reference may defer collection. Pending or uncertain snapshots have no TTL. Cancellation before start relies only on the immutable personal-owner operation and its still-reserved state; it releases only that operation's guards even if the shared old entry changed/expired or the host disappeared. The same transaction competes with start, so a possibly started mutation can never use this cancellation path. Terminal metadata receipts remain available under the existing metadata policy, without ciphertext after purge.

Implementation order and required tests:

• [vault/hosts/password/reservation/schema] Model operation, encrypted revision snapshots and binding guard; register unique owner/operation and active-binding indexes.
• [vault/hosts/password/reservation/transactions] Add reserve/read/start/cancel/confirm with paired CLI/Python contracts; reject access-key principals, foreign owners, released bindings, invalid calendar times, stale host/enrollment, expired revisions and unknown fields without secret output.
• [vault/hosts/password/reservation/retention] Include snapshot ciphertext/context in KMS preview/migration/recovery and cleanup; prove old entry overwrite/expiry/retirement/purge cannot lose pending recovery bytes; freeze the dedicated replacement against every semantic mutation and new reference while permitting context-preserving KMS rewrap.
• [vault/hosts/password/reservation/races] Real HTTP/Mongo tests: reserve versus reserve, unbind, supersede, entry write, retirement, GC and KMS migration; same-request lost-response recovery; cancel versus start and confirm versus later binding changes. Only one overlapping operation wins. Include replacement reuse by another binding/access key, delayed pre-reservation writes, replacement expiry rejection, tampered missing/revised replacement at confirm, pending reads after old-entry expiry, 30-day terminal collection, changed final binding, and deleted/recreated HostId. A successful confirmation must always resolve to a readable live replacement revision.
• [vault/hosts/password/rotation/remote] Client-owned privileged adapter and immutable cross-client journal; faults before/after every durable boundary, reused account, wrong recovery key, new failure, old refusal followed by new failure, and both-password refusal. Keep recovery material after every uncertain outcome.
• [vault/hosts/password/rotation/live] Fresh disposable target and jump accounts, native/npm/Python parity and cleanup ownership, using the reviewed private SSHD fixture. Verify actual password changes separately from the already accepted password-only probe.

A deleted host leaves an owner-readable unresolved operation and retained snapshots. Neither recreating a name nor passing another HostId can confirm it. The remaining explicit rollback/recovery-resolution flow must independently attest the original remote account has been restored or removed and establish a verified final credential/binding (or account retirement) before releasing guards or starting the 30-day cleanup clock. It is a required subsequent lifecycle transition, not an automatic timeout, permission bypass or an excuse to mark password rotation complete.

Tasks and acceptance

• [vault/hosts/password/design] Separate saved bytes, actual authentication and non-additive rotation; inspect existing consent/FD/binding and key-only cleanup contracts.
• [vault/hosts/password/probe] Implement paired CLI/Python password-only probes with exact authentication classification, private askpass IPC and pinned owner/entry/remote scope.
• [vault/hosts/password/input] Test selected private-file/pipe input through native and actual npm launchers, exact UTF-8 policy and zero secret output.
• [vault/hosts/password/live] Run both clients against private loopback SSHD instances and disposable Unix users, including direct jump and target-through-jump positive/denial cases.
• [vault/hosts/password/faults] Prove prompt injection, alternate-auth fallback, forged denial, wrong host key, target/jump separation, stale bindings, account recreation and bounded process/log cleanup.
• [vault/hosts/password/reservation] Review modeled operation schema/API, owner/CAS guards and pending-reference GC interaction before password mutation.
• [vault/hosts/password/rotation] Implement paired mutation/recovery lifecycle, every durable-boundary failure and concurrent one-winner tests.
• [vault/hosts/password/release] Review, release and verify hosted enrollment/KMS acceptance separately from source-fixture evidence.

The live fixture uses complete private daemon configs, unique host keys, PID files and loopback ports; only randomly marked users are allowed. Existing authenticated admin SSH provides bounded local forwards and trusted fixture host-key retrieval. No system daemon/config or existing user is modified. Cleanup checks account UID/GECOS/home and daemon executable/start time/config immediately before deleting or signaling, verifies absence, then releases synthetic bindings and retires entries. Failed cleanup preserves recovery material.

Official references: OpenSSH askpass and debug logs, private daemon authentication policy, shadow-utils chpasswd stdin and PAM failure behavior.

Required deployment order and retained-record compatibility

Password snapshots add a fourth encrypted record kind. Before deploying reservation-capable writers, disable KMS migration on every existing writer, verify the disabled runtime, and drain older writers. Deploy every new writer with migration still disabled; verify each reports retainedRecordSchemaVersion: 2 and all four retainedRecordKinds. Only then enable migration. A rolling deployment with older migration-enabled writers is unsafe because those writers do not inventory password snapshots. Capability metadata alone cannot fence old code.

New CLI/Python clients display passwordSnapshotCount separately and totalRetainedRecordCount across entries, write receipts, HOTP receipts, and password snapshots. They acknowledge schema 2 on migration start/resume. The server rejects an older client with KMS_RETAINED_RECORD_UPGRADE_REQUIRED before migration progress when snapshots are retained. Prefixes without snapshots remain compatible. Inventory checks share the tenant transaction guard with reservation creation; an operation's required schema ratchets to 2 and never falls back, including when snapshots appear after preview or start. Read-only status remains available for recovery. No count is hidden inside entryCount.