# Command line programs

  The second UDF type is a program you run rather than a function you import.
  The unit of work is an argv line, not a callable — which makes it the right
  shape for anything already written as a script, a binary, or a tool you did
  not author.

The first type, a [simple function](/lakeshore/patterns.md), travels as a
reference the worker imports. A command line program travels as a **string the
worker executes**. Everything else — the queue it lands in, the workers that
claim it, the provider that supplied them — is identical.

## Declaring the interface with params-proto

A bare script has no declared interface: the flags it accepts exist only inside
its argument parser. [params-proto](https://github.com/geyang/params-proto)
turns the signature itself into the interface, so the flags, types, defaults and
help text all come from one place.

```python file="encode.py"
from params_proto import proto


@proto.cli
def encode(
    source: str,                 # Input video key, relative to the workdir
    target: str = "frames/",     # Output key prefix, relative to the workdir
    fps: int = 4,                # Frames sampled per second of source video
    width: int = 854,            # Output frame width in pixels; height follows aspect
    overwrite: bool = False,     # Re-encode even when target already holds frames
):
    """Sample frames from a video and write them into the run scope.

    Reads the video at `source`, samples one frame every `1 / fps` seconds,
    resizes each to `width`, and writes them as JPEGs under `target`. Existing
    frames are left alone unless `--overwrite` is passed.

    Prints one written key per line on stdout so the caller can collect the
    result without re-listing the prefix.
    """
    ...


if __name__ == "__main__":
    encode()
```

Two things carry over into the CLI automatically. The **docstring becomes the
program description**, and each **inline comment becomes that flag's help
text** — so `--help` is generated from the same source the function is defined
by, and cannot drift from it:

```bash file="terminal"
$ python encode.py --help
Sample frames from a video and write them into the run scope.
...
  --source      STR    Input video key, relative to the workdir
  --target      STR    Output key prefix, relative to the workdir    (frames/)
  --fps         INT    Frames sampled per second of source video     (4)
  --width       INT    Output frame width in pixels; height follows aspect  (854)
  --overwrite   BOOL   Re-encode even when target already holds frames (False)
```

Underscores in a parameter name become dashes on the command line, so
`batch_size` is `--batch-size`. Booleans are flags: `overwrite: bool = False`
gives `--overwrite`, and a parameter defaulting to `True` gives `--no-<name>`.

## Invoking it

```bash file="terminal"
lakeshore exec --queue gpu "python encode.py --source raw/run01.mp4 --fps 8"
```

The queue is the same object a Python UDF submits to — same claim discipline,
same workers, same elasticity. See [Queue model](/lakeshore/queues.md).

You can pin to a specific worker instead of routing through a queue, but not
both; `queue` and `worker_id` are mutually exclusive.

## Declaring access

A [simple function](/lakeshore/patterns.md) declares what it may reach on the
decorator — `source=` and `target=`, patterns relative to the workdir. A command
line program has no decorator, so the declaration moves to the exec body,
alongside the command:

```json file="POST /v1/namespaces/:ns/exec"
{
  "queue": "gpu",
  "command": "python encode.py --source raw/run01.mp4 --fps 8",
  "workdir": "/workspace",
  "source": ["raw/**"],
  "target": ["frames/**"],
  "mounts": ["raw-video"]
}
```

Same three fields, same meaning, same workdir-relative patterns — see
[Declaring access](/lakeshore/access.md). A repo-wide `.dreamrc` supplies defaults
for both UDF types, so a program that inherits everything it needs declares
nothing here.

> **Note:** The parameters above are *named* `source` and `target` so both UDF types read
>   the same way. Nothing in the runtime binds them.
>
>   That is deliberate. The declaration and the argument are different layers: the
>   declaration says which family of keys this program may ever touch, and the
>   argument says which one this run touches. A magic parameter would collapse
>   them — and would mean any program with a parameter incidentally called
>   `target` acquired behaviour it never asked for.
>
>   So name your primary input `source` and your primary output `target` because
>   it reads well and matches the spec. A program with three inputs should give
>   them descriptive names rather than contorting to fit one reserved word.

> **Warning:** The exec body accepts `queue`, `worker_id`, `command`, `env`, `workdir`,
>   `timeout_s`, `stdin_b64`, and the payload fields. `source`, `target` and
>   `mounts` are not among them, and nothing enforces access on either UDF type
>   today.

## The JSON representation

Two JSON objects matter here, and they are easy to confuse.

### The parsed configuration

params-proto will hand you the resolved parameters as a plain dict, which is
what you serialize when you want to record what a run was actually configured
with:

```json file="config.json"
{
  "source": "raw/run01.mp4",
  "target": "frames/",
  "fps": 8,
  "width": 854,
  "overwrite": false
}
```

Reach it with `dict(encode)` or `encode._dict`. Values are resolved by
precedence — CLI arguments beat direct assignment, which beats a bound context,
which beats environment variables, which beats the declared defaults — so the
dict reflects the run rather than the source.

### The exec wire

What actually crosses to the control plane is the exec body. `command` is the
argv line as a single string:

```json file="POST /v1/namespaces/:ns/exec"
{
  "queue": "gpu",
  "command": "python encode.py --source raw/run01.mp4 --fps 8",
  "env": { "PYTHONUNBUFFERED": "1" },
  "workdir": "/workspace",
  "timeout_s": 900,
  "payloadInline": "<base64>"
}
```

| Field | Meaning |
| --- | --- |
| `queue` | Queue name to route through. Required unless `worker_id` is given; the two are mutually exclusive. |
| `worker_id` | Pin to one worker instead of routing. |
| `command` | The command to run. Required, non-empty. |
| `env` | String-valued environment overlay. |
| `workdir` | Working directory on the worker. |
| `timeout_s` | Wall-clock budget; exceeding it is a terminal `timeout`. |
| `stdin_b64` | Base64 stdin, if the program reads any. |
| `payloadInline` / `payloadRef` | The input payload — inline base64 below the server's inline threshold, otherwise a `{ bucket, key, size }` reference. **Exactly one** may be set. |

The response is a `202` carrying the handle:

```json
{ "exec_id": "01JD2K7Q...", "worker_id": "wkr_..." }
```

Results come back by that id, the same way an invocation's do:

```
POST /v1/namespaces/:ns/jobs/:id/result     { resultInline } or { resultRef }
GET  /v1/namespaces/:ns/jobs/:id/result?wait=N
  → { state, resultInline?, resultRef?, exit_code?, error? }
```

`exit_code` is the part with no analogue on the Python side — a command line
program reports failure by exiting non-zero rather than by raising.

> **Warning:** Command line programs dispatch through `POST /v1/namespaces/:ns/exec`, not
>   through the invocation path a `@udf` submit takes. They share the queue and
>   the workers, but they are two wires: exec jobs do not appear as `Invocation`
>   rows, so they are absent from invocation listings and from anything built on
>   invocation lineage. Treating both as "UDF types" is the intended direction,
>   not a description of one unified code path.

## Large inputs

`payloadInline` is capped server-side. Above that threshold, presign a key
through the payload store and send `payloadRef` instead — the key must begin
with `payloads/`, which is what the presign route mints. The rule is the same
one the Python side follows: bytes stay out of the wire and travel by
reference.

    `source=`, `target=` and `mounts=` in full — patterns, wildcards, and how
    the workdir prefix is assembled.

    The queue both types submit to, field by field.