# `@dl.pipeline` A pipeline file contains many decorated functions, but exactly one of them is the pipeline. `@dl.pipeline` marks it. Its signature is fixed — a plain `def` that takes no parameters — because everything the graph knows comes out of the body, and nothing comes in from the outside. ## The signature ```python @dl.pipeline def camera_pose_trajectory(): ... ``` That is the whole contract. No arguments, no return value that matters, no configuration object. The function name becomes the pipeline's `id` and `title`; its first docstring line, if it has one, becomes the `subtitle`. The zero-argument rule is not a stylistic preference. A pipeline gets its input by *calling a source* — a UDF declared `kind="source"` — so there is nothing for a parameter to carry. If the pipeline took an argument, the tracer would have to invent a synthetic upstream node with no source text behind it, and the graph would contain a node that corresponds to no line of code. ## How the tracer finds it The tracer scans the module's top-level statements for function definitions and looks at each one's decorator list. For every decorator it takes **the attribute name and nothing else**: - `@dl.pipeline` → `pipeline` - `@dreamlake.pipeline` → `pipeline` - `@anything.pipeline` → `pipeline` - `@pipeline` → `pipeline` The recognised set is the single-element set `{"pipeline"}`. All four spellings above are equally valid, and there is no way to configure a different name. A decorator written as a call — `@dl.pipeline(...)` — resolves the same way, and its keyword arguments are collected but currently unused. (The parallel set for node decorators is `{"udf", "node"}`; see [`@ls.udf`](/pipelines/udf.md).) Nothing is imported and nothing is resolved. The tracer never asks where `dl` came from, whether it is a module or a local variable, or whether the attribute exists. The corollary is the useful part: **a pipeline file does not need a working `dreamlake` install to trace.** A file whose imports would all fail at runtime produces exactly the same graph as one whose imports resolve, because the import lines are never executed and the names in them are never looked up. Two consequences worth stating outright: - **The first match wins.** If a file defines two `@dl.pipeline` functions, the second one is silently ignored — it is not an error and it does not merge. - **Top level only.** The scan walks the module body, not nested scopes. A `@dl.pipeline` function defined inside another function or inside a class is invisible, and the trace fails with *"no `@dl.pipeline` function found in source"*. ## No import, no exec The tracer parses the file's AST. It never imports the module and it never executes a single line of it. This is why every UDF body in every shipped example is a bare `...` and the graph still comes out complete. The body of a stage contributes nothing to the graph: the input ports come from the parameter list, the output schema comes from the return annotation, and the edges come from the *call sites* inside the pipeline body. A stage that has not been written yet traces identically to one that has, so the graph is available from the first draft of the file. The consequence for authors runs the other way, and it is worth being blunt about. A pipeline file is checked by **parsing**, not by running. A syntax error fails the trace loudly. A type error inside a UDF body, a misspelled attribute, an import that does not exist, a call that would raise — none of these are caught here, and none of them will ever be caught here. The trace tells you the shape of the graph. It tells you nothing about whether the code works. ## A complete pipeline, and the graph it derives Here is a short pipeline end to end. Four stages, a source and a sink among them, wrapped in a batching loop: ```python file="pipelines/camera_pose_trajectory.py" import dreamlake as dl import lakeshore as ls from dreamlake import batch, requeue, to_dataset from lakeshore.types import Tensor, Tuple, Mask @ls.udf(kind="source") def load_frames() -> Tuple["frames", "intrinsics"]: ... @ls.udf def estimate_pose(frames, intrinsics) -> Tuple["pose", "confidence"]: ... @ls.udf def fit_trajectory(poses) -> Tuple["traj", "residual"]: ... @ls.udf(kind="sink") def save_trajectory(rows): to_dataset(rows) @dl.pipeline def camera_pose_trajectory(): src = load_frames() for items in batch(src, n=8): poses = estimate_pose(items.frames, items.intrinsics) traj = fit_trajectory(poses) save_trajectory(traj) ``` the graph @dl.pipeline above derives — no run required. The{' '} for items in batch(src, n=8) line produced no node. } /> Drag a card, scroll to pan, ⌘/ctrl-scroll to zoom. This is the real renderer reading a real trace fixture, not a picture of one. Four things in that graph are worth reading off deliberately: - **The `for` loop left no trace.** `batch(src, n=8)` sits in a loop iterator position, where the tracer relaxes its decoration rule and passes the argument's provenance straight through. `items` inherits whatever `src` descended from, and no node appears. Loops never unroll either — the body is walked exactly once, so a loop over eight batches still yields one `estimate_pose`. See [batching](/pipelines/batching.md) and [control flow](/pipelines/control-flow.md). - **Two edges run into `estimate_pose`, not one.** An edge is keyed on `(from, fromPort, to, toPort)`, and the two arguments land on two different input ports. `items.frames` and `items.intrinsics` are attribute reads, which are not nodes — they carry the provenance of `items` and therefore both point back at `load_frames`. - **`estimate_pose` is still a `transform`.** Two inbound data edges would normally infer the `merge` kind, but edges arriving from a `source` node are excluded from that count, so a stage fed entirely by its source stays a plain transform. See [transform and merge](/pipelines/transform-and-merge.md). - **`save_trajectory` has no output port.** It has no return annotation, and that absence *is* the definition of a terminal — one input port, zero output ports, an empty column list. ## One entry point per file The tracer walks the decorated function and only the decorated function. Everything else in the file is a *definition* until the pipeline body reaches for it. Helper functions defined alongside the pipeline are therefore traced only when the pipeline body calls them, and calling one is only legal if it carries a node decorator. Every call the walk encounters must resolve to a `@ls.udf` or `@dl.node`; an undecorated call raises rather than passing silently through: > pipeline calls undecorated function `normalise` — every function called in a > `@dl.pipeline` must be a `@ls.udf` / `@dl.node` (wrap framework helpers like > `to_dataset`/`requeue` in a udf) Failing loudly is the deliberate choice. Passing an unknown call through would produce a graph that is quietly missing a stage, and a derived graph that omits work is worse than no graph at all. The one exemption is the loop-iterator position described above, which exists precisely so `batch(...)` and `stream(...)` do not have to be pretend stages. Method calls, subscripts, and operators are not calls in this sense — they pass provenance through without creating a node, which is what makes `traj[ok]`-style gating render as an edge tag rather than a box. > **Warning:** `@dl.pipeline` does not exist in shipped Python. The `dreamlake` package exports > no `pipeline`, `node`, `batch`, `stream`, `to_dataset`, or `requeue`. > > `lakeshore.udf`'s real signature is > `udf(fn=None, *, queue=None, transport="auto", config=None)` — there is **no > `kind=`** parameter. `kind` is read by the *tracer* out of the decorator's > keyword arguments and never reaches the runtime. `lakeshore.types` does not > exist either, and the canonical import for the real package is > `import dreamlake.lakeshore as dls`, not `import lakeshore as ls`. > > None of that stops the example above from tracing, and the reason is the whole > point of this page: these files are a **specification format that happens to be > Python syntax**. They trace precisely because nothing is ever imported.