D
DreamLake

Overview

Concepts

The one-paragraph story

You have many tracks. Each track has a dtype (e.g. joint_angles, imu_6dof, video). Track data lives in m3u8 + chunks on the server. You want to visualize that data two ways — as a standalone view of the current instant state, or as a lane in a multi-track timeline showing trends over time. Both dispatch by dtype. Both are extensible.

That's the library.

The mental model

┌─────────────────────── DATA ───────────────────────┐
│                                                    │
│   Track { id, name, path, dtype, src }             │
│                                                    │
│   — path      "robot/joint_positions"              │
│   — dtype     "joint_angles" (names the schema)    │
│   — src       m3u8 URL pointing to JSONL chunks    │
│                                                    │
└──────────────────────┬─────────────────────────────┘
                       │
           ┌───────────┴────────────┐
           │                        │
           ▼                        ▼
┌── TRACKER ───────────────┐   ┌── TIMELINE ──────────────┐
│                          │   │                          │
│  Instant state           │   │  Trends across time      │
│  at currentTime          │   │  across many tracks      │
│                          │   │                          │
│  <TrackerContainer       │   │  <TimelineContainer      │
│     config={config}      │   │     config={config}      │
│     views={…} />         │   │     views={…} />         │
│                          │   │                          │
│  dispatch dtype → view   │   │  dispatch dtype → lane   │
│                          │   │                          │
└──────────────────────────┘   └──────────────────────────┘
 
Both sides take the SAME `config` object. Wrap both in one
<ClockProvider> and play/seek syncs across them.

The data exists independently of how you choose to draw it. The same TimelineConfig can feed a stack of instant-state views (<TrackerContainer>) and a scrubbable trend-timeline (<TimelineContainer>) at the same time. Swap components to swap presentations — no data transformation, no glue code.

1 — Data

Tracks are where everything starts. A track is a topic: a specific stream of time-stamped samples, produced once (on the server) and potentially visualized many ways (on the client).

  • path is the topic identifier — "robot/joint_positions", "cameras/wrist", "events". /-separated segments naturally form a tree; the timeline synthesizes groups from path prefixes.
  • dtype is the data-type identifier — "joint_angles", "imu_6dof", "video", "action_label". It names a schema documented in the Dtypes registry: chunk format, JSONL line shape, unit/range defaults, compatible presentations.
  • src is the m3u8 URL. Chunks follow HLS conventions; the library handles parse + fetch + decode + prefetch + cache.

Everything else (validation, server enforcement, grouping) is convention — the server stores opaque bytes, the frontend trusts the author to keep chunks and dtype in sync.

See Tracks, the Dtypes registry, and M3U8 Transport for details.

2 — Two visualization modes

vuer-m3u covers two distinct visualization needs with two presentation surfaces that share the data model.

A. Instant state — Views (and <TrackerContainer>)

Single-pane visualizations that render the current instant (at clock.time).

Direct import — when you know which view fits:

<JointAngleView src="/robot/arm/qpos.m3u8" />
<VideoPlayer src="/cams/wrist.m3u8" />

Dispatch by dtype — when you have a TimelineConfig and want one view per track:

<TrackerContainer config={config} views={defaultTrackerViews} />

<TrackerContainer> walks config.tracks and renders each one using the view component its dtype resolves to in the views map. Same TimelineConfig that <TimelineContainer> accepts — one object, two presentations.

vuer-m3u ships 10 built-in views covering video, IMU, joint angles, poses, action labels, detections, subtitles. Custom views follow the same hooks pattern — see Custom Views.

B. Trends over time — Timeline

A multi-track editor-style surface (<TimelineContainer>) that shows all tracks at once, scrubable through time. Each track renders as a lane — a horizontal strip dispatched from the track's dtype via a views map.

<TimelineContainer
  config={{
    container: { id: 'ep1', duration: 30 },
    tracks: [
      { id: 'cam',  path: 'cams/wrist', dtype: 'video',        src: '...' },
      { id: 'qpos', path: 'robot/arm',  dtype: 'joint_angles', src: '...' },
      { id: 'ops',  path: 'events',     dtype: 'action_label', src: '...' },
    ],
  }}
  views={defaultTimelineViews}
/>

The views prop maps dtype → lane component. defaultTimelineViews wires the 13 built-in dtypes to 4 built-in lanes; apps override individual dtypes to plug in custom lanes.

See Timeline Overview for anatomy and Authoring for the config format.

Shared foundation

Both modes run on the same TimelineClock — wrap them in one <ClockProvider> and they're automatically time-synchronized. A playhead move in the timeline updates the joint-angle view in real time; a seek in one surface propagates everywhere.

Both modes consume the same data — one track, two presentations, zero duplication.

3 — Extensibility

Three extension points, all with the same flavor: register a thing, use it.

ExtensionAPIWhere
Register a custom dtyperegisterDtype({ id, name, defaults })Dtypes
Register a custom chunk decoderregisterDecoder(format, fn)Custom Decoders
Plug a custom lane<TimelineContainer views={{ dtype: { lane: MyLane } }} />Custom Lanes
Write a custom viewJust a React component + hooksCustom Views

Common pattern: register → render.

registerDtype({
  id: 'gripper_force',
  name: 'Gripper force',
  defaults: { range: [-50, 50], unit: 'N' },
})

Then any track with dtype: 'gripper_force' inherits those defaults on the timeline, and any view that reads {ts, data} shape can render it.

4 — Under the hood

The transport and clock layers sit below both visualization surfaces. You usually don't touch them directly — hooks abstract most of what you need — but they're public API:

  • TimelineClock — pure time source; emits tick + seek, knows nothing about playlists
  • Playlist — parse m3u8, fetch chunks, decode, prefetch, cache; per-track instance
  • Hooks — 4-layer model: useSegment → useSegmentTrack → useMergedTrack → useTrackSample
  • Decoders — built-in jsonl / vtt / ts, plus registerDecoder for custom formats

Clock is a ref, not a prop — views pick their own re-render rate via useClockValue(fps) or subscribe imperatively via clock.on('tick').

What the library does NOT do

  • Not a schema validator. dtype is convention — chunks that don't match the documented schema make their view break, not the library.
  • Not a track database. Server storage is your concern (we recommend the dreamlake-py SDK).
  • Not opinionated about domains. Ships robotics dtypes (IMU, joint angles, pose) because that's our use case, but the registry is open — add your own.
  • Not a UI kit. Components are functional; styling is mostly Tailwind + theme tokens you can override.

Next