Overview

Introduction

BSS is the S3-first media storage and streaming server. All binary data in DreamLake — videos, audio, labels, text tracks — flows through BSS to S3.

Architecture

Client upload
    │
    ├── Multipart upload ──→ BSS ──→ S3 staging pool
    │                                  /{owner}/{project}/staging/{hash}
    │
    └── POST /lambdas/hls-split ──→ AWS Lambda
                                       │
                                       ├── HLS chunks → S3 chunks/{hash}.ts
                                       └── Playlist   → S3 videos/{id}/stream/{hash}.m3u8

Core Concepts

Concept	Description
Staging Pool	Flat S3 prefix `{owner}/{project}/staging/{hash}` — holds raw uploaded files
HLS Splitting	Lambda splits video into chunked TS segments + m3u8 playlists
Content Addressing	Chunks are deduped by hash in a global `chunks/` pool
Metadata	JSON sidecar files in S3 per video/audio/track

S3 Layout

{owner}/{project}/
  staging/{hash}                    # Raw uploaded files
  videos/{videoId}/
    meta.json                       # Video metadata
    stream/{hash}.m3u8              # HLS playlists
  tracks/
    labels/{labelId}/meta.json      # Label track metadata
    text/{textTrackId}/meta.json    # Text track metadata
    audio/{audioId}/meta.json       # Audio metadata
chunks/
  {hash}.ts                         # Global chunk pool (deduped)

API Endpoints

Method	Path	Description
`POST`	`/videos/upload/multipart/init`	Start multipart upload
`POST`	`/videos/upload/multipart/complete`	Finalize upload
`POST`	`/lambdas/hls-split`	Trigger HLS splitting Lambda
`POST`	`/lambdas/audio-process`	Trigger audio processing Lambda
`GET`	`/videos/:id/stream/master.m3u8`	HLS playback
`GET`	`/videos/:id/raw`	Presigned download URL (302 redirect)

Quick Start

cd big-streaming-server && pnpm dev
# → http://localhost:10234

Tech Stack

Component	Technology
Framework	Express
Database	MongoDB (via Prisma)
Storage	AWS S3 / MinIO
Processing	AWS Lambda (HLS splitter, audio processor)
Auth	JWT