Lambda

Lambda Design

BSS delegates heavy media processing to AWS Lambda. A single Lambda function (hls-splitter) handles both video and audio, dispatched by the type field in the event payload.

Architecture

BSS Server
 │
 ├─ POST /lambdas/hls-split { videoId }
 │   → Lambda invoke (type: "video")
 │
 └─ POST /lambdas/audio-process { audioId }
     → Lambda invoke (type: "audio")
 
Lambda (hls-splitter)
 │
 ├─ Download raw file from S3 staging pool
 ├─ Process with ffmpeg / ffprobe
 ├─ Upload results back to S3
 └─ Optional callback PATCH to BSS

Event Payload

interface MediaEvent {
  type?: 'video' | 'audio'    // default: 'video'
  videoId?: string             // required for video
  audioId?: string             // required for audio
  owner: string                // S3 namespace (owner slug)
  project: string              // S3 project slug
  stagingHash?: string         // SHA256[:16] of staged file
  rawHash?: string             // deprecated, fallback for stagingHash
  fps?: number                 // hint fps for video (overridden by ffprobe)
  callbackUrl?: string         // optional PATCH endpoint on completion
}

Video Pipeline

9-step pipeline that splits a raw video into HLS segments:

Step	Action	S3 Reads	S3 Writes
1	Download raw file from `{owner}/{project}/staging/{hash}`	1	—
2	Probe actual fps with ffprobe	—	—
3	Split with ffmpeg into TS segments (no re-encode, `copy` codec)	—	—
4	Parse ffmpeg m3u8 output for segment durations	—	—
5	Hash each segment, upload to global dedup pool `chunks/{hash}.ts`	N (dedup check)	0–N
6	Build stream m3u8 playlist (bare hex hashes for BSS rewrite)	—	—
6.5	Generate thumbnail + sprite sheet (non-fatal)	—	0–2
7	Upload stream playlist to `videos/{id}/stream/{hash}.m3u8`	—	1
8	Update `videos/{id}/meta.json` with length, fps, duration, streams, thumbnail/sprite	1	1
9	Optional callback PATCH to BSS	—	—

Chunk Deduplication

Each TS segment is content-addressed by SHA256[:16]. Before uploading, the Lambda checks if chunks/{hash}.ts already exists. Duplicate segments (common in re-uploads or similar content) are skipped.

M3U8 Format

The stream playlist uses bare hex hashes as chunk references:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:6.000,
a1b2c3d4e5f67890
#EXTINF:4.500,
1234567890abcdef
#EXT-X-ENDLIST

BSS's rewriteM3u8() replaces each 16-char hex hash with a full CDN/presigned URL before sending to the client.

Thumbnail & Sprite Sheet

Generated alongside HLS segments (non-fatal — skipped on error):

Asset	Size	Location
Thumbnail	640px wide, single frame at ~10% of duration	`videos/{id}/thumb.jpg`
Sprite	160x90 tiles, 1 frame every 5s, max 10 columns	`videos/{id}/sprite.jpg`

Both URLs are stored in meta.json and served via CDN.

Video Result

{
  videoId: string
  streamHash: string      // hash of the m3u8 playlist
  chunks: number          // number of TS segments
  durationSec: number
  totalFrames: number
}

Audio Pipeline

Simpler than video — probe metadata only, no splitting:

Step	Action
1	Download raw file from `{owner}/{project}/staging/{hash}`
2	Probe with ffprobe: sampleRate, channels, codec, bitRate, duration
3	Optional callback PATCH to BSS with metadata

Audio Result

{
  audioId: string
  durationSec: number
  sampleRate: number      // Hz (e.g. 44100, 48000)
  channels: number        // 1 = mono, 2 = stereo
  codec: string           // e.g. "aac", "opus", "pcm_s16le"
  bitRate: number
}

Configuration

Env Var	Default	Description
`S3_BUCKET`	(required)	S3 bucket name
`AWS_REGION`	`us-east-1`	AWS region
`FFMPEG_PATH`	`/opt/bin/ffmpeg`	Path to ffmpeg binary
`FFPROBE_PATH`	`/opt/bin/ffprobe`	Path to ffprobe binary
`SEGMENT_DURATION`	`6`	HLS segment duration in seconds
`CDN_BASE`	S3 direct URL	CDN base URL for thumbnail/sprite

Error Handling

ffprobe failure: falls back to hint fps (video) or empty metadata (audio)
Thumbnail/sprite failure: non-fatal, URLs set to null in meta.json
Callback failure: non-fatal, logged as warning
Segment mismatch: throws error (ffmpeg output corrupted)

S3 Layout After Processing

{owner}/{project}/
  staging/{hash}                         # original upload (retained)
  videos/{videoId}/
    meta.json                            # updated with length, fps, streams, etc.
    stream/{streamHash}.m3u8             # HLS playlist
    thumb.jpg                            # thumbnail (optional)
    sprite.jpg                           # sprite sheet (optional)
chunks/
  {hash}.ts                              # global dedup pool