Engineering Multimodal Media Pipelines at Scale: Inside Repurplex

Introduction

Processing gigabytes of uncompressed media content per minute requires an elastic, fault-tolerant ingestion pipeline. Repurplex handles thousands of daily user uploads by decoupling media chunking from language generation tasks.

Architecture Breakdown

1. Ingestion Layer: Cloudflare Workers route incoming media chunks directly to AWS S3 buckets.
2. Audio Extraction: Rust workers leverage FFmpeg compiled to WebAssembly for sub-second audio isolation.
3. Whisper Transcription Cluster: GPU instances run batched Whisper models with custom phrase dictionaries.
4. Synthesis Engine: Fine-tuned Claude models generate platform-tailored summaries and threads in parallel.