VIS
Make video as interactive as text.
The problem
Working with objects inside video — masking faces, removing objects, extracting clips, counting — normally means configuring complex AI pipelines instead of just… pointing at the thing.
How it works
Upload a video that plays immediately; click objects or type commands (“blur this”, “count cars”). An orchestrator fuses clicks and text into actions against a video-intelligence engine — YOLO detection, promptable SAM-3 segmentation with mask propagation, blur/mask, inpainting removal, and clip extraction — in a three-tier execution model (live preview → refine → final render) so the UI never freezes. Explicitly not surveillance: no face recognition, no biometric persistence.
Key features
- Click-to-select + natural-language commands on a video canvas and timeline
- Promptable segmentation (SAM-3) with mask propagation across frames
- Blur/mask, object removal via inpainting, clip extraction, object counting
- Three-tier real-time execution targeting <300ms click→mask
- Hard privacy constraints — no identity, biometrics, or cross-video linkage
Architecture
A React (Vite) frontend talks to a FastAPI backend over WebSockets (real-time) and REST (batch): an interaction orchestrator, a video-intelligence engine (YOLOv8 + SAM-3 + tracking), and an action/render engine (FFmpeg + LaMa inpainting).