Skip to content
ARGUS User Manual

AI detection overview

ARGUS has three detection sources that all converge on the same overlay pipeline so the UI doesn’t care which one produced a given bounding box. You’ll see the same coloured boxes and labels on the video regardless of whether the source was YOLO in the cloud, DJI SEI from the aircraft itself, or the training simulator.

The three sources

YOLO11 cloud agent

An argus-agent worker joins the room as a bot participant, subscribes to drone video tracks, runs YOLO11 inference at ~30 fps, and publishes detections on the LiveKit data channel as { type: 'detections', participantIdentity, detections[] }. Each detection carries left/top/right/bottom (normalised 0-1), trackingId (short-term BoT-SORT, resets on occlusion), persistentId (DINOv2 + FAISS re-ID, stable across occlusions — format p_xxxx), and a multi-label confidence map.

DJI SEI (on-aircraft)

DJI Dock 3 aircraft embed detections in the H.264 SEI payload as they stream. Our in-browser parser in dji-sei-parser.ts strips emulation-prevention bytes, decodes the multi-byte SEI length, identifies the transport marker (0xF5 for WebRTC, 0x65 for legacy Agora), and returns bounding boxes normalised to 0-1.

Detection simulator

The PolygonSimService runs scripted drone-path simulations with synthetic detection output. Used for training, demos, and E2E testing the watchlist / copilot-alert paths without live aircraft.

How they converge

All three flow into DjiAiDetectionsService which exposes:

  • detections$ — RxJS stream of the latest frame per drone.
  • currentDetections() — synchronous snapshot for rAF-driven overlay renderers (the drone-stream tile’s overlay uses this to avoid tearing).
  • History is capped at 500 frames so a 30 Hz stream over 5 minutes doesn’t blow the heap.

The drone-stream tile’s overlay subscribes and renders each detection as a bounding box with class label + confidence chip. See stream tile overlays.

Auto-flagging (the honest story)

Watchlist-driven auto-flagging (detect X → drop flag / fire alert) is not currently wired — the underlying plumbing exists but the copilot_alerts listener specifically filters for type='cas' right now. The closest behaviour today is the CAS agent: collision-avoidance logic writes alerts when aircraft separation drops below configured thresholds. Other detection-driven auto-flags are planned.

Thresholding

Two places filter detection confidence:

  1. Per-drone AI tab in the drone settings drawer — slider 0-1, default 0.3. Boxes below this threshold don’t render.
  2. Model confidence — the agent’s own YOLO pre-filter. Not exposed in the UI, set in the agent deployment.

Class filtering (e.g. “only show persons”) isn’t exposed in the UI. You get every class the model produces, coloured per category.