ArgusAiTR — Architecture#

Posture#

ArgusAiTR is a non-intrusive AiTR overlay for Remote Weapon Stations (CROWS class). It taps the RWS video output, runs detection / classification / multi-object tracking / monocular ranging on commodity edge compute, and renders overlays plus slew-to-cue back to the operator. CROWS firmware is unchanged. If the AiTR pipeline drops, the operator retains full standard RWS capability.

This posture is deliberate. The CSO calls for "seamless integration with current RWS architectures" and the ability to "revert to standard operational capabilities without any performance loss." A non-intrusive overlay achieves both with the minimum integration risk and the fastest path through the 3-month post-award window. Co-located and deeper integration are planned as Part 2 follow-on and the interfaces here are designed to accept them without rewriting the core.

Data flow#

                       ┌──────────────────────────────────────┐
                       │ PerturbationSim (test/demo harness    │
                       │ injecting jitter, muzzle flash,       │
                       │ vibration, dropout)                   │
                       └──────────────┬───────────────────────┘
                                      │
 VideoSource ─► Stabilizer ─► Detector ─► Tracker ─► Ranger ─► Classifier ─► HMI
   │                                                                          │
   ├─ FileSource (PoC)                                                        │
   ├─ RTSPSource (stub)                                                       ▼
   ├─ ThermalSource (stub)                                              OperatorAction
   └─ CROWSAdapter (mock now / GVA post-award)                          (HITL)
                                                                              │
                                                                              ▼
                                                                         AuditLogger
                                                                         (JSONFileSink
                                                                          for PoC; TAK
                                                                          / DDS sinks
                                                                          post-award)

The orchestrator (src/argus_aitr/pipeline.py) runs the chain on a background thread. The HMI and other consumers read a thread-safe SharedState that the orchestrator publishes into; SSE subscribers are notified via asyncio.Queue (cross-thread via loop.call_soon_threadsafe).

Module contracts (Pydantic)#

Type	Purpose
`Frame`	One image + provenance (dataclass; not serialized — image is bytes-heavy)
`Detection`	Per-frame bbox + class + confidence (frozen)
`TrackedTarget`	Persistent track + range + velocity + threat level (mutable; ranger and classifier annotate in place)
`OperatorAction`	HITL event: confirm / dismiss / engage_request / mode_change (frozen)
`SystemHealth`	aitr_status (nominal/degraded/offline) + fps + notes

Stages#

`ingest/` — VideoSource#

VideoSource ABC with open(), next_frame() → Frame | None, close(), intrinsics property. Three implementations:

FileSource (real) — cv2.VideoCapture; loops at EOF for the demo; defaults to a coarse CROWS daylight-TV intrinsics block, override-able per clip.
RTSPSource (post-award stub) — instantiable for config introspection; open / next_frame raise NotImplementedError with a clear "post-award" message.
ThermalSource (post-award stub) — same pattern. Production wires cooled MWIR + EO/IR fusion.

`stabilize/` — Stabilizer + Perturbation sim#

IdentityStabilizer (default) — pass-through. At PoC distances detection/tracking are robust without it.
FeatureStabilizer — Shi-Tomasi corners + LK optical flow + partial-affine; best-effort. Documented as the kernel of the post-award production stabilizer (which adds low-pass trajectory smoothing, IMU fusion, bounded-search ROI).
Perturbator — test/demo harness with 4 named profiles: clean, vehicle_idle, engagement (jitter + muzzle flash + vibration), severe (engagement params + dropout). Switchable from the HMI; directly demonstrates the CSO adverse-conditions requirement.

`detect/` — Detector#

Detector ABC. Default YOLODetector wraps Ultralytics. Out of the box the PoC uses YOLOv8n on COCO weights filtered to {airplane, bird, kite} as UAS proxies, uniformly relabeled to uas. The filter and weights are constructor parameters so a UAS-fine-tuned checkpoint drops in for the post-award prototype with no code change.

`track/` — Tracker#

Tracker ABC. PoC ships IoUTracker: IoU matrix + scipy Hungarian assignment, tentative → confirmed after N hits, deleted after M misses, per-track velocity in px/s derived from bbox centers and frame timestamps. Sufficient for sparse C-UAS scenes. ByteTrack/BoT-SORT documented as the post-award upgrade for dense / occlusion-heavy scenes — same Tracker contract.

`range/` — MonocularRanger#

R = (real_size_m × focal_length_px) / bbox_size_px. class_priors table maps class_label → (size_m, prior_confidence). Confidence scaled by bbox pixel size (sub-pixel quantization dominates at small boxes) and prior tightness. Documented ±~30 % error at long range. Post-award upgrade paths: platform-motion triangulation (slewing turret parallax) + limited- duration laser ranging (CSO permits this).

`classify/` — ThreatHeuristic#

Rules-based, transparent. Per-track range history → closing speed → time-to-arrival → threat level:

Condition	Threat
No range yet, or fewer than 2 samples	`unknown`
Range rate ≥ −1 m/s (stable or receding)	`non-threat`
Closing 1–5 m/s	`unknown`
Closing ≥ 5 m/s, predicted ToA ≥ 10 s	`probable`
Closing ≥ 5 m/s, predicted ToA < 10 s	`high`

Every verdict carries a human-readable reasoning string into the audit log. A supervised threat classifier (with environmental and RWS-context features) is the post-award upgrade.

`integrate/` — CROWS adapter + telemetry sink#

CROWSAdapter ABC: send_slew_command(az, el, rate), read_telemetry, report_health.
MockCROWSAdapter (PoC) writes every command to audit/crows_commands.jsonl and serves canned telemetry — thread-safe.
GVAAdapter (post-award stub) targets the CROWS control box over Generic Vehicle Architecture (GVA) per STANAG 4754.
TelemetrySink ABC: publish(event).
JSONFileSink (PoC) — line-buffered append-only writer.
TAKTelemetrySink (post-award stub) — Cursor on Target over TAK Server Protocol so the operator's TAK client sees Argus tracks alongside other sensors.

`hmi/` — Operator console#

FastAPI app exposing:

Endpoint	Purpose
`GET /`	Operator console HTML
`GET /stream.mjpg`	MJPEG of overlay-annotated frames
`GET /events`	SSE of `{tracks, health, seq}` on every pipeline tick
`GET /state`, `GET /health`	Snapshot variants of the same payload
`GET /actions?n=`	Recent operator actions for the audit panel
`POST /op/{action}`	Record HITL action (confirm / dismiss / engage_request)
`POST /perturbation/{profile}`	Switch perturbation profile
`POST /aitr/disable`, `/enable`	Failsafe toggle (drives the "offline" banner demo)

The frontend is vanilla HTML + JS with Canvas-free overlays drawn server-side onto each frame. All DOM injection uses safe methods (createElement / textContent) — no innerHTML — even though data originates from our own backend (defense in depth).

`audit/` — AuditLogger#

AuditLogger fans structured JSON events to one or more TelemetrySinks. Per-event kinds: detection, classification, operator_action, crows_command, pipeline_event. Failing sinks never raise out of the logger; auditing must not crash the pipeline.

The audit log is the substrate for the DoD AI Ethics "Traceable" and "Governable" principles — see docs/ai-ethics.md.

Threading model#

Thread	Responsibility
`argus-pipeline`	The orchestrator loop: ingest → ... → publish to `SharedState`. CPU-bound.
asyncio loop	FastAPI request handling + SSE generators. I/O-bound.

SharedState bridges them with a threading.RLock plus asyncio.Queues registered by SSE clients. The pipeline thread pushes events into those queues via loop.call_soon_threadsafe.

Failure modes#

Failure	Behavior
Detector raises	`consecutive_errors` counter; status → degraded; banner shown in HMI
Source EOF (file ends)	Pipeline logs and emits a final degraded snapshot; loop ends
FPS drops below floor	Status → degraded with quantitative note
AiTR explicitly disabled by operator	Status → offline; raw video continues to display; HITL still gated
Audit sink raises	Logged, ignored — pipeline continues

Open System / interoperability#

Pydantic contracts at every stage boundary.
ABCs for VideoSource, Detector, Tracker, Stabilizer, CROWSAdapter, TelemetrySink — every component is swappable without changing the orchestrator.
Stubs for the post-award integration surfaces (RTSP, thermal, GVA, TAK) with explicit NotImplementedErrors carrying a "post-award" message so misuse fails loudly.
No closed proprietary formats; on-the-wire data is JSON.

See docs/integration-contract.md for the production integration plan.

ArgusAiTR — Architecture#

Posture#

Data flow#

Module contracts (Pydantic)#

Stages#

ingest/ — VideoSource#

stabilize/ — Stabilizer + Perturbation sim#

detect/ — Detector#

track/ — Tracker#

range/ — MonocularRanger#

classify/ — ThreatHeuristic#

integrate/ — CROWS adapter + telemetry sink#

hmi/ — Operator console#

audit/ — AuditLogger#