AgentForge — How to Read This Dashboard

A 5-minute field guide for the CISO, the on-call engineer, and the demo-day audience.

What you're looking at (90-second version)

AgentForge is an adversarial AI security platform: a fleet of cooperating agents that continuously red-team a target LLM application, judge the results with a locked rubric, file vulnerability reports, and propose fixes — all without a human in the inner loop. The dashboard is the platform's ops center, the single screen a hospital CISO can walk up to and answer “is this thing working, and is it getting better?” It shows live agent state, attack throughput, coverage, cost burn, severity trend, and the platform's own bug inbox — six panes across one row plus a topology hero strip on top. Clicking any attack or vulnerability row opens a plain-English Incident view; the topology has an optional CLOSED LOOP overlay that exposes the V4 feedback satellites; and a top-right environment selector lets you swing the whole UI between your local fleet and the public demo.

Quick interactions cheat sheet

The Topology

What it is

A live, auto-drawn map of every agent in the platform and the messages flowing between them. Each node is a worker; each edge is a typed message channel (campaign, attack_attempt, verdict, vuln_report, patch_suggestion, observe). The view polls/v1/agents/topology every second and is rendered from the live jobs and heartbeats tables — there is no separate “diagram” file anywhere. If a node is missing, the agent is not running.

Why it's needed

A multi-agent platform without a topology view is opacity behind a REST API. A CISO is not going to read a job table. She needs to see, at a glance, whether the platform is running, where the traffic is concentrated, and where alerts are firing — and she needs to see it in the same visual idiom she sees DataDog APM or Honeycomb service maps in. The topology is also the only pane that makes the Watchdog visible as an architectural choice rather than just a database table; it's how we show that the platform watches itself.

What to expect

• A clean left-to-right flow: Orchestrator → Red Team → Target → Judge → Docs Agent → Patch Advisor.
• A curved feedback edge from Target back to Red Team, routed below the main rail — this is how partial successes drive the next mutation.
• The Watchdog sits on a parallel rail below the main flow, with dashed gray observation lines reaching up to every agent it watches. That spatial choice is the architecture: Watchdog observes the bus; it never participates in grading or attacking.
• Node colors: green pulsing = active and emitting heartbeats; amber = degraded (refusal spike, heartbeat lag, dry-well); red = alert (something needs human attention); black = offline.
• Edge animation pulses when traffic is flowing in the last 30 seconds; the small label on each edge shows the message kind and recent msg/sec.

How it meets the requirements

The Other Five Panes

The Incident view

Every row in Recent traces and Open vulnerabilities opens the same Incident view. The top of the modal is the “hero” — five plain-English fields, color-coded on their left border, that a CISO can read in ten seconds: Attacker input (cyan), Target output (amber), Issue (red), Severity (red, with the judge's confidence % alongside), and Suggested fix (green). This is the “what happened, how bad, what do I do” pane.

The full 9-column engineering trace is still there but now hidden behind a ▸ Show technical trace disclosure toggle, so the modal opens lean. Inside the trace, errors are rank-badged: the first error in a trace gets a red 1st badge and a left-border accent (“this is the root cause”); subsequent errors get a muted #N badge (“cascading from the first”). Hover the badges for the same explanation in a tooltip. Source: app/components/IncidentHero.tsx and AttackModal.tsx.

The closed-loop topology overlay

The topology header has a small toggle — DEFAULT ⇄ CLOSED LOOP — that is off by default. Flip it on and 12 V4 satellite nodes appear around the existing agent chain, connected by purple dashed feedback edges. These are the parts of the platform that make the loop actually close: adaptive_mutator, kill_chain_planner, novelty_seeder, property_fuzzer, event_ingress, reproducer, eval_promoter, eval_suite, fix_validator, cve_report_generator, regression_monitor, and pressure_gauge.

Click any satellite for a 480 px popover with four blocks — WHAT IT IS / WHY IT EXISTS / WHAT IT DOES / WHY IT MATTERS — and a CISO view ⇄ Technical view toggle at the top that persists across opens. The same content is also rendered as twelve cards on the V4 Roadmap page, alongside the architecture-as-bands diagram (GENERATE / EVALUATE / CONVERT / DOCUMENT) and a forced-on mirror of this closed-loop topology. Source: app/components/ClosedLoopToggle.tsx, app/lib/v4SatelliteDocs.ts, TopologyV3.tsx :: V4_SATELLITES.

Vulnerabilities, traces, and the threat dots

Recent traces and Open vulnerabilities are now real HTML tables with semantic headers and plain-language column labels (Time, Attack category, Result, Cost, Tokens, Models / Vulnerability ID, Severity, Attack category, What happened, Age). Values are pretty-printed — prompt_injection renders as “Prompt injection”, success as “Attack succeeded”, fail/refused as “Refused”. Click ⛶ top-right of either pane to open the same data in a 1400 px modal with an extra What happened column and natural line wrapping.

Hovering any row in either pane pulses the matching nodes on the topology — the orchestrator that scheduled it, the redteam worker, the target chat, the judge, and (for vulns) the cve report generator. Pure CSS drop-shadow pulse; no topology re-render. Source: app/lib/focusNode.ts.

On the topology itself, each TARGET node now shows a row of small colored dots, one per unique TM-0X threat that touches it. For example, supervisor has dots for TM-01, TM-04, and TM-06. This replaces the previous single-color border, which incorrectly implied each node had exactly one primary threat. Hover a dot for the category name; the legend across the bottom of the topology pane lists T1–T6.

The six TARGET tools (tool_resolve_patient, tool_get_row, tool_get_document, tool_list_overnight, tool_retrieve_evidence, tool_attach_extract) are rendered as a single collapsed tools (6) node to keep the worker fan-out legible; hover the node to see the sub-tools listed. Per-tool rows in the trace are a known gap — see docs/gaps/GAP-TRACE-TOOLS.md. A COLLAPSE_TOOLS = true flag at the top of TopologyV3.tsx flips it back to the 6-tool view if needed. (Edge polish note: TARGET internals use smoothstep edges with per-edge-kind parallel offsets and idle de-emphasis so the message flow reads more cleanly.)

The Pressure pane (V4 #11)

The strip above the main 5-pane grid is the live stress meter: attacks/min × rolling success rate. The big number is the current 5-minute pressure; the arrow is the delta vs the 60-minute baseline (▲ rising, ▼ falling, = stable); the bars are the per-TM (TM-01..06) breakdown so you can see which threat category is currently driving the load. Backed by /v1/pressure/now, polled every 5 seconds. If the backend returns a 404 (the public-demo Tailscale Funnel was cut before V4 #11), the pane shows a clear “unavailable on this environment” message rather than spinning — flip the environment selector to Local fleet to see live pressure.

Apply fix to production

Safety bar first. This workflow is queue-only — clicking the button files a row in patch_applications and enqueues an apply_patch_request bus job. There is no out-of-process executor that actually touches the running target; that is a deliberate next step gated on additional review. The feature is also disabled by default: the API process must have APPLY_TO_PROD_ENABLED=true set in its environment, otherwise both endpoints return 403 with a clear message. And the UI gates the click behind a typed confirmation phrase.

Mechanics: for any published vulnerability report (VR-NNN), the Incident view's Suggested fix section shows an Apply fix to production target button. Clicking it opens a confirmation modal that requires an operator name, a justification of at least 20 characters, and the exact typed phrase APPLY TO PROD before submit is enabled. The submit is a two-step POST: /v1/vuln-reports/{vr_id}/apply/request-token first returns a 5-minute single-use token, then /v1/vuln-reports/{vr_id}/apply takes the token plus operator and justification. Source: agentforge/routers/apply_patch.py, alembic/versions/0022_patch_applications.py, app/components/ApplyFixButton.tsx.

Environments

The top-right dropdown chooses which backend the dashboard reads from: Local fleet (your local FastAPI on :8701) or Public demo (the always-on Spark-fleet deployment at https://spark1.tailea2c1b.ts.net/api). Loopback browsers default to Local fleet on first load; once you flip to Public demo the choice persists across reloads. Some panes (notably Pressure and a few V4 satellites) haven't been deployed to the public demo yet; rather than spinning forever they render a friendly “unavailable on this environment” state.

Pane subtitles, by the way, now live inside the existing ? help icon's popover at the top of each pane — divided from the engineering help block by a line — to reclaim about 30 px of vertical room per pane. The top bar also carries a ⤴ V4 Roadmap button that links to /enhancements.

State Colors

When You Don't See Activity

The honest gotcha: this is a live demo of a live platform. If no agents are running, every node shows offline and the feed will be empty. If workers are running but no campaign has been fired, the agents will be idle / active with heartbeats but no message traffic on the edges. To wake everything up:

make -C infra demo-up && make -C infra demo-traffic

The first command brings the agent fleet online; the second fires a campaign through the bus. Give it ~10 seconds before expecting verdicts.

Where the Data Comes From

Every pane reads from the FastAPI backend on :8701, which in turn reads from a single Postgres instance on :5701. There is no second data store and no derived cache; the dashboard and the Orchestrator see the same source of truth (per OBS-07: the observability layer is the data substrate for the Orchestrator, not just a human dashboard).

For the deeper architecture story (agent inventory, trust boundaries, orchestration strategy) read ARCHITECTURE.md in the repo root. For the precise requirement IDs cited here, see docs/REQUIREMENTS.md §7.