Admin Trading Monitor

Real-time monitoring of all trading activity across all users

🏗️ System Architecture Overview

The Admin Trading Monitor aggregates data from multiple trading subsystems to provide a unified view for troubleshooting, compliance monitoring, and performance validation.

Data Flow Architecture

flowchart TB subgraph UI["Admin Trading Monitor UI"] Dashboard["Stats Dashboard"] Tabs["7 Data Tabs"] Filters["Filter Controls"] end subgraph API["API Layer"] Monitor["/admin/trading/monitor"] RunDetail["/admin/trading/monitor/:runId"] StopRun["POST stop run"] end subgraph DataSources["Data Sources"] subgraph AlgoSystem["Algorithm Trading System"] AlgoRuns["algorithm_runs"] AlgoTrades["algorithm_trades"] AlgoKills["algorithm_kills"] AlgoStatus["algorithm_run_status"] AlgoMetrics["algorithm_live_metrics"] end subgraph MockSystem["Mock/Manual Trading System"] MockPortfolios["mock_portfolios"] MockTrades["mock_trades"] Positions["positions"] end subgraph BrokerSystem["Broker Execution"] BrokerOrders["broker_orders"] BrokerFills["broker_fills"] end subgraph BillingSystem["AI Billing"] CreditTx["ai_credit_transactions"] UserBilling["user_ai_billing"] end end UI --> API Monitor --> AlgoSystem Monitor --> MockSystem Monitor --> BrokerSystem Monitor --> BillingSystem RunDetail --> AlgoRuns StopRun --> AlgoRuns

📈 Trading Progression Flow

Tracks the full progression from internal testing through to production trading, with validation gates at each stage.

Mock → Paper → Live Progression

flowchart LR subgraph Mock["1. MOCK MODE"] direction TB M1["Simulated $100k Credits"] M2["No Broker Connection"] M3["Internal Testing"] M4["Source: USER/AI/ALGO/AUTO"] end subgraph Paper["2. PAPER MODE"] direction TB P1["Connected Broker"] P2["Real Market Prices"] P3["Simulated Execution"] P4["Latency Tracking"] end subgraph Live["3. LIVE MODE"] direction TB L1["Real Money"] L2["Real Broker Execution"] L3["Full Latency Monitoring"] L4["Fill Rate Tracking"] end Mock -->|"Validate Performance"| Paper Paper -->|"Pass Graduation Gates"| Live style Mock fill:#1a1a2e,stroke:#f59e0b,color:#fff style Paper fill:#1a1a2e,stroke:#3b82f6,color:#fff style Live fill:#1a1a2e,stroke:#ef4444,color:#fff

Mock Mode Validation

  • Test algorithm logic
  • Validate signal generation
  • Check position sizing
  • Verify guardrail triggers

Paper Mode Validation

  • Real market data execution
  • Broker API latency
  • Fill simulation accuracy
  • Slippage estimation

Live Mode Monitoring

  • Actual fill rates
  • Real slippage impact
  • P&L tracking
  • Risk limit adherence

⏱️ Broker Latency Monitoring

Tracks order lifecycle timestamps to measure execution quality across all connected brokers.

Order Lifecycle Timestamps

sequenceDiagram participant Algo as Algorithm Executor participant API as Trading Monitor API participant Broker as Broker Adapter participant Exchange as Exchange/Market Algo->>Broker: submitOrder() Note over Algo,Broker: submitted_at = NOW() Broker->>Exchange: Place Order Exchange-->>Broker: Order Acknowledged Note over Broker,Exchange: accepted_at = NOW() Exchange-->>Broker: Fill(s) Note over Exchange: filled_at = NOW() Broker-->>Algo: OrderStatus API->>API: Calculate Latencies Note over API: accept_latency = accepted_at - submitted_at Note over API: fill_latency = filled_at - submitted_at

Latency Thresholds

Metric Good (Green) Warning (Yellow) Poor (Red)
Accept Latency <500ms 500-1000ms >1000ms
Avg Fill Latency <1000ms 1000-2000ms >2000ms
P95 Fill Latency <1000ms 1000-2000ms >2000ms
Fill Rate ≥95% 80-95% <80%

🧠 AI Decision Latency Tracking

Measures time from tick start to trade decision across each phase of AI processing. Helps identify bottlenecks in price fetching, context loading, PINN models, and rule evaluation.

Decision Pipeline Phases

sequenceDiagram participant Sched as Scheduler participant Exec as Executor participant Price as Price Service participant Context as Context Prefetch participant PINN as PINN Models participant DSL as DSL Evaluator participant Broker as Broker (Live) Note over Sched,Exec: tick_started_at = NOW() Sched->>Exec: tickRun(runId) Exec->>Price: getCurrentPrice() + getHistoricalPrices() Price-->>Exec: Prices Note over Exec,Price: prices_fetched_at = NOW() Exec->>Context: prefetchTickContext() Note over Context: Bonds, Macro, Sentiment,
Prediction Markets Context-->>Exec: Context Data Note over Exec,Context: context_prefetched_at = NOW() opt PINN Models Enabled Exec->>PINN: runSizingPrediction() + getRegime() PINN-->>Exec: Predictions Note over Exec,PINN: pinn_predictions_at = NOW() end Note over Exec: entry_eval_started_at = NOW() Exec->>DSL: evaluateBoolean(entry.when, ctx) DSL-->>Exec: shouldEnter / shouldExit Note over Exec,DSL: decision_made_at = NOW() opt Trade Executed alt Live Mode Exec->>Broker: adapter.submitOrder() Note over Exec,Broker: trade_submitted_at = NOW() Broker-->>Exec: Fill Confirmation Note over Exec,Broker: trade_filled_at = NOW() else Paper Mode Exec->>Exec: simulateFill() end end Note over Exec: tick_completed_at = NOW() Exec->>Exec: recordDecisionLatency()

Phase Latency Breakdown

pie title Typical Tick Latency Distribution "Price Fetch" : 15 "Context Prefetch" : 35 "PINN Predictions" : 30 "Rule Evaluation" : 5 "Broker Submit+Fill" : 15

Decision Latency Thresholds

Metric Healthy (Green) Warning (Yellow) Investigate (Red)
Total Decision Time <2000ms 2000-5000ms >5000ms
Price Fetch <200ms 200-500ms >500ms
Context Prefetch <500ms 500-1000ms >1000ms
PINN Prediction <1000ms 1000-2000ms >2000ms
Rule Evaluation <100ms 100-500ms >500ms

Common Latency Bottlenecks

flowchart TD Slow["Slow Tick Detected
(>5000ms)"] Slow --> CheckPrice{"Price Fetch
Slow?"} CheckPrice -->|Yes| PriceFix["Check:
• Yahoo API limits
• Network latency
• Symbol validation"] CheckPrice -->|No| CheckContext{"Context
Slow?"} CheckContext -->|Yes| ContextFix["Check:
• FRED rate limits
• Bond data availability
• Sentiment backlog"] CheckContext -->|No| CheckPinn{"PINN
Slow?"} CheckPinn -->|Yes| PinnFix["Check:
• TensorFlow init
• Model complexity
• Cache misses"] CheckPinn -->|No| CheckRules{"Rules
Slow?"} CheckRules -->|Yes| RulesFix["Check:
• Complex DSL
• Many symbols
• Heavy indicators"] CheckRules -->|No| CheckBroker{"Broker
Slow?"} CheckBroker -->|Yes| BrokerFix["Check:
• Connection issues
• Order queue
• Market hours"] style Slow fill:#ef4444,stroke:#ef4444,color:#fff style PriceFix fill:#22c55e,stroke:#22c55e,color:#fff style ContextFix fill:#22c55e,stroke:#22c55e,color:#fff style PinnFix fill:#22c55e,stroke:#22c55e,color:#fff style RulesFix fill:#22c55e,stroke:#22c55e,color:#fff style BrokerFix fill:#22c55e,stroke:#22c55e,color:#fff

🏷️ Trade Source Classification

Trades are classified by their origin to understand the mix of manual vs automated activity.

Source Classification Logic

flowchart TD Trade["Incoming Trade"] Trade --> CheckSignal{"Has signal_id?"} CheckSignal -->|Yes| CheckConfidence{"signal_confidence >= 0.85?"} CheckSignal -->|No| CheckAlgo{"Has algorithms_used?"} CheckConfidence -->|Yes| Auto["AUTONOMOUS
Full AI control"] CheckConfidence -->|No| AI["AI_SIGNAL
AI-assisted"] CheckAlgo -->|Yes| Algo["ALGORITHM
Rule-based"] CheckAlgo -->|No| User["USER
Manual trade"] style Auto fill:#ef4444,stroke:#ef4444,color:#fff style AI fill:#3b82f6,stroke:#3b82f6,color:#fff style Algo fill:#a855f7,stroke:#a855f7,color:#fff style User fill:#6b7280,stroke:#6b7280,color:#fff
AUTO - Fully autonomous (confidence ≥85%)
AI - AI signal-assisted
ALGO - Algorithm/rule-based
USER - Manual trades

🛑 Admin Stop Algorithm Flow

Super admins can force-stop any running algorithm, which flattens all positions and records an audit trail.

Force Stop Sequence

sequenceDiagram participant Admin as Super Admin participant UI as Trading Monitor UI participant API as /admin/trading/monitor/:runId participant Executor as Algorithm Executor participant Broker as Broker Adapter participant DB as Database Admin->>UI: Click "Stop" button UI->>UI: Confirm dialog Admin->>UI: Confirm stop UI->>API: POST /admin/trading/monitor/:runId API->>API: requireSuperAdmin() API->>Executor: stopRun(userId, runId) alt Paper Mode Executor->>Executor: flattenAllPositions() Executor->>DB: Record closing trades at market else Live Mode Executor->>Broker: closeAll() Broker-->>Executor: Positions flattened end Executor->>DB: Update run ended_at Executor->>DB: Delete algorithm_run_status API->>DB: INSERT algorithm_kills
(triggered_by='admin') API-->>UI: { success: true } UI->>UI: Refresh monitor data

🗄️ Database Tables

Tables Used by Trading Monitor

Schema.Table Purpose Key Fields
predict.algorithm_runs Active/completed algorithm runs mode, started_at, ended_at, starting_equity
predict.algorithm_trades Trades from DSL algorithms symbol, side, executed_qty, executed_price, pnl
predict.algorithm_kills Guardrail stop events triggered_by, layer, reason, metric_value
predict.algorithm_run_status Real-time tick status last_tick_at, last_action, indicators_json
predict.algorithm_live_metrics Equity curve telemetry equity, daily_pnl, current_drawdown
trading.mock_portfolios User portfolios with credits total_equity, current_balance, settings
trading.mock_trades All mock/paper/live trades signal_id, signal_confidence, algorithms_used
predict.broker_orders Order timestamps for latency submitted_at, accepted_at, filled_at
billing.ai_credit_transactions LLM usage records model, prompt_tokens, completion_tokens, amount_cents

📊 Monitor Tabs

Active Runs

  • Paper + Live algorithms
  • Equity, PnL, drawdown
  • Expandable indicator details
  • Admin stop button

Algo Trades

  • DSL algorithm trades
  • Mode (Paper/Live/Backtest)
  • Slippage in bps
  • Realized PnL

Mock/Manual

  • Credits system trades
  • Source classification
  • Portfolio attribution
  • Mode tracking

Guardrail Kills

  • L1-L4 layer stops
  • Trigger source
  • Metric values
  • Kill reasons

AI Usage

  • LLM calls
  • Billing mode
  • Token counts
  • Cost tracking

Broker Latency

  • Per-broker metrics
  • Paper vs Live comparison
  • P95 latency
  • Fill rates