Platform Testing Suite

Comprehensive proof, validation, and backtest testing for Agencio Predict

3
Validation Suites
2
Backtest Suites
2
Proof Suites
50+
Individual Tests
V

Validation Testing

Validates mathematical accuracy and correctness of calculations. These are the foundation tests.

Backtest Metrics Pending

validate-backtest-metrics.ts

Tests all performance metrics against known values with hand-calculated expected results.

  • Sharpe Ratio (positive, negative, zero volatility)
  • Sortino Ratio (downside deviation only)
  • Calmar Ratio (return / max drawdown)
  • Max Drawdown (peak-to-trough calculation)
  • Value at Risk (VaR) at 95% confidence
  • Conditional VaR (Expected Shortfall)
  • Skewness (left/right skew detection)
  • Kurtosis (fat tail detection)
  • Beta (market correlation)
  • Alpha (excess return)
  • Edge cases (empty, single, identical returns)

DSL Primitives Pending

validate-dsl-primitives.ts

Validates all 500+ DSL primitives return expected types and handle edge cases.

  • Price primitives (price, close, open, high, low, volume)
  • Technical indicators (RSI, SMA, EMA, MACD, Bollinger, ATR)
  • Sentiment primitives (news, social, fear/greed)
  • Macro indicators (VIX, DXY, treasury yields)
  • Position primitives (size, P&L, direction)
  • Sizing primitives (Kelly, half-Kelly)
  • Quant primitives (vol, cointegration, momentum)
  • Institutional (13F holdings, smart money)
  • Activist (13D/13G filings)
  • Manipulation detection primitives
  • Crisis resilience primitives

Quant Modules Pending

validate-quant-modules.ts

Tests the 12 quantitative analysis modules for institutional-grade accuracy.

  • ADF test (stationary vs non-stationary)
  • Engle-Granger cointegration
  • Spread Z-score and half-life
  • Cross-correlation at multiple lags
  • Granger causality (proper OLS AR)
  • Fama-French factor exposure (multivariate OLS)
  • Realized volatility
  • GARCH forecast
  • Time-series and 12-1 momentum
  • Chow test (structural breaks)
  • CUSUM test
  • Pearson and rolling correlation

Mathematical Accuracy Improvements (2026-05-21)

Recent expert-level audit identified and fixed three mathematical issues:

1. Granger Causality

Replaced simple average predictor with proper OLS AR coefficient estimation.

F = ((RSS_r - RSS_u) / k) / (RSS_u / (n - 2k - 1))

2. ADF P-Values

Implemented MacKinnon (1994, 2010) regression-based critical values.

c(p, n) = β_∞ + β₁/n + β₂/n²

3. Fama-French

Full multivariate OLS to avoid omitted variable bias.

β = (X'X)⁻¹ X'Y
B

Backtest Testing

Historical backtesting against known market periods to validate trading behavior and identify anomalies.

Historical Trades Pending

backtest-historical-trades.ts

Tests trading strategies against known historical periods and market regimes.

  • Buy and Hold (benchmark comparison)
  • RSI Mean Reversion strategy
  • SMA Crossover trend following
  • Volatility Breakout strategy
  • Gold/Silver Pairs trading
  • 2024 Bull Market period
  • 2023 Recovery period
  • 2022 Bear Market period
  • COVID Crash & Recovery period
  • Consistency checks (deterministic results)
  • P&L reconciliation (trades vs equity curve)
  • Stop-loss trigger validation

Anomaly Detection Pending

backtest-anomaly-detection.ts

Scans for data quality issues and suspicious patterns that could affect trading.

  • Price range validation (per asset class)
  • Negative price detection (critical)
  • Zero price detection
  • Price spike detection (>10-15% moves)
  • OHLC validity (High ≥ Low invariant)
  • Gap detection (missing trading days)
  • Stale data detection (freshness check)
  • Zero volume anomalies
  • Database trade anomalies
  • Impossible prices check
  • Future-dated trades check
  • Severity classification (critical/high/medium/low)
P

Proof Testing

End-to-end proof that platform features work as advertised. Demonstrates complete workflows.

AI Algorithm Builder Pending

proof-algorithms.ts

Proves the complete algorithm workflow from natural language to paper trading.

  • Natural Language → DSL translation
  • Simple RSI Strategy translation
  • Moving Average Crossover translation
  • Volatility Filter Strategy translation
  • Backtest execution with metrics
  • Paper trading start/stop lifecycle
  • LLM rejections audit trail
  • Backtest history tracking
  • Run history tracking

AI Stock Hunter Pending

proof-stock-hunter.ts

Proves the complete stock research workflow from scan to alerts.

  • Preset scan execution
  • Technology Leaders preset
  • Dividend Kings preset
  • Recommendation generation
  • Research report generation (8 sections)
  • Price alert creation
  • Recommendation archiving
  • Scan history tracking
  • Preset availability check (39 presets)

How to Run Tests

# Navigate to project root
cd /path/to/agencioPredict

# Run all tests (generates HTML report)
npx tsx scripts/testing/run-all-tests.ts

# Run specific suite
npx tsx scripts/testing/validate-backtest-metrics.ts
npx tsx scripts/testing/backtest-anomaly-detection.ts

# Run filtered suites (e.g., all validation tests)
npx tsx scripts/testing/run-all-tests.ts validate

Test Results History

No results yet
Date Suite Status Duration Tests Pass Fail Details
Run the test suite to populate results.
Results will be saved to scripts/testing/results/

Test Coverage Summary

Category Component Tests Coverage
Validation Backtest Metrics (Sharpe, Sortino, VaR, etc.) 15 Complete
Validation DSL Primitives (500+ primitives) 16 categories Complete
Validation Quant Modules (12 modules) 12 Complete
Backtest Historical Strategies 5 strategies × 4 periods Complete
Backtest Anomaly Detection 7 symbols × 12 checks Complete
Proof Algorithm Builder E2E 9 tests Complete
Proof Stock Hunter E2E 7 tests Complete
Planned Trading Signals TBD Planned
Planned Risk Guardrails (L1-L4) TBD Planned
Planned Fund Manager E2E TBD Planned