PINN Architecture & Flow Diagrams
Physics-Informed Neural Network System - Agencio Predict
All 6 Phases Complete: Price PINN, Volatility PINN, Sizing PINN, Correlation PINN, Market Regime PINN, and Cointegration PINN are fully implemented with 150+ unit tests passing. All phases are integrated with the algorithmic trading DSL and executor.
Important: PINN uses TensorFlow.js neural networks trained via backpropagation on historical price data.
This is different from LLMs (Large Language Models like Claude) used elsewhere in the system for algorithm translation.
PINN models are trained to embed physics constraints directly into the learning process.
1. System Overview
The PINN (Physics-Informed Neural Network) predictor is a trainable neural network that embeds trading physics constraints directly into the learning process. Unlike traditional ML approaches that apply guardrails post-prediction, PINN learns constrained dynamics from the start.
PINN System Overview - All 6 Phases
flowchart TB
subgraph DataSources["Data Sources"]
PRICE["Price History\n(Yahoo/Finnhub/CoinGecko)"]
VOL["Volume Data"]
VIX["Market Indicators\n(VIX, Credit Spreads)"]
CORR["Correlation Data\nPairwise Returns"]
PAIR["Pair Price Data\nCointegrated Assets"]
end
subgraph PINNSystem["PINN System (6 Phases - All Complete)"]
P1["Phase 1: Price PINN\nPrice + velocity + liquidation\n128 unit tests"]
P2["Phase 2: Volatility PINN\nGARCH + regime switching\n34 unit tests"]
P3["Phase 3: Sizing PINN\nAlmgren-Chriss impact\n34 unit tests"]
P4["Phase 4: Correlation PINN\nDCC + Cholesky + Markowitz\n94 unit tests"]
P5["Phase 5: Regime PINN\nHMM + crisis detection\n9 files"]
P6["Phase 6: Cointegration PINN\nOU dynamics + pair trading\n35+ unit tests"]
end
subgraph Outputs["Outputs"]
PRED["AlgorithmPrediction\ndirection, confidence, signals"]
SIZE["SizingDecision\nrecommendedSizeUsd, slippage"]
WEIGHTS["PortfolioWeights\nMarkowitz optimal weights"]
REGIME["RegimePINNOutput\ncrisisWarning, recovery"]
COINT["CointegrationOutput\nhedgeRatio, zScore, signals"]
end
subgraph Integration["Integration Points"]
SIG["Signal Generator\nweight: 0.22"]
DSL["DSL Primitives\n60+ primitives"]
EXEC["Executor\nregime sizing, crisis pause"]
BACKTEST["Backtest Engine\nphysics validation"]
end
PRICE & VOL & VIX & CORR --> P1 & P2 & P3 & P4 & P5
P1 --> PRED
P2 --> P3
P3 --> SIZE
P4 --> WEIGHTS
P5 --> REGIME
PRED --> SIG
SIZE --> EXEC
WEIGHTS --> EXEC
REGIME --> EXEC
P1 & P2 & P3 & P4 & P5 --> DSL
DSL --> BACKTEST
Key Advantages
| Feature | Description | Phase |
| Physically plausible predictions | Velocity consistent with price change | Phase 1 |
| Liquidation risk awareness | Built into the model output | Phase 1 |
| Kelly sizing constraints | Soft penalties in loss function | Phase 1 |
| GARCH volatility dynamics | σ² mean-reversion, persistence | Phase 2 |
| Almgren-Chriss market impact | Slippage prediction, optimal sizing | Phase 3 |
| PSD correlation matrices | Cholesky parameterization guarantees validity | Phase 4 |
| DCC dynamics | Time-varying correlations | Phase 4 |
| Markowitz optimization | Mean-variance portfolio weights | Phase 4 |
| Regime-aware sizing | Automatic position reduction in crisis | Phase 5 |
| HMM state transitions | Row-sum=1, persistence constraints | Phase 5 |
| OU mean-reversion dynamics | dS = θ(μ - S)dt + σdW for spread | Phase 6 |
| Hedge ratio optimization | Stability constraints, rolling regression | Phase 6 |
| Pair trading signals | Entry/exit based on spread z-score | Phase 6 |
2. Academic Foundations
The PINN implementation in Agencio Predict is grounded in peer-reviewed research from both machine learning and quantitative finance domains.
PINN Research Lineage
flowchart TB
subgraph Foundations["Foundational Papers"]
R1["Raissi et al. (2019)\nPhysics-informed neural networks:\nA deep learning framework for\nsolving forward and inverse problems\nJournal of Computational Physics"]
R2["Noguer i Alonso & Camarena (2023)\nPhysics-Informed Neural Networks\n(PINNs) in Finance\nSSRN 4598180"]
end
subgraph CoreConcept["Core PINN Concept"]
C1["Loss = L_data + L_physics\n\nPhysics equations embedded\nas soft constraints in loss"]
end
subgraph Finance["Finance Applications"]
F1["Black-Scholes PDE\ndV/dt + 0.5σ²S²d²V/dS² + rS dV/dS - rV = 0"]
F2["Heston Stochastic Volatility\ndS = μS dt + √v S dW_S\ndv = κ(θ-v)dt + σ_v√v dW_v"]
F3["DCC Model (Engle 2002)\nQ_t = (1-a-b)Q̄ + a×ε_{t-1}ε'_{t-1} + b×Q_{t-1}"]
F4["Almgren-Chriss (2001)\nImpact ∝ σ × √(Q/V)"]
end
subgraph Agencio["Agencio Predict Implementation"]
A1["Phase 1: Price PINN\nKelly + liquidation constraints"]
A2["Phase 2: Volatility PINN\nGARCH constraints"]
A3["Phase 3: Sizing PINN\nAlmgren-Chriss physics"]
A4["Phase 4: Correlation PINN\nDCC + Cholesky + Markowitz"]
A5["Phase 5: Regime PINN\nHMM constraints"]
end
R1 --> C1
R2 --> F1 & F2
C1 --> A1 & A2 & A3 & A4 & A5
F1 & F2 -.-> A1 & A2
F3 -.-> A4
F4 -.-> A3
Key Academic References
| Paper | Contribution | Relevance to Implementation |
Raissi, Perdikaris & Karniadakis (2019) Journal of Computational Physics |
Original PINN framework: encode physical laws as soft constraints in the neural network loss function |
Core architecture: L = L_data + L_physics |
Noguer i Alonso & Camarena (2023) SSRN 4598180 |
Application of PINNs to financial PDEs: Black-Scholes and Heston stochastic volatility models |
Validates physics-based approach for option pricing and volatility modeling |
Engle (2002) Journal of Business & Economic Statistics |
Dynamic Conditional Correlation (DCC) model for time-varying correlations |
Phase 4: DCC residual as physics loss |
Almgren & Chriss (2001) Journal of Risk |
Optimal execution with market impact: Impact ∝ σ × √(Q/V) |
Phase 3: Market impact physics constraint |
Markowitz (1952) Journal of Finance |
Mean-variance portfolio optimization |
Phase 4: Portfolio weights with Sharpe maximization |
3. Six PINN Phases
PINN Evolution: 6 Phases (All Complete)
flowchart LR
subgraph Phase1["Phase 1"]
P1["Price PINN"]
P1S["Status: Complete"]
P1A["16 input features\n3 outputs\nKelly + liquidation constraints"]
end
subgraph Phase2["Phase 2"]
P2["Volatility PINN"]
P2S["Status: Complete"]
P2A["GARCH constraints\n3 regimes: low/normal/high\nMarkov transitions"]
end
subgraph Phase3["Phase 3"]
P3["Sizing PINN"]
P3S["Status: Complete"]
P3A["Almgren-Chriss impact\nRegime-aware sizing\n34 unit tests"]
end
subgraph Phase4["Phase 4"]
P4["Correlation PINN"]
P4S["Status: Complete"]
P4A["DCC constraints\nCholesky decomposition\n94 unit tests"]
end
subgraph Phase5["Phase 5"]
P5["Regime PINN"]
P5S["Status: Complete"]
P5A["HMM constraints\n5 market regimes\nCrisis response"]
end
subgraph Phase6["Phase 6"]
P6["Cointegration PINN"]
P6S["Status: Complete"]
P6A["OU dynamics\nPair trading\n53 tests"]
end
P1 --> P2 --> P3 --> P4 --> P5 --> P6
| Phase | PINN Type | Physics Constraints | Tests | Migration |
| Phase 1 |
Price PINN |
Kelly sizing, liquidation boundaries, mean-reversion (Hurst) |
Complete |
244 |
| Phase 2 |
Volatility PINN |
GARCH: σ² ≥ 0, α + β < 1, Markov transitions |
34 tests |
248 |
| Phase 3 |
Sizing PINN |
Almgren-Chriss: impact ∝ σ × √(Q/V) |
34 tests |
253 |
| Phase 4 |
Correlation PINN |
DCC, Cholesky (PSD), Markowitz mean-variance |
94 tests |
254 |
| Phase 5 |
Regime PINN |
HMM: row-sum=1, diagonal-dominant, persistence |
9 files |
251 |
| Phase 6 |
Cointegration PINN |
OU: dS = θ(μ - S)dt, hedge ratio stability, signal calibration |
53 tests |
— |
4. Neural Network Architecture
Price PINN Model Architecture (Phase 1)
flowchart TB
subgraph Input["Input Layer"]
IN["Sequence Input\n[20 bars x 16 features]"]
end
subgraph Processing["Processing Layers"]
LSTM["LSTM Layer\n64 units\nreturnSequences: false"]
D1["Dense 1\n64 units, ReLU"]
DROP1["Dropout 0.1"]
D2["Dense 2 + Residual\n64 units, ReLU"]
DROP2["Dropout 0.1"]
D3["Dense 3 + Residual\n64 units, ReLU"]
DROP3["Dropout 0.1"]
D4["Dense 4 + Residual\n64 units, ReLU"]
end
subgraph OutputHeads["Output Heads"]
O1["Price Output\nSigmoid (0-1)\nprice_t+1 normalized"]
O2["Velocity Output\nTanh (-1 to 1)\ndS/dt"]
O3["Liquidation Output\nSigmoid (0-1)\nrisk probability"]
end
subgraph Final["Final Output"]
CONCAT["Concatenate\n[3 values]"]
end
IN --> LSTM --> D1 --> DROP1 --> D2 --> DROP2 --> D3 --> DROP3 --> D4
D4 --> O1 & O2 & O3
O1 & O2 & O3 --> CONCAT
Model Parameters by Phase
| Phase | Input Shape | Hidden Units | Output |
| Phase 1: Price | [20, 16] | 64 (LSTM + 4 Dense) | 3 (price, velocity, liq_risk) |
| Phase 2: Volatility | [20, 12] | 64 (LSTM + 4 Dense) | 6 (vol, regimes x3, alpha, beta) |
| Phase 3: Sizing | [10, 12] | 64 (4 Dense) | 6 (size, slippage, perm/temp impact, residual, confidence) |
| Phase 4: Correlation | [batch, 166] | 128 (4 Dense) | N(N+1)/2 Cholesky params + N weights + 1 risk |
| Phase 5: Regime | [batch, 12] | 128 (4 Dense) | 8 (5 regimes, transition, crisis, recovery) |
5. Model Creation Process
TensorFlow.js Model Building
flowchart TB
subgraph Entry["Entry Point"]
REQ["runPINNPrediction()\npredictor.ts:65"]
end
subgraph LoadOrTrain["Load or Train Decision"]
CHECK["Check cache/S3\nloadPINNModel()"]
CHECK --> EXISTS{"Model exists\nand fresh?"}
EXISTS --> |"Yes < 24h old"| USE["Use cached model"]
EXISTS --> |"No or stale"| TRAIN["Train new model"]
end
subgraph Training["Training Path"]
TRAIN --> DETECT["detectAssetClass()\nstock/crypto/forex"]
DETECT --> BUILD{"bars > 500?"}
BUILD --> |"Yes"| FULL["buildPINNModel()\nFull LSTM architecture"]
BUILD --> |"No"| SIMPLE["buildSimplePINNModel()\nFlattened MLP"]
FULL & SIMPLE --> COMPILE["compilePINNModel()\nAdam optimizer\nPhysics loss function"]
end
subgraph TrainLoop["Training Loop"]
COMPILE --> FIT["model.fit()\nepochs: 100\nbatchSize: 32"]
FIT --> EARLY{"Early stopping?\npatience: 10"}
EARLY --> |"No improvement"| STOP["Stop training"]
EARLY --> |"Improving"| FIT
end
subgraph Persist["Persistence"]
STOP --> SAVE["savePINNModel()\nS3 + metadata"]
SAVE --> CACHE["Cache in memory\nTTL: 24 hours"]
end
REQ --> CHECK
USE --> INFER["Run inference"]
CACHE --> INFER
6. Training Pipeline
Complete Training Pipeline
sequenceDiagram
participant P as Predictor
participant T as Trainer
participant F as Features
participant M as Model
participant S as S3/Cache
P->>T: trainPINNModel(symbol, bars)
T->>F: computeNormalizationParams(bars)
F-->>T: NormalizationParams
T->>F: extractFeatureSequences(bars, normParams)
F-->>T: {sequences, targets, timestamps}
Note over T: Split 80/20 train/val
T->>M: buildPINNModel() or buildSimplePINNModel()
T->>M: compilePINNModel(model, config)
loop Training Loop (max 100 epochs)
T->>M: model.fit(trainX, trainY)
M-->>T: {loss, val_loss}
alt val_loss improved
T->>T: bestValLoss = val_loss
T->>T: epochsWithoutImprovement = 0
else no improvement
T->>T: epochsWithoutImprovement++
alt epochsWithoutImprovement >= 10
T->>M: model.stopTraining = true
end
end
end
T->>T: computeLossBreakdown()
T->>T: computeValidationMetrics()
T->>S: savePINNModel(model, metadata, normParams)
T-->>P: PINNTrainResult
7. Physics Loss Functions
Physics-Informed Loss Functions by Phase
flowchart TB
subgraph Phase1Loss["Phase 1: Price PINN Loss"]
L1["L = λ_mse × MSE(price)\n+ λ_physics × MSE(velocity)\n+ λ_constraint × MSE(liq_risk)\n+ λ_kelly × ReLU(|v| - max)"]
end
subgraph Phase2Loss["Phase 2: Volatility PINN Loss"]
L2["L = λ_mse × MSE(vol)\n+ λ_garch × |σ² - (ω + α×ε² + β×σ²_{t-1})|\n+ λ_markov × CrossEntropy(regime)\n+ λ_persistence × ReLU(α + β - 0.999)"]
end
subgraph Phase3Loss["Phase 3: Sizing PINN Loss"]
L3["L = λ_mse × MSE(size)\n+ λ_almgren × |impact - σ×√(Q/V)|\n+ λ_slippage × MSE(slippage)\n+ λ_boundary × ReLU(size - maxPos)"]
end
subgraph Phase4Loss["Phase 4: Correlation PINN Loss"]
L4["L = λ_corr × MSE(Cholesky params)\n+ λ_psd × ReLU(-minEigenvalue)\n+ λ_dcc × |Q_t - (1-a-b)Q̄ - a×εε' - b×Q_{t-1}|\n+ λ_markowitz × (1/Sharpe)\n+ λ_weights × |Σw - 1|"]
end
subgraph Phase5Loss["Phase 5: Regime PINN Loss"]
L5["L = λ_regime × CrossEntropy(regime)\n+ λ_hmm × |row_sum - 1|²\n+ λ_persistence × ReLU(off_diag - diag)\n+ λ_crisis × BCE(crisis_warning)\n+ λ_smooth × |regime_t - regime_{t-1}|²"]
end
Loss Weights by Phase
| Phase | Primary Loss | Physics Losses | Default Weights |
| Phase 1 |
MSE (price) |
velocity, liquidation, Kelly |
mse: 1.0, physics: 0.1, constraint: 0.05, kelly: 0.02 |
| Phase 2 |
MSE (volatility) |
GARCH residual, Markov, persistence |
mse: 1.0, garch: 0.3, markov: 0.2, persistence: 0.1 |
| Phase 3 |
MSE (size) |
Almgren-Chriss, slippage, boundary |
mse: 1.0, almgren: 0.5, slippage: 0.2, boundary: 0.1 |
| Phase 4 |
MSE (Cholesky) |
PSD, DCC, Markowitz, weight sum |
correlation: 1.0, psd: 0.5, dcc: 0.3, markowitz: 0.2, weightSum: 0.5 |
| Phase 5 |
CrossEntropy |
HMM row-sum, persistence, crisis |
regime: 1.0, hmm: 0.3, persistence: 0.2, crisis: 0.5 |
8. Feature Engineering
Feature Extraction Pipeline
flowchart TB
subgraph Input["Price History"]
BARS["PriceHistory[]\nopen, high, low, close, volume"]
end
subgraph Normalization["Normalization Params"]
NORM["computeNormalizationParams()\npriceMin, priceMax\nvolumeMean, volumeStd\nreturnsMean, returnsStd"]
end
subgraph Features["16 Features Extracted"]
F1["priceNormalized\n(price - min) / range"]
F2["returns5d\n5-day return scaled"]
F3["returns20d\n20-day return scaled"]
F4["volatility10d\n10-day realized vol"]
F5["volatility30d\n30-day realized vol"]
F6["rsi14\nRSI / 100"]
F7["hurstExponent\nR/S analysis"]
F8["regimeBull\none-hot"]
F9["regimeBear\none-hot"]
F10["regimeSideways\none-hot"]
F11["regimeCrisis\none-hot"]
F12["regimeRecovery\none-hot"]
F13["leverage\ncurrent / 125"]
F14["marginUsage\n0-1"]
F15["liquidationDistance\n0-1"]
F16["volumeNormalized\nvs 20d avg"]
end
subgraph Output["Feature Vector"]
VEC["PINNFeatureVector\n[16 numbers]\nfor each of 20 bars"]
end
BARS --> NORM
BARS --> F1 & F2 & F3 & F4 & F5 & F6 & F7 & F8 & F9 & F10 & F11 & F12 & F13 & F14 & F15 & F16
F1 & F2 & F3 & F4 & F5 & F6 & F7 & F8 & F9 & F10 & F11 & F12 & F13 & F14 & F15 & F16 --> VEC
9. Prediction Flow
End-to-End Prediction Flow
sequenceDiagram
participant SG as Signal Generator
participant PP as PINN Predictor
participant PS as Price Service
participant TF as TensorFlow
participant S3 as S3 Storage
SG->>PP: runPINNPrediction symbol history 1d
PP->>PP: Check bars at least 100
alt Insufficient data
PP-->>SG: null
end
PP->>S3: loadPINNModel symbol
alt Model exists and fresh
S3-->>PP: model and normParams
else Missing or stale
PP->>PP: trainPINNModel
PP->>S3: savePINNModel
end
PP->>PP: Build feature sequence last 20 bars
loop For each bar
PP->>PP: extractFeaturesForBar
PP->>PP: featuresToVector
end
PP->>TF: predict model sequence
TF-->>PP: priceT1 dSdt liquidationRisk
PP->>PP: denormalizePrice
PP->>PP: computeConfidence
PP->>PP: detectMarketRegime
PP->>PP: calculateHurstExponent
PP->>PP: buildSignals
PP-->>SG: AlgorithmPrediction with direction confidence signals
10. Correlation PINN (Phase 4)
Phase 4 Complete: 94 unit tests passing. Implements DCC (Dynamic Conditional Correlation) physics, Cholesky parameterization for guaranteed PSD matrices, and Markowitz mean-variance optimization.
Correlation PINN Architecture
flowchart TB
subgraph Input["Input Features (166 for 20 assets)"]
I1["Per-Asset Features (8 each)\nreturns5d, returns20d\nvolatility, regime (3)\nrsi14, sectorId"]
I2["Global Features (6)\nvix, marketRegime\nriskTolerance, correlationRegime\nnumAssets, rebalanceUrgency"]
end
subgraph Model["Dense Layers (4x128 + Residual)"]
DENSE["Dense + ReLU + Dropout 0.1"]
end
subgraph Outputs["Multi-Head Output"]
O1["Cholesky Params\nN(N+1)/2 values\nGuarantees PSD R = LL'"]
O2["Portfolio Weights\nSoftmax normalized\nΣw_i = 1"]
O3["Portfolio Risk\nSigmoid (0-1)\nAnnualized volatility"]
end
subgraph Loss["Physics Loss Components"]
L1["Correlation MSE\nMSE(Cholesky_true, Cholesky_pred)"]
L2["PSD Loss\nGershgorin eigenvalue bounds\nReLU(-minEig)"]
L3["DCC Residual\n|Q_t - (1-a-b)Q̄ - a×εε' - b×Q_{t-1}|"]
L4["Weight Sum\n|Σw - 1|²"]
L5["Markowitz Loss\n-Sharpe or -return/risk"]
end
I1 & I2 --> DENSE --> O1 & O2 & O3
O1 & O2 & O3 --> L1 & L2 & L3 & L4 & L5
Cholesky Parameterization
Correlation matrices must be Positive Semi-Definite (PSD). The Cholesky parameterization guarantees this by construction:
// Correlation matrix R = LL' where L is lower triangular
// L parameters: diagonal (softplus for positivity) + off-diagonal (unconstrained)
Cholesky L: Correlation R = LL':
| l11 0 0 | | 1 r12 r13 |
| l21 l22 0 | → | r12 1 r23 |
| l31 l32 l33| | r13 r23 1 |
// Diagonal of R = sum of squares of L rows = 1 (normalized)
// Off-diagonal = dot product of L rows / norms
DCC (Dynamic Conditional Correlation) Physics
// Engle (2002) DCC model embedded as physics loss:
Q_t = (1 - a - b) × Q̄ + a × ε_{t-1} × ε'_{t-1} + b × Q_{t-1}
Where:
- Q̄ = unconditional correlation matrix (long-term average)
- ε_t = standardized residuals (returns / volatility)
- a = shock reaction parameter (how fast correlations react)
- b = persistence parameter (how long shocks last)
- Constraint: a + b < 1 for stationarity
Phase 4 Files
| File | Lines | Purpose |
correlation/types.ts | 180 | Type definitions, feature constants |
correlation/cholesky-utils.ts | 250 | Cholesky decomposition, PSD validation |
correlation/correlation-features.ts | 400 | Feature extraction, normalization |
correlation/correlation-model.ts | 300 | TensorFlow.js model architecture |
correlation/correlation-loss.ts | 350 | Physics loss function |
correlation/correlation-predictor.ts | 450 | Inference wrapper |
correlation/index.ts | 50 | Module exports |
Phase 4 Test Coverage (94 tests)
| Test File | Tests | Coverage |
| cholesky-utils.test.ts | 31 | Cholesky decomposition, PSD validation, TF.js ops |
| correlation-features.test.ts | 28 | Feature extraction, normalization, sector mapping |
| correlation-model.test.ts | 23 | Model architecture, forward pass, memory |
| correlation-loss.test.ts | 12 | Physics loss, component isolation |
11. Market Regime PINN (Phase 5)
Market Regime PINN Architecture
flowchart TB
subgraph Input["12 Input Features"]
I1["returns5d, returns20d, returns60d"]
I2["volatility20d, volatilityRatio"]
I3["volumeRatio, breadth"]
I4["vix, creditSpread, yieldCurve"]
I5["priceVsSma50, priceVsSma200"]
end
subgraph Model["Dense Layers (4x128 + Residual)"]
DENSE["Dense + ReLU + Dropout"]
end
subgraph Outputs["Multi-Head Output"]
O1["regime_probs (5)\nSoftmax"]
O2["transition_prob\nSigmoid (0-1)"]
O3["crisis_warning\nSigmoid (0-1)"]
O4["recovery_signal\nSigmoid (0-1)"]
end
subgraph Loss["HMM Physics Loss"]
L1["CrossEntropy(regime)"]
L2["|row_sum - 1|^2"]
L3["ReLU(off_diag - diag)"]
L4["BCE(crisis)"]
L5["|regime[t] - regime[t-1]|^2"]
end
I1 & I2 & I3 & I4 & I5 --> DENSE --> O1 & O2 & O3 & O4
O1 & O2 & O3 & O4 --> L1 & L2 & L3 & L4 & L5
Five Market Regimes
| Regime | Description | Risk Multiplier | Effect |
| BULL | Sustained uptrend, low volatility | 1.1x | Slightly increase position size |
| BEAR | Sustained downtrend, elevated vol | 0.6x | Significant reduction |
| SIDEWAYS | Range-bound, neutral momentum | 0.9x | Slight reduction |
| CRISIS | Sharp decline, VIX spike | 0.4x | Major reduction |
| RECOVERY | Rebound from crisis | 0.7x | Cautious increase |
Crisis Response Actions
| Action | Trigger | Effect |
| Pause Entries | crisisWarning >= threshold (50%) | Block new position entries |
| Reduce Exposure | crisisWarning >= threshold | Apply regime sizing multiplier |
| Flatten All | crisisWarning >= flattenThreshold (80%) | Close ALL positions |
12. Cointegration PINN (Phase 6)
Cointegration PINN models the relationship between pairs of assets that move together in the long run (cointegrated pairs). It uses Ornstein-Uhlenbeck (OU) dynamics as physics constraints to learn mean-reverting spread behavior for pair trading strategies.
Cointegration PINN Architecture
flowchart TB
subgraph Input["28 Input Features (20 timesteps)"]
I1["Asset 1: price, return, vol, rsi, momentum, volume, zscore, trend"]
I2["Asset 2: price, return, vol, rsi, momentum, volume, zscore, trend"]
I3["Pair: spread, zscore, hedgeRatio, correlation, halflife, etc."]
end
subgraph Model["LSTM + Dense (Residual)"]
LSTM["LSTM(64)"]
DENSE["4x Dense(128) + ReLU + Dropout"]
end
subgraph Outputs["5-Head Output"]
O1["spread_zscore\nTanh × 4"]
O2["hedge_ratio\nSigmoid × 3"]
O3["entry_signal\nSigmoid (0-1)"]
O4["exit_signal\nSigmoid (0-1)"]
O5["spread_velocity\nTanh × 0.5"]
end
subgraph Loss["OU Physics Loss"]
L1["spreadMSE (0.3)"]
L2["hedgeRatioMSE (0.2)"]
L3["ouDynamics (0.25)"]
L4["stationarity (0.1)"]
L5["hedgeStability (0.1)"]
L6["signalCalibration (0.05)"]
end
I1 & I2 & I3 --> LSTM --> DENSE --> O1 & O2 & O3 & O4 & O5
O1 & O2 & O3 & O4 & O5 --> L1 & L2 & L3 & L4 & L5 & L6
Ornstein-Uhlenbeck Physics Constraints
The OU process models mean-reverting dynamics: dS = θ(μ - S)dt + σdW
| Constraint | Description | Formula |
| Mean Reversion Speed (θ) | How fast spread reverts to mean | θ ∈ [0.01, 0.5] per bar |
| Half-Life | Bars to revert halfway | ln(2) / θ |
| OU Dynamics | Velocity points toward mean | sign(v) = -sign(z) when |z| > 0.5 |
| Stationarity | Z-score stays bounded | |z| < maxZScore (default 4) |
| Hedge Stability | Hedge ratio changes slowly | |Δβ| < maxHedgeChange |
Pair Trading Signals
| Signal | Condition | Action |
| Entry (Long Spread) | z-score < -2, entry_signal > 0.6 | Buy Y, Sell β × X |
| Entry (Short Spread) | z-score > +2, entry_signal > 0.6 | Sell Y, Buy β × X |
| Exit | |z-score| < 0.5, exit_signal > 0.6 | Close both legs |
| Flat | |z-score| < 1.5, signals weak | No position |
12 DSL Primitives (Phase 6)
| Primitive | Description | Returns |
pinn_hedge_ratio(sym1, sym2) | Optimal hedge ratio β in Y = β×X | number (0.3-3.0) |
pinn_spread_zscore(sym1, sym2) | Spread z-score from mean | number (-4 to +4) |
pinn_mean_reversion_speed(sym1, sym2) | OU θ parameter | number (0.01-0.5) |
pinn_halflife(sym1, sym2) | Half-life in bars | number (2-100) |
pinn_pair_entry_signal(sym1, sym2) | Entry signal strength | number (0-1) |
pinn_pair_exit_signal(sym1, sym2) | Exit signal strength | number (0-1) |
pinn_pair_direction(sym1, sym2) | Suggested direction | 'long_spread' | 'short_spread' | 'flat' |
pinn_cointegration_confidence(sym1, sym2) | Confidence pair is cointegrated | number (0-1) |
pinn_pair_confidence(sym1, sym2) | Overall prediction confidence | number (0-1) |
pinn_is_cointegrated(sym1, sym2, thresh?) | True if cointegration confidence > threshold | boolean |
pinn_spread_in_entry_zone(sym1, sym2, thresh?) | True if |z| > threshold (default 2) | boolean |
pinn_spread_in_exit_zone(sym1, sym2, thresh?) | True if |z| < threshold (default 0.5) | boolean |
Example: Pair Trading Strategy DSL
// GLD-SLV pair trading strategy using Cointegration PINN
{
"universe": ["GLD", "SLV"],
"entry": "and(
pinn_is_cointegrated('GLD', 'SLV', 0.7),
pinn_spread_in_entry_zone('GLD', 'SLV', 2),
gte(pinn_pair_entry_signal('GLD', 'SLV'), 0.6)
)",
"exit": "or(
pinn_spread_in_exit_zone('GLD', 'SLV', 0.5),
gte(pinn_pair_exit_signal('GLD', 'SLV'), 0.7)
)",
"direction": "pinn_pair_direction('GLD', 'SLV')",
"hedgeRatio": "pinn_hedge_ratio('GLD', 'SLV')"
}
Cointegration Model Files
| File | Lines | Purpose |
cointegration/types.ts | 280 | Type definitions, constants, graduation gates |
cointegration/cointegration-features.ts | 450 | Feature extraction, OU parameter estimation |
cointegration/cointegration-model.ts | 300 | LSTM+Dense architecture, physics loss |
cointegration/cointegration-training.ts | 250 | Training pipeline, early stopping |
cointegration/cointegration-predictor.ts | 400 | Inference, caching, graduation validation |
cointegration/index.ts | 70 | Module exports |
Test Coverage
| Test File | Tests | Coverage |
| cointegration-features.test.ts | 35+ | Feature extraction, OU estimation, normalization |
| cointegration-model.test.ts | 25+ | Model architecture, forward pass, physics loss |
13. Algorithmic Trading Integration
Full Integration Verified: PINN predictions are accessible to algorithmic trading via 50+ DSL primitives, signal generator ensemble (weight 0.22), executor crisis response, and regime-based sizing.
PINN Integration with Algorithmic Trading
flowchart TB
subgraph PINNPredictions["PINN Predictions (6 Phases)"]
PRICE["Price PINN\ndirection, confidence, velocity"]
VOL["Volatility PINN\nforecast, regime, GARCH params"]
SIZE["Sizing PINN\noptimal size, slippage, impact"]
CORR["Correlation PINN\npairwise correlations, weights"]
REGIME["Regime PINN\nmarket regime, crisis warning"]
COINT["Cointegration PINN\nhedge ratio, spread z-score"]
end
subgraph Prefetch["Prefetch Layer (tick-context.ts)"]
PF["prefetchPINNData(symbols)\nCalled once per tick\nCaches all predictions"]
end
subgraph DSL["DSL Evaluator (60+ Primitives)"]
DSL1["Price: pinn_direction(), pinn_confidence()"]
DSL2["Vol: pinn_vol_forecast(), pinn_vol_regime()"]
DSL3["Size: pinn_optimal_size(), pinn_market_impact()"]
DSL4["Corr: pinn_correlation(), pinn_optimal_weight()"]
DSL5["Regime: pinn_crisis_warning(), pinn_regime_risk_multiplier()"]
DSL6["Coint: pinn_hedge_ratio(), pinn_spread_zscore()"]
end
subgraph Executor["Paper/Live Executor"]
EVAL["Evaluate Strategy DSL\nwith PINN primitives"]
CRISIS["Crisis Check\ncheckCrisisResponse()"]
SIZING["Position Sizing\nregime multiplier applied"]
TRADE["Execute Trade\nwith PINN-informed decisions"]
end
subgraph SignalGen["Signal Generator"]
ENS["Ensemble Predictor\nPINN weight: 0.22 (primary)"]
ADAPT["Adaptive Weights\nby market regime"]
end
subgraph Backtest["Backtest Engine"]
BT["Bar-by-bar evaluation\nPINN primitives available"]
VALID["Physics validation\nresidual thresholds"]
end
PRICE & VOL & SIZE & CORR & REGIME & COINT --> PF
PF --> DSL1 & DSL2 & DSL3 & DSL4 & DSL5 & DSL6
DSL1 & DSL2 & DSL3 & DSL4 & DSL5 & DSL6 --> EVAL
REGIME --> CRISIS
SIZE & REGIME --> SIZING
EVAL --> TRADE
CRISIS --> TRADE
SIZING --> TRADE
PRICE --> ENS
REGIME --> ADAPT
DSL1 & DSL2 & DSL3 & DSL4 & DSL5 --> BT
BT --> VALID
Integration Points
| Component | File | PINN Usage |
| Tick Context |
algorithms/paper/tick-context.ts |
Imports prefetchPINNData, calls at tick start to populate cache |
| Executor |
algorithms/paper/executor.ts |
Uses macroContext.pinnData for all PINN primitives, crisis checks, regime sizing |
| DSL Evaluator |
algorithms/dsl/evaluator.ts |
60+ PINN primitives wired to context functions |
| DSL Types |
algorithms/dsl/types.ts |
PINN primitive definitions (pinn category) |
| Signal Generator |
trading/services/signal-generator.ts |
PINN weight: 0.22 (primary ML predictor in ensemble) |
| Backtest Engine |
algorithms/backtest/engine.ts |
PINN primitives available during backtest evaluation |
Example: Using PINN in DSL Strategy
// Example DSL strategy using PINN primitives
{
"entry": {
"rules": [
// Use Phase 1 Price PINN for direction
{ "fn": "pinn_is_bullish", "args": ["$symbol", 0.6] },
// Use Phase 2 Volatility PINN for regime filter
{ "fn": "not", "args": [
{ "fn": "eq", "args": [
{ "fn": "pinn_vol_regime", "args": ["$symbol"] },
"high_vol"
]}
]},
// Use Phase 5 Regime PINN for crisis avoidance
{ "fn": "lt", "args": [
{ "fn": "pinn_crisis_warning" },
0.3
]}
]
},
"sizing": {
// Use Phase 3 Sizing PINN for optimal position
"fn": "pinn_optimal_size",
"args": ["$symbol", 30] // 30 bps expected edge
},
"risk_controls": {
"regimeSizingEnabled": true, // Apply Phase 5 regime multiplier
"maxPositionUsd": {
"fn": "mul",
"args": [
10000,
{ "fn": "pinn_regime_risk_multiplier" }
]
}
}
}
Executor Crisis Response Flow
// From executor.ts - Crisis check using Phase 5 Regime PINN
const regimePrediction = macroContext.pinnData?.regimePrediction ?? null;
crisisCheck = await checkCrisisResponse(state.algorithmId, regimePrediction);
if (crisisCheck.shouldPauseEntries) {
crisisPauseEntries = true;
// Block new position entries
}
if (crisisCheck.shouldFlattenAll) {
// Emergency: close all positions
await flattenAllPositions(state, macroContext, 'crisis_flatten');
}
// Regime sizing multiplier applied to all new positions
if (riskControls?.regimeSizingEnabled !== false && macroContext.pinnData?.regimePrediction) {
const regimeMultiplier = dslPinnRegimeRiskMultiplierSync(macroContext.pinnData);
// Crisis: 0.4x, Bear: 0.6x, Recovery: 0.7x, Sideways: 0.9x, Bull: 1.1x
positionSizeUsd *= regimeMultiplier;
}
13. DSL Primitives (50+)
Price PINN Primitives (Phase 1)
| Primitive | Returns | Description |
pinn_direction(symbol) | string | 'long', 'short', 'neutral' |
pinn_confidence(symbol) | number | Prediction confidence (0-1) |
pinn_predicted_return(symbol) | number | Predicted % return |
pinn_velocity(symbol) | number | Price velocity (dS/dt) |
pinn_liquidation_risk(symbol) | number | Risk probability (0-1) |
pinn_physics_residual(symbol) | number | Constraint residual (lower = better) |
pinn_is_bullish(symbol, threshold?) | boolean | True if bullish with conf > threshold |
pinn_is_bearish(symbol, threshold?) | boolean | True if bearish with conf > threshold |
pinn_accuracy(symbol) | number | Historical directional accuracy |
Volatility PINN Primitives (Phase 2)
| Primitive | Returns | Description |
pinn_vol_forecast(symbol) | number | Next-bar annualized vol % |
pinn_vol_regime(symbol) | string | 'low_vol', 'normal_vol', 'high_vol' |
pinn_vol_regime_confidence(symbol) | number | Regime confidence (0-1) |
pinn_vol_persistence(symbol) | number | GARCH persistence (α + β) |
pinn_vol_alpha(symbol) | number | GARCH α (shock reaction) |
pinn_vol_beta(symbol) | number | GARCH β (persistence) |
pinn_vol_regime_duration(symbol) | number | Expected bars until regime change |
pinn_vol_regime_shift(symbol) | number | 1 if regime changed this bar, 0 otherwise |
pinn_vol_is_low(symbol, threshold?) | boolean | True if low volatility regime |
pinn_vol_is_high(symbol, threshold?) | boolean | True if high volatility regime |
pinn_vol_regime_prob(symbol, regime) | number | Probability of specific regime |
Sizing PINN Primitives (Phase 3)
| Primitive | Returns | Description |
pinn_expected_slippage(symbol, sizeUsd) | number | Expected slippage in bps |
pinn_optimal_size(symbol, edgeBps) | number | Optimal position size in USD |
pinn_market_impact(symbol, sizeUsd) | number | Expected market impact in bps |
pinn_sizing_confidence(symbol) | number | Sizing model confidence (0-1) |
pinn_almgren_residual(symbol) | number | Almgren-Chriss physics residual |
pinn_regime_size_multiplier(symbol) | number | Regime-based size adjustment |
Correlation PINN Primitives (Phase 4)
| Primitive | Returns | Description |
pinn_correlation(symbol1, symbol2) | number | Predicted pairwise correlation (-1 to 1) |
pinn_correlation_regime(symbol) | string | 'low', 'normal', 'high' |
pinn_correlation_confidence(symbol1, symbol2) | number | Correlation prediction confidence |
pinn_optimal_weight(symbol) | number | Markowitz optimal weight (0-1) |
pinn_portfolio_risk() | number | Portfolio volatility (annualized) |
pinn_portfolio_sharpe() | number | Portfolio Sharpe ratio |
pinn_dcc_residual() | number | DCC physics residual |
pinn_markowitz_efficiency() | number | Distance from efficient frontier |
Market Regime PINN Primitives (Phase 5)
| Primitive | Returns | Description |
pinn_market_regime() | string | Current regime: bull/bear/sideways/crisis/recovery |
pinn_regime_probability(regime) | number | Probability of specific regime |
pinn_regime_confidence() | number | Classification confidence |
pinn_regime_transition_prob() | number | Probability of regime change |
pinn_regime_next_likely() | string | Most likely next regime |
pinn_regime_duration() | number | Expected bars in current regime |
pinn_crisis_warning() | number | Crisis probability (0-1) |
pinn_recovery_signal() | number | Recovery strength (0-1) |
pinn_regime_risk_multiplier() | number | Position size multiplier (0.4-1.1) |
pinn_regime_is(regime) | boolean | True if current regime matches |
14. Database Schema
PINN Database Tables
erDiagram
pinn_models {
uuid id PK
varchar symbol
varchar asset_class
text s3_path
int version
timestamptz trained_at
int training_bars
real physics_residual
real validation_sharpe
real validation_accuracy
jsonb config
boolean is_active
}
pinn_training_jobs {
uuid id PK
varchar symbol
varchar status
jsonb config
int epochs_completed
real current_loss
uuid model_id FK
text error_message
timestamptz started_at
}
pinn_predictions {
uuid id PK
uuid model_id FK
varchar symbol
varchar timeframe
real predicted_price
real current_price
varchar direction
real confidence
real velocity
real liquidation_risk
real physics_residual
varchar market_regime
}
pinn_model_performance {
uuid id PK
uuid model_id FK
timestamptz period_start
timestamptz period_end
int total_predictions
real accuracy
real avg_physics_residual
real sharpe_ratio
}
pinn_correlation_models {
uuid id PK
text s3_path
int num_assets
text[] symbols
timestamptz trained_at
real dcc_residual
real psd_violations
}
pinn_regime_history {
uuid id PK
timestamptz recorded_at
varchar regime
real[] probabilities
real crisis_warning
real recovery_signal
}
pinn_models ||--o{ pinn_training_jobs : "produces"
pinn_models ||--o{ pinn_predictions : "generates"
pinn_models ||--o{ pinn_model_performance : "tracked_by"
15. API Endpoints
Admin Endpoints
| Method | Path | Description |
| GET | /api/predict/v1/admin/pinn/models | List all models |
| GET | /api/predict/v1/admin/pinn/models/:id | Get model details |
| POST | /api/predict/v1/admin/pinn/train | Trigger training |
| DELETE | /api/predict/v1/admin/pinn/cache | Clear cache |
| GET | /api/predict/v1/admin/pinn/stats | Get statistics |
| GET | /api/predict/v1/admin/pinn/monitoring | Get monitoring data |
| GET | /api/predict/v1/admin/pinn/jobs | List training jobs |
Regime Endpoints
| Method | Path | Description |
| GET | /api/predict/v1/pinn/regime | Get current regime |
| GET | /api/predict/v1/pinn/regime/history | Get regime history |
| GET | /api/predict/v1/admin/pinn/regime/status | Model status |
| POST | /api/predict/v1/admin/pinn/regime/train | Train regime model |
16. File Reference
Core Module
| File | Lines | Purpose |
pinn/index.ts | 605 | Module exports for all 5 phases |
pinn/types.ts | 337 | Type definitions |
pinn/model.ts | 270 | TF.js model architecture + physics loss |
pinn/features.ts | 400 | Feature engineering |
pinn/trainer.ts | 330 | Training pipeline |
pinn/predictor.ts | 470 | Inference wrapper |
pinn/persistence.ts | 450 | S3 + cache |
pinn/validation.ts | 400 | Graduation gates |
pinn/dsl-wrappers.ts | 1500+ | 50+ DSL primitives |
Phase-Specific Modules
| Phase | Directory | Key Files | Tests |
| Phase 2 | pinn/volatility/ | volatility-model.ts, volatility-features.ts, regime-dynamics.ts | 34 tests |
| Phase 3 | pinn/sizing/ | sizing-model.ts, sizing-features.ts, sizing-loss.ts | 34 tests |
| Phase 4 | pinn/correlation/ | correlation-model.ts, cholesky-utils.ts, correlation-loss.ts | 94 tests |
| Phase 5 | pinn/regime/ | regime-model.ts, regime-labeling.ts, regime-predictor.ts | 9 files |
Migrations
| Migration | Purpose |
| 244_pinn_models.sql | Core PINN tables: models, jobs, predictions, performance |
| 247_pinn_ab_testing_leaderboard.sql | A/B testing infrastructure |
| 248_pinn_volatility.sql | Volatility PINN tables |
| 251_pinn_regime.sql | Regime PINN + history |
| 252_regime_sizing_and_crisis.sql | Crisis response config + actions |
| 253_pinn_sizing.sql | Sizing PINN tables |
| 254_pinn_correlation.sql | Correlation PINN tables |
Validation Thresholds (Graduation Gates)
| Metric | Threshold | Purpose |
| Physics Residual | ≤ 0.05 | Velocity prediction accuracy |
| Constraint Violations | ≤ 5% | Kelly/liquidation compliance |
| Directional Accuracy | ≥ 52% | Better than coin flip |
| Brier Score | ≤ 0.25 | Risk calibration |
| Sharpe Ratio | > 0.5 | Risk-adjusted returns |
| Max Drawdown | < 20% | Risk control |
PINN Architecture Documentation - Agencio Predict
Generated: 2026-05-19 | All 6 Phases Complete | 150+ Unit Tests
Updated with Phase 4 Correlation PINN and Algorithmic Trading Integration