# 13 — Real-Broker Integration

> **Status (as of 2026-05-09): ALL 5 BROKERS ENABLED.**
>
> All broker adapters are fully implemented and enabled in production. Users can configure their accounts at `/settings/brokers`.

## Current Broker Status

| Broker | Auth Type | Enabled | Live Trading | Asset Classes | Notes |
|--------|-----------|---------|--------------|---------------|-------|
| **Alpaca** | API Key | ✅ | ✅ | stock, etf, crypto | Commission-free, $100k paper balance |
| **Binance** | API Key | ✅ | ✅ | crypto | Spot + futures, testnet for paper |
| **Interactive Brokers** | Session | ✅ | ✅ | stock, etf, options, futures, forex, bonds | Requires IBKR Gateway running locally |
| **Pepperstone (cTrader)** | OAuth 2.0 | ✅ | ✅ | forex, cfd, index, commodity, crypto | 30-day token, auto-refresh |
| **Charles Schwab** | OAuth 2.0 | ✅ | ✅ | equity, etf, options | ⚠️ 7-day token limit, weekly re-auth required |

## Implementation Files

| Broker | Adapter | Lines | Migration |
|--------|---------|-------|-----------|
| Alpaca | `packages/be/src/brokers/alpaca.ts` | 276 | 033 |
| Binance | `packages/be/src/brokers/binance.ts` | 800+ | 082 |
| IBKR | `packages/be/src/brokers/ibkr.ts` | 704 | 082 |
| Pepperstone | `packages/be/src/brokers/pepperstone.ts` | 543 | 189 |
| Schwab | `packages/be/src/brokers/schwab.ts` | 527 | 190 |

## User Setup Requirements

| Broker | What Users Need |
|--------|-----------------|
| **Alpaca** | API Key + Secret from alpaca.markets |
| **Binance** | API Key + Secret (testnet credentials for paper trading) |
| **IBKR** | Username + Password + IBKR Gateway running on localhost:5000 |
| **Pepperstone** | Click "Connect" → OAuth authorize via cTrader |
| **Schwab** | Click "Connect" → OAuth authorize (must re-auth every 7 days) |

## Environment Variables Required

```bash
# OAuth Redirect Base URL
OAUTH_REDIRECT_BASE_URL=https://predict.agencio.cloud

# Credential encryption (all brokers)
CREDENTIALS_ENCRYPTION_KEY=xxx  # AES-256-GCM key for storing API keys/tokens
```

## BYOK OAuth (User-Provided Credentials)

**Pepperstone and Schwab require users to register their own OAuth applications.** There are no platform-level OAuth credentials - each user must:

### Pepperstone (cTrader)
1. Go to https://openapi.ctrader.com
2. Create a developer account
3. Register a new application
4. Set redirect URI to: `https://predict.agencio.cloud/api/predict/v1/user/brokers/pepperstone/oauth/callback`
5. Copy the `client_id` and `client_secret`
6. Enter them in `/settings/brokers` before connecting

### Charles Schwab
1. Go to https://developer.schwab.com
2. Create a developer account
3. Register a new application
4. Set redirect URI to: `https://predict.agencio.cloud/api/predict/v1/user/brokers/schwab/oauth/callback`
5. Copy the `client_id` and `client_secret`
6. Enter them in `/settings/brokers` before connecting
7. **Important:** You must re-authenticate every 7 days (Schwab security policy)

---

> **Default posture:** Every user starts in `executionMode='mock'`; paper-sandbox is the promotion path for algorithms. **Live execution is gated behind an admin-granted per-user role** (`auth.users.live_trading_enabled`, migration 036) and requires MFA confirmation.
>
> **Read this doc for:** the broker-layer technical design (BrokerAdapter interface, 4-gate preflight, idempotency keys, credential encryption, audit log) and the admin-gated live-execution flow.
>
> **Configuration split:** Global (admin-controlled, platform-wide) + Per-User (account credentials & risk limits).

---

## Goal

Move from "trades are stored in `trading.mock_portfolios`" to "trades hit a real broker API and the resulting fills/positions/balances are reflected in the user's portfolio." The algorithm builder's `mode='live'` becomes meaningful.

---

## Two-Layer Configuration

### Global Settings (admin only)

Lives at `/admin/brokers`. Controls **which brokers are available platform-wide** and the platform-level rules.

| Setting | Example value | Why global |
|---|---|---|
| Enabled brokers | `[alpaca, ibkr, binance]` | Platform owner decides which adapters are vetted/maintained |
| Per-broker fee schedule overrides | `{ alpaca: { equity_bps: 0 }, ibkr: { equity_bps: 0.5 } }` | Used by backtest for accurate cost simulation |
| Per-broker rate limits | `{ alpaca: 200rps }` | Protects the platform from per-user runaway loops |
| Per-broker compliance regions | `{ alpaca: ['US'], ibkr: ['*'] }` | Hide brokers from users in unsupported jurisdictions |
| Sandbox/paper endpoints | `{ alpaca_paper: 'https://paper-api.alpaca.markets' }` | All users share these |
| Platform-level kill switch | `live_trading_globally_enabled: bool` | Big red button — admin can halt all live trading platform-wide |
| Maximum order notional | `100000` (USD) | Defense against algorithm bugs |
| Maximum daily volume per user | `1000000` (USD) | Same |

Stored in a new table `predict.broker_configs` (admin-writable only via RBAC).

### Per-User Settings

Lives at `/settings/brokers`. Users configure **their own broker credentials and personal risk caps**.

| Setting | Example | Why per-user |
|---|---|---|
| Broker selection | `alpaca` | Each user picks their broker |
| API key + secret | `(encrypted)` | Belongs to the user's broker account |
| Account ID | `(broker-issued)` | Their account |
| Default mode | `paper` / `live` | User chooses readiness |
| Personal max daily loss | `500` (USD) | Stricter than platform default |
| Personal max position size | `10000` (USD) | Stricter than platform default |
| Trading hours | `[09:30-16:00 ET]` | User's preferred window |
| Asset class allowlist | `[stocks, etfs, crypto]` | What this account is approved for |
| MFA confirmation for live escalation | `required` | Confirm before paper→live |

Stored in a new table `predict.user_broker_credentials` (encrypted via the existing `CREDENTIALS_ENCRYPTION_KEY`).

**Important security invariants:**
- API keys never leave the backend. Frontend only sees `hasKey: true` and last 4 chars.
- The `user_broker_credentials` table has RLS enforcing `user_id = current_user_id()`.
- Live mode requires the user to have completed an in-app risk acknowledgement form.
- The effective risk limit is `min(global, user)` — user can be stricter than global, never more permissive.

---

## BrokerAdapter Interface

This interface is what every broker implementation must satisfy. The algorithm executor calls this; broker-specific quirks are hidden.

```ts
export interface BrokerAdapter {
  readonly id: string;                           // 'alpaca' | 'ibkr' | 'binance'
  readonly mode: 'paper' | 'live';

  // Connection
  authenticate(creds: UserBrokerCredentials): Promise<{ ok: boolean; accountId: string }>;
  ping(): Promise<{ ok: boolean; latencyMs: number }>;

  // Account state
  getAccount(): Promise<BrokerAccount>;          // cash, equity, buying_power, marginUsed
  getPositions(): Promise<BrokerPosition[]>;
  getOpenOrders(): Promise<BrokerOrder[]>;

  // Order placement (returns the broker's order id)
  submitOrder(order: BrokerOrderRequest): Promise<BrokerOrder>;
  cancelOrder(orderId: string): Promise<void>;
  modifyOrder(orderId: string, changes: Partial<BrokerOrderRequest>): Promise<BrokerOrder>;

  // Realtime — adapter owns the websocket; emits events to the executor
  subscribeFills(handler: (fill: BrokerFill) => void): () => void;
  subscribePositions(handler: (pos: BrokerPosition[]) => void): () => void;

  // Closing
  flatten(symbol?: string): Promise<void>;       // close one or all positions
  closeAll(): Promise<void>;                     // panic-button hook
}
```

The algorithm executor depends only on this interface. Each adapter (`AlpacaAdapter`, `IBKRAdapter`, `BinanceAdapter`, `PepperstoneAdapter`, `SchwabAdapter`) handles its own auth, websocket plumbing, and idempotency.

---

## Phased Broker Rollout

### Phase 1 — Alpaca paper only ✅ COMPLETE
- Easiest API, free, US equities + crypto, generous rate limits
- Wires up the adapter interface end-to-end with no real money risk
- Enables testing of the algorithm builder in `mode='live'` against a real broker without dollars at stake

### Phase 2 — Alpaca live ✅ COMPLETE
- User flow: complete risk-ack form → enter live API key → MFA confirm → upgrade strategies one at a time
- Live order audit log surfacing in `/settings/brokers/audit`
- Daily P&L email summary

### Phase 3 — IBKR ✅ COMPLETE (shipped 2026-05-09)
- Multi-asset (US + international equities, options, futures, FX, bonds)
- Session-based auth via Client Portal API (username/password)
- Requires IBKR Gateway running locally (`localhost:5000`)
- **Futures multipliers hardcoded** (ES=50, NQ=20, CL=1000, GC=100, etc.) — IBKR doesn't expose via API
- Auto-keepalive every 55 seconds (sessions expire in 5 min)
- 2-second fill polling (no native websocket)

### Phase 4 — Binance ✅ COMPLETE
- Crypto spot + perpetual futures
- HMAC-SHA256 signed REST API
- Testnet for paper trading (`testnet.binancefuture.com`)
- 24/7 trading support
- Leverage and margin type configurable per symbol

### Phase 5 — Pepperstone / cTrader ✅ COMPLETE (shipped 2026-05-07)
- **OAuth 2.0 authentication** — no API keys, users click "Connect with Pepperstone" and authorize via cTrader
- Forex (70+ pairs), indices, commodities, crypto CFDs
- Demo and live accounts supported
- Automatic token refresh (30-day expiry, refreshed 48 hours before)
- PKCE + state parameter for security
- See `docs/55-pepperstone-broker-integration.md` for full details

### Phase 6 — Charles Schwab ✅ COMPLETE (shipped 2026-05-08)
- **OAuth 2.0 authentication** via Schwab Trader API
- US Stocks, ETFs, and Options trading
- Commission-free stock trades, $0.65/contract options
- **⚠️ 7-day hard refresh token limit** — users must re-authenticate weekly (Schwab security policy)
- Multiple account types supported (Individual, IRA, 401k, Trust, Joint, etc.)
- Paper and live via same API (account type determines mode)
- PKCE + state parameter for security
- Token status tracking with expiry warnings (24h before)
- See `docs/56-schwab-broker-integration.md` for full details

---

## Database Schema (Migration `033_broker_integration.sql`)

```sql
predict.broker_configs           -- id, broker_id, enabled, fee_schedule_json,
                                 -- rate_limit_rps, regions_jsonb, paper_endpoint,
                                 -- live_endpoint, max_order_notional_usd, updated_by

predict.user_broker_credentials  -- id, user_id, broker_id, encrypted_api_key,
                                 -- encrypted_api_secret, account_id, mode,
                                 -- created_at, last_used_at, status

predict.user_broker_limits       -- id, user_id, broker_id,
                                 -- max_daily_loss_usd, max_position_usd,
                                 -- max_orders_per_day, trading_hours_json,
                                 -- asset_class_allowlist
                                 -- (effective limit = LEAST(this, broker_configs))

predict.broker_orders            -- id, run_id (algorithm_runs), user_id, broker_id,
                                 -- broker_order_id, symbol, side, qty, type,
                                 -- limit_price, status, submitted_at, filled_at,
                                 -- filled_qty, avg_fill_price, fees, commission

predict.broker_fills             -- id, broker_order_id, qty, price, ts, venue

predict.broker_audit_log         -- id, user_id, broker_id, action, payload_redacted,
                                 -- result, ts (everything that hits the broker
                                 -- adapter is logged for compliance)
```

---

## How It Plugs Into the Algorithm Builder

In `/packages/be/src/algorithm-executor/`, the live executor's order placement path changes from:

```ts
// Today (mock):
await mockTrading.executeTradeAction(decision);
```

to:

```ts
// With broker integration:
const adapter = await brokerRegistry.getAdapter(strategy.brokerId, user.id);

// Pre-flight: per-strategy + per-user + global guardrail check
const allowed = await guardrails.check(decision, user, strategy);
if (!allowed.ok) { halt(allowed.reason); return; }

// Place order through the adapter
const order = await adapter.submitOrder({
  symbol: decision.symbol,
  side: decision.side,
  qty: decision.qty,
  type: decision.type,
  limit_price: decision.limitPrice,
});

// Audit trail
await db.broker_audit_log.insert({ user_id, broker_id, action: 'submit', ... });

// Wait for fill via the realtime subscription, then update algorithm_trades
```

The algorithm builder's L1–L5 stop-trading hierarchy stays exactly the same — broker integration just changes what happens at the bottom of the pipeline.

---

## Anti-Hallucination Safeguards Specific to Live Trading

The Module-11 gateway from doc 12 already protects most surfaces, but **live broker calls add new risk vectors**:

1. **No LLM in the order-placement path.** The DSL evaluator decides what to trade. The LLM monitor can only suggest pausing/reducing positions — it cannot synthesise an order.
2. **All orders pre-validated against position state from the broker, not from local cache.** Defends against "LLM thinks we have $X cash but actually have $Y."
3. **Idempotency keys on every order.** If the executor restarts mid-flight, replay doesn't double-place.
4. **Reconciliation job every 60s.** Compares local position state vs broker's `getPositions()`. Mismatch → halt all strategies for that user, alert admin.
5. **Order count rate limiter per minute, per user.** Hard cap. Even if the algorithm bug-loops, the broker isn't hammered.

---

## Ship Criteria

The broker layer is ready when:

- [ ] Alpaca paper adapter passes all algorithm-builder integration tests
- [ ] User can configure Alpaca credentials → see `hasKey: true` → run a strategy in `mode='paper-broker'` → see real fills in `/settings/brokers/audit`
- [ ] Reconciliation job correctly detects an injected position discrepancy and halts the strategy
- [ ] Pulling `/admin/brokers` global kill switch halts ALL live trading within 1 second across all users
- [ ] Per-user limit (e.g., max_daily_loss_usd) actually halts the user's strategies when reached, even if global limit allows more

---

## Out of Scope (deferred)

- Smart order routing across brokers
- Cross-margin between asset classes
- Algorithmic execution algos (TWAP, VWAP, Iceberg) — phase 5
- Tax-loss harvesting automation
- Direct exchange connections (FIX protocol) — far future