# AI API Keys & Features Guide

> Complete reference for AI capabilities in Agencio Predict: Platform Keys vs Organization BYOK, feature gating by subscription tier, and usage tracking.

## Overview

Agencio Predict uses a **three-layer access model** for AI features:

1. **Platform API Keys** (global) — Environment-level keys managed by the platform operator
2. **Organization BYOK** (Bring Your Own Key) — Customer-provided keys for dedicated billing
3. **Subscription Tier Gating** — Feature access controlled by Free/Pro/Enterprise plan

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                         AI ACCESS HIERARCHY                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  Layer 1: API KEY SOURCE (who pays for tokens)                              │
│  ───────────────────────────────────────────────                             │
│  Platform Keys (env vars)  OR  Organization BYOK (encrypted in DB)          │
│  └── Same AI capabilities, different billing path                           │
│                                                                              │
│  Layer 2: SUBSCRIPTION TIER (what features you can access)                  │
│  ─────────────────────────────────────────────────────────                   │
│  Free → Pro → Enterprise                                                    │
│  └── Unlocks algorithm builder, LLM jury, autonomous AI, etc.              │
│                                                                              │
│  Layer 3: USAGE LIMITS (how much you can use)                               │
│  ────────────────────────────────────────────────                            │
│  Token limits per billing period                                            │
│  └── 10K (Free) → 500K (Pro) → 5M (Enterprise)                             │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
```

---

## Layer 1: API Key Sources — Platform vs BYOK

### Key Source Comparison Matrix

| Aspect | Platform Keys | Organization BYOK |
|--------|---------------|-------------------|
| **Configuration** | Environment variables | UI at `/settings/ai` |
| **Storage** | Server-side only | AES-256-GCM encrypted in DB |
| **Billing** | Absorbed into subscription | Direct to customer's API account |
| **Quota** | Shared platform allocation | Dedicated to organization |
| **Setup Required** | None (pre-configured) | Admin/Owner adds keys |
| **Visibility** | Never visible to users | Last 4 chars only |
| **Rotation** | Platform operator | Org admin self-service |
| **AI Features** | All features (per tier) | All features (per tier) |

### Platform API Keys (Default)

Platform keys are configured via environment variables and power all AI features:

```bash
# Primary: LLM Router (recommended - load balancing, caching, fallbacks)
LLMROUTER_BASE_URL=https://llmrouter.agencio.ai
LLMROUTER_API_KEY=sk-router-...

# Fallback: Direct Anthropic
CLAUDE_API_KEY=sk-ant-...
# or
ANTHROPIC_API_KEY=sk-ant-...

# Alternative: Amazon Bedrock (uses IAM auth)
USE_BEDROCK=true
AWS_REGION=us-east-1
```

**Provider Resolution Order:**
1. LLM Router (if `LLMROUTER_BASE_URL` + `LLMROUTER_API_KEY` set)
2. Anthropic Direct (if `CLAUDE_API_KEY` or `ANTHROPIC_API_KEY` set)
3. Amazon Bedrock (if `USE_BEDROCK=true`)
4. Error — no provider configured

### Organization BYOK (Optional)

Organizations can optionally provide their own API keys for:
- **Direct billing** — Charges go to customer's OpenAI/Anthropic account
- **Dedicated quota** — Not shared with other platform users
- **Enterprise compliance** — Some orgs require using their own API accounts

**Who Can Manage BYOK Keys:**

| Role | Can Use Keys | Can View Keys | Can Edit/Delete Keys |
|------|-------------|---------------|----------------------|
| Owner | ✅ | ✅ (masked) | ✅ |
| Admin | ✅ | ✅ (masked) | ✅ |
| Member | ✅ | ❌ | ❌ |
| Viewer | ✅ | ❌ | ❌ |

**Supported Providers:**
- OpenAI (`sk-...`)
- Anthropic (`sk-ant-...`)

**Key Validation:** Keys are validated before storage via test API calls:
- OpenAI: `GET /v1/models`
- Anthropic: `POST /v1/messages` (minimal test)

---

## Layer 2: Subscription Tier Feature Matrix

AI features are gated by subscription tier, **regardless of key source** (platform or BYOK).

### AI Feature Access by Tier

| AI Feature | Free | Pro | Enterprise | Description |
|------------|:----:|:---:|:----------:|-------------|
| **Console AI Insights** | ✅ | ✅ | ✅ | AI-generated market insights in terminal |
| **Predictions (AI Engine)** | ✅ | ✅ | ✅ | Signal-based predictions with confidence |
| **Dashboard AI Summaries** | ✅ | ✅ | ✅ | AI-synthesized market overview |
| **All-Seeing Eye** | ✅ | ✅ | ✅ | Unified signal aggregation |
| **Black Swan Detection** | ❌ | ✅ | ✅ | Anomaly and crisis detection alerts |
| **Paper Trading** | ✅ | ✅ | ✅ | Simulated trading execution |
| **Algorithm Builder** | ❌ | ❌ | ✅ | Create algorithms from natural language |
| **AI Modifications** | ❌ | ❌ | ✅ | LLM-proposed strategy improvements |
| **LLM Jury System** | ❌ | ❌ | ✅ | Proposer/Critic/Judge for modifications |
| **Live Trading** | ❌ | ✅ (MFA) | ✅ | Real broker execution (Alpaca) |
| **Autonomous AI Trading** | ❌ | ❌ | ✅ | Fully autonomous AI execution |
| **Custom AI Models** | ❌ | ❌ | ✅ | Choose preferred AI model |
| **Realtime Ticks** | ❌ | ❌ | ✅ | Sub-minute price data |
| **EOD Critic** | ❌ | ❌ | ✅ | Daily AI review of algorithm performance |
| **Marketing Predictions** | ❌ | ✅ | ✅ | Campaign performance forecasting |
| **Brief System** | ❌ | ✅ | ✅ | AI-generated daily decision briefs |

### Token Limits by Tier

| Tier | AI Tokens/Month | Predictions/Month | API Calls/Month |
|------|-----------------|-------------------|-----------------|
| **Free** | 10,000 | 100 | 1,000 |
| **Pro** | 500,000 | 5,000 | 50,000 |
| **Enterprise** | 5,000,000 | Unlimited | Unlimited |

### Feature Details

#### Algorithm Builder (Enterprise)

The AI-powered algorithm builder enables:
- **NL-to-DSL Translation** — Convert natural language to trading rules
- **40 Signal Primitives** — Including 6 bond/yield and 2 human-vs-automation
- **90-Day Backtest** — Cross-validation on any proposed strategy
- **9 Anti-Hallucination Defenses** — Zod validation, primitive whitelist, cite-or-refuse

#### LLM Jury System (Enterprise)

Three-agent adversarial review:
1. **Proposer** (Sonnet, t=0.3) — Proposes modifications
2. **Critic** (Sonnet, t=0.0) — Finds flaws, cites evidence
3. **Judge** (Haiku, t=0.0) — Merges decisions mechanically

If `min(proposer_confidence, critic_confidence) < 0.85`, the system forces human escalation.

#### Autonomous AI Trading (Enterprise)

Full autonomous execution with institutional-grade defenses:
- **Slippage-aware sizing** — Caps orders where expected slippage > edge/3
- **Correlation haircut** — Shrinks positions when strategies correlate
- **EOD critic** — Daily Sonnet sign-off or escalation
- **L1-L5 guardrails** — Max DD, daily loss, leverage, position size, panic button

---

## Layer 3: Usage Tracking & Metering

All AI calls are tracked for billing and observability.

### What Gets Logged

| Field | Description |
|-------|-------------|
| `user_id` | User who initiated the request |
| `organization_id` | Organization context |
| `model` | Model used (e.g., `claude-sonnet-4.5`) |
| `provider` | Key source (`agencio`, `openai`, `anthropic`) |
| `prompt_tokens` | Input tokens consumed |
| `completion_tokens` | Output tokens generated |
| `estimated_cost_cents` | Calculated cost based on model rates |
| `latency_ms` | Response time |
| `endpoint` | API endpoint that triggered the call |

### Cost Calculation Rates

| Model | Input $/1M | Output $/1M | Use Case |
|-------|-----------|-------------|----------|
| claude-sonnet-4 | $3.00 | $15.00 | Algorithm critique, jury, translate |
| claude-haiku-4.5 | $1.00 | $5.00 | Jury judge, RAG, sentiment |
| claude-opus-4.5 | $15.00 | $75.00 | Complex reasoning (on request) |
| gpt-4o | $1.50 | $7.50 | If customer BYOK uses OpenAI |
| gpt-4o-mini | $0.075 | $0.30 | Lightweight OpenAI tasks |
| voyage-3-lite | $0.02 | — | Embeddings |

### Usage Dashboard

Organization admins can view AI usage at `/settings/ai-usage`:
- Total tokens & API calls (period)
- Estimated cost
- Usage by model (chart)
- Daily trend (chart)
- Cost breakdown table

---

## BYOK Setup Guide

### Step 1: Navigate to AI Settings

Go to `/settings` → **AI Settings** tab (requires Admin or Owner role).

### Step 2: Select Provider

Choose `custom` to use your own API keys instead of platform keys.

### Step 3: Add API Keys

Enter your API keys:
- **OpenAI API Key** — Format: `sk-...`
- **Anthropic API Key** — Format: `sk-ant-...`

Keys are validated before storage. Invalid keys will show an error.

### Step 4: Choose Model Preference

Select a preferred model or leave as `auto`:
- `auto` — System chooses based on task
- `claude-sonnet-4.5` — Balanced performance
- `claude-haiku-4.5` — Fast, cost-effective
- `gpt-4o` — If using OpenAI BYOK

### Step 5: Verify

After saving, your organization's AI calls will be billed to your API account.

---

## API Endpoints

### AI Settings Management

| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET | `/api/predict/v1/user/ai-settings` | Get effective AI settings | `requireAuth` |
| PUT | `/api/predict/v1/user/ai-settings` | Update settings (keys, model) | Admin/Owner |
| DELETE | `/api/predict/v1/user/ai-settings/keys/:type` | Delete API key | Admin/Owner |
| POST | `/api/predict/v1/user/ai-settings/validate` | Validate an API key | Admin/Owner |

### AI Usage

| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET | `/api/predict/v1/user/ai-usage?days=30` | Get usage statistics | `requireAuth` |

---

## When to Use BYOK vs Platform Keys

### Use Platform Keys (Default) When:
- You're on a standard subscription plan
- You don't need dedicated API quota
- You prefer simpler setup and management
- You want usage included in your subscription

### Use Organization BYOK When:
- Your company requires using its own API accounts (compliance)
- You need quota isolation from other platform users
- You want direct billing visibility with OpenAI/Anthropic
- You have volume discounts with a provider
- You're hitting platform token limits and need more capacity

---

## Security Considerations

1. **Encryption at Rest** — BYOK keys encrypted with AES-256-GCM
2. **Server-Side Only** — Keys never sent to frontend; only last 4 chars visible
3. **Validation Before Storage** — Keys tested before saving
4. **Access Logging** — All key access logged for audit
5. **Role-Based Access** — Only Admin/Owner can manage keys
6. **Rotation Support** — Keys can be updated without service interruption

---

## Troubleshooting

### "No LLM provider configured"
Ensure at least one of:
- `LLMROUTER_BASE_URL` + `LLMROUTER_API_KEY` (platform)
- `CLAUDE_API_KEY` or `ANTHROPIC_API_KEY` (platform)
- Organization BYOK key (in `/settings/ai`)

### "Usage limit exceeded" (402 Error)
User has exceeded plan's AI token limit. Options:
1. Upgrade to a higher tier
2. Wait for billing period reset
3. Add BYOK keys to bypass platform quotas

### "Feature requires Enterprise subscription"
The requested AI feature (algorithm builder, LLM jury, etc.) is gated to Enterprise tier. Upgrade your subscription at `/settings/billing`.

### BYOK key validation failing
1. Check key format (OpenAI: `sk-...`, Anthropic: `sk-ant-...`)
2. Ensure key has not been revoked
3. Verify billing is enabled on your provider account
4. Check for API access restrictions on the key

---

## Key Files Reference

| File | Purpose |
|------|---------|
| `packages/be/src/algorithms/llm/client.ts` | LLM provider routing, model mapping |
| `packages/be/src/user/ai-settings-service.ts` | BYOK encryption, org-level settings |
| `packages/be/src/billing/services/feature-gate.ts` | Subscription tier feature checks |
| `packages/be/src/billing/services/metering-service.ts` | Usage tracking and limits |
| `db/migrations/021_ai_settings.sql` | AI settings schema |
| `db/migrations/105_update_plan_features.sql` | Tier feature definitions |

---

## Related Documentation

- [31-ai-llm-system.md](./31-ai-llm-system.md) — Complete AI/LLM infrastructure details
- [25-data-feed-llm-opex.md](./25-data-feed-llm-opex.md) — Cost projections and OPEX models
- [12-algorithm-builder.md](./12-algorithm-builder.md) — AI algorithm builder details
- [BILLING.md](./BILLING.md) — Billing and subscription system