# Platform Integration Secrets

This document describes how API keys and secrets are managed for third-party integrations.

## Overview

Platform integration secrets (API keys, tokens, etc.) can be configured in two ways:

1. **Database storage** (preferred) - Encrypted in `predict.platform_integration_secrets`
2. **Environment variables** (fallback) - Traditional env var approach

**Database takes precedence** - If a secret exists in the database, it will be used instead of the environment variable.

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    getSecret(integrationId, keyName)        │
│                                                             │
│  1. Check in-memory cache (5-minute TTL)                   │
│     └─ If cached and not expired → return cached value     │
│                                                             │
│  2. Query database (predict.platform_integration_secrets)  │
│     └─ If found → decrypt, cache, return                   │
│                                                             │
│  3. Fall back to process.env[keyName]                      │
│     └─ Cache result, return                                │
└─────────────────────────────────────────────────────────────┘
```

## Supported Integrations

| Integration ID | Key Name | Description |
|---------------|----------|-------------|
| `finnhub` | `FINNHUB_API_KEY` | Stock data, news, sentiment, economic calendar |
| `metaculus` | `METACULUS_TOKEN` | Forecasting/prediction markets |
| `fred` | `FRED_API_KEY` | Federal Reserve economic data |
| `newsapi` | `NEWSAPI_KEY` | News headlines |
| `polygon` | `POLYGON_API_KEY` | FX OHLC, equity ticks |
| `coinglass` | `COINGLASS_API_KEY` | Crypto liquidations |
| `twitter` | `TWITTER_BEARER_TOKEN` | Social sentiment |
| `claude` | `ANTHROPIC_API_KEY` | LLM analysis |
| `openai` | `OPENAI_API_KEY` | Embeddings fallback |
| `voyage` | `VOYAGE_API_KEY` | Text embeddings |
| `stripe` | `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET` | Billing |
| `azure-graph` | `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_SENDER_EMAIL` | Email |
| `vapid` | `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY`, `VAPID_EMAIL` | Push notifications |

See `packages/be/src/integrations/platform-secrets.ts` for the complete registry.

## Database Schema

```sql
-- predict.platform_integration_secrets
CREATE TABLE predict.platform_integration_secrets (
  id TEXT PRIMARY KEY,
  integration_id TEXT NOT NULL,
  key_name TEXT NOT NULL,
  encrypted_value TEXT NOT NULL,     -- AES-256-GCM encrypted
  value_last4 TEXT,                  -- Last 4 chars for UI display
  created_by TEXT,
  updated_by TEXT,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW(),
  UNIQUE(integration_id, key_name)
);
```

## Admin UI Configuration

Super admins can manage secrets via `/admin/integrations`:

1. Navigate to `/admin/integrations`
2. Find the integration (e.g., "Finnhub")
3. Click to expand/configure
4. Enter the API key value
5. Save - the key is encrypted and stored in the database

The UI shows:
- **Source**: "db" (database) or "env" (environment variable)
- **Last 4**: Last 4 characters of the key for verification
- **Updated**: When the DB value was last updated

## Fallback Behavior

Each integration has graceful fallback when not configured:

| Integration | Behavior When Not Configured |
|------------|------------------------------|
| Finnhub | Returns empty arrays, logs warning |
| Metaculus | Returns empty arrays, logs "403 - requires METACULUS_TOKEN" |
| NewsAPI | Returns empty arrays |
| Polygon | No-op (optional enhancement) |
| CoinGecko | Works without key (public API) |
| Yahoo Finance | Works without key (public API) |

## Code Usage

### Reading a Secret

```typescript
import { getSecret } from './platform-secrets';

// Async - checks DB first, then env var
const apiKey = await getSecret('finnhub', 'FINNHUB_API_KEY');
if (!apiKey) {
  console.warn('Finnhub not configured');
  return [];
}
```

### Setting a Secret (Admin)

```typescript
import { setSecret } from './platform-secrets';

// Encrypts and stores in DB
await setSecret('finnhub', 'FINNHUB_API_KEY', 'your-api-key', userId);
```

### Checking Configuration

```typescript
import { isConfigured } from './platform-secrets';

// Returns true if ALL required keys for the integration are set
const ready = await isConfigured('finnhub');
```

## Caching

- **In-memory cache**: 5-minute TTL per key
- **Cache invalidation**: Automatic on `setSecret()` or `deleteSecret()`
- **Manual invalidation**: `invalidateCache(integrationId?)` clears specific or all cached values

## Security

- **Encryption**: AES-256-GCM via `packages/be/src/brokers/crypto.ts`
- **Key storage**: Encryption key from `BROKER_ENCRYPTION_KEY` env var
- **Access control**: Only super_admin can view/modify via admin UI
- **Audit trail**: `created_by`, `updated_by`, timestamps recorded

## Deployment

### Option 1: Database (Recommended)

1. Deploy the application
2. Log in as super_admin
3. Go to `/admin/integrations`
4. Configure each integration's API keys

### Option 2: Environment Variables

Add to EC2 ecosystem.config.js or PM2 env:

```javascript
module.exports = {
  apps: [{
    name: 'agencio-scheduler',
    env: {
      FINNHUB_API_KEY: 'your-key',
      METACULUS_TOKEN: 'your-token',
      // ... other keys
    }
  }]
};
```

### Option 3: Hybrid

- Store sensitive production keys in database
- Use env vars for development/testing

## Troubleshooting

### "403 - API requires token"

The integration requires an API key that isn't configured. Set it via:
- `/admin/integrations` (database)
- Environment variable

### "Rate limited"

The API key is valid but hitting rate limits. Consider:
- Upgrading to a paid tier
- Reducing request frequency
- Adding request caching

### Keys not taking effect

1. Check the cache hasn't expired (5 min TTL)
2. Call `invalidateCache('integration-id')` to force refresh
3. Restart the scheduler: `pm2 restart agencio-scheduler`

## Related Files

- `packages/be/src/integrations/platform-secrets.ts` - Core service
- `packages/be/src/brokers/crypto.ts` - Encryption utilities
- `apps/web/src/app/api/predict/v1/admin/integrations/route.ts` - Admin API
- `apps/web/src/app/admin/integrations/page.tsx` - Admin UI