# SILO Cortex Integration

**Version:** 2.0.0
**Date:** 2026-05-08
**Status:** Production Ready

---

## Overview

Agencio Predict integrates with [SILO Cortex](https://docs.silo.red) for AI agent behavioral monitoring. SILO provides real-time Trust Deficit Score (TDS) calculation, prompt injection detection, and anomaly monitoring for all LLM interactions within Predict.

**Key Benefits:**
- Real-time prompt injection detection
- Trust-based trading restrictions
- Anomaly detection for unusual trading patterns
- Black swan event correlation
- LLM hallucination detection

**Integration is Optional:** All SILO features are opt-in. Predict operates normally without SILO - there are no hard dependencies.

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                         Agencio Predict                                      │
│                                                                              │
│  ┌───────────────┐  ┌───────────────┐  ┌───────────────┐                   │
│  │   Assistant   │  │ Algo Builder  │  │    Trades     │                   │
│  │   Service     │  │   Service     │  │   Service     │                   │
│  └───────┬───────┘  └───────┬───────┘  └───────┬───────┘                   │
│          │                  │                  │                            │
│          └──────────────────┴──────────────────┘                            │
│                             │                                               │
│                    ┌────────▼────────┐                                      │
│                    │   SILO Client   │  packages/be/src/lib/silo/           │
│                    │  (Optional)     │                                      │
│                    └────────┬────────┘                                      │
│                             │                                               │
│     Fire-and-Forget ────────┼──────── Blocking (waits for TDS)             │
│     emitAssistantTurn()     │         sendAssistantTurn()                   │
│     emitTradeExecute()      │         sendTradeExecute()                    │
│                             │                                               │
└─────────────────────────────┼───────────────────────────────────────────────┘
                              │ HTTPS POST + HMAC-SHA256
                              ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                           SILO Cortex                                        │
│                                                                              │
│  ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐         │
│  │ Webhook Handler │ →  │   Correlator    │ →  │   TDS Engine    │         │
│  │                 │    │                 │    │                 │         │
│  └─────────────────┘    └─────────────────┘    └────────┬────────┘         │
│                                                         │                   │
│                              ┌───────────────────────────┘                   │
│                              ▼                                               │
│                    ┌─────────────────────┐                                  │
│                    │   TDS Feedback      │                                  │
│                    │   Response          │                                  │
│                    └─────────────────────┘                                  │
└─────────────────────────────────────────────────────────────────────────────┘
```

---

## Quick Start

### 1. Admin Enrollment

Super admins enroll Predict with SILO Cortex via the admin API:

```bash
# Enroll with auto-generated secret
curl -X POST https://predict.example.com/api/predict/v1/admin/silo/config \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "webhookUrl": "https://cortex.example.com/api/v1/predict/webhook",
    "generateSecret": true,
    "label": "Production Predict"
  }'
```

**Response:**
```json
{
  "success": true,
  "message": "Successfully enrolled with SILO Cortex (latency: 45ms)",
  "generatedSecret": "abc123def456...",
  "secretNote": "Save this secret securely. Configure as PREDICT_WEBHOOK_SECRET in Cortex."
}
```

### 2. Configure Cortex

Set the same secret in your SILO Cortex instance:

```bash
# In Cortex environment
PREDICT_WEBHOOK_SECRET=abc123def456...
```

### 3. Verify Connection

```bash
# Run diagnostics
curl -X POST https://predict.example.com/api/predict/v1/admin/silo/diagnostics \
  -H "Authorization: Bearer <admin-token>"
```

---

## SILO Client Library

The SILO client is located at `packages/be/src/lib/silo/`.

### Module Structure

```
packages/be/src/lib/silo/
├── index.ts          # Public exports
├── client.ts         # Core client (emit/send functions, WebSocket)
├── events.ts         # Event types and payloads
├── enrollment.ts     # Admin enrollment service
├── redaction.ts      # Content redaction
└── tds-signal.ts     # TDS signal weighting for algorithms
```

### Basic Usage

```typescript
import {
  emitAssistantTurn,
  sendAssistantTurn,
  hashContent,
  isSiloEnabled,
} from '@agencio-predict/be/lib/silo';

// Check if SILO is enabled (optional - functions handle this internally)
if (isSiloEnabled()) {
  // Fire-and-forget (non-blocking)
  emitAssistantTurn(userId, {
    conversation_id: conversationId,
    role: 'assistant',
    content_hash: hashContent(response.content),
    content_length: response.content.length,
    model: 'gpt-4',
    provider: 'openai',
    prompt_tokens: usage.promptTokens,
    completion_tokens: usage.completionTokens,
    latency_ms: elapsed,
  });

  // OR: Blocking (wait for TDS feedback)
  const result = await sendAssistantTurn(userId, payload);
  if (result?.recommended_action === 'block') {
    throw new Error('Request blocked by SILO');
  }
}
```

---

## Event Types

### Assistant Turn

Emit after every LLM interaction:

```typescript
emitAssistantTurn(userId, {
  conversation_id: string;
  role: 'user' | 'assistant';
  content_hash: string;          // SHA-256 hash of content
  content_length: number;
  model: string;                 // e.g., 'gpt-4', 'claude-3'
  provider: string;              // e.g., 'openai', 'anthropic'
  prompt_tokens: number;
  completion_tokens: number;
  latency_ms: number;
  injection_detected?: boolean;  // If you have local detection
  injection_patterns?: string[]; // Detected patterns
});
```

### Algorithm Translate

Emit when translating natural language to DSL:

```typescript
emitAlgorithmTranslate(userId, {
  algorithm_id: string;
  algorithm_name: string;
  natural_language_hash: string;
  generated_dsl_hash: string;
  parse_success: boolean;
  retry_count: number;
  whitelist_violations: string[];
  confidence_score: number;
});
```

### Jury Decision

Emit after LLM Jury consensus:

```typescript
emitJuryDecision(userId, {
  algorithm_id: string;
  proposer_decision: string;
  critic_decision: string;
  judge_decision: 'approve' | 'reject' | 'modify';
  confidence: number;
  override_attempted: boolean;
});
```

### Trade Execute

Emit when executing trades:

```typescript
emitTradeExecute(userId, {
  algorithm_id: string;
  trade_id: string;
  symbol: string;
  side: 'buy' | 'sell';
  quantity: number;
  price: number;
  execution_mode: 'paper' | 'live';
  order_type: 'market' | 'limit';
  idempotency_key?: string;
});
```

### Workflow Run

Emit when workflows trigger:

```typescript
emitWorkflowRun(userId, {
  workflow_id: string;
  workflow_name: string;
  trigger_type: 'schedule' | 'event' | 'manual';
  action_types: string[];
  external_destinations: string[];
  sensitive_data_detected: boolean;
});
```

### Black Swan Detected

Emit on unusual market events:

```typescript
emitBlackSwanDetected(userId, {
  detection_id: string;
  indicators: BlackSwanIndicator[];
  market_impact: 'low' | 'medium' | 'high' | 'critical';
  affected_symbols: string[];
  confidence: number;
});
```

### Hallucination Detected

Emit when LLM output validation fails:

```typescript
emitHallucinationDetected(userId, {
  conversation_id: string;
  model: string;
  provider: string;
  hallucination_type: 'factual' | 'numerical' | 'temporal' | 'entity';
  confidence: number;
  original_claim_hash: string;
  evidence_hash?: string;
});
```

### Model Registered

Emit when registering custom models:

```typescript
emitModelRegistered(userId, {
  model_id: string;
  model_type: 'pattern' | 'divergence' | 'tick' | 'sentiment';
  version: string;
  training_date: string;
  performance_metrics: Record<string, number>;
});
```

---

## Blocking vs Fire-and-Forget

### Fire-and-Forget (emit*)

Use for non-critical events where you don't need to wait for SILO's response:

```typescript
// Returns immediately, doesn't block execution
emitAssistantTurn(userId, payload);
emitWorkflowRun(userId, payload);
```

**Use cases:**
- Background monitoring
- Audit logging
- Non-critical events

### Blocking (send*)

Use when you need SILO's analysis before proceeding:

```typescript
// Waits for SILO response with TDS feedback
const result = await sendAssistantTurn(userId, payload);

if (result?.recommended_action === 'block') {
  // SILO recommends blocking this action
  throw new Error('Action blocked by security policy');
}

if (result?.tds_feedback?.threat_level === 'ISOLATE') {
  // User's TDS is high - restrict to paper trading
  await restrictUserToPaperMode(userId);
}
```

**Use cases:**
- Trade execution (especially live trades)
- High-risk algorithm generation
- Sensitive data operations

---

## TDS Signal Weighting

The `tds-signal` module provides DSL primitives for incorporating TDS into algorithm decisions:

```typescript
import {
  createTdsDslPrimitives,
  canPerformAction,
  applyTdsWeight,
} from '@agencio-predict/be/lib/silo';

// Get DSL primitives for algorithm builder
const silo = createTdsDslPrimitives(userId);

// Use in algorithm expressions
const weight = silo.tds();              // Raw TDS value (0-100)
const level = silo.threat_level();      // 'OBSERVE' | 'RESTRICT' | 'ISOLATE' | 'TERMINATE'
const isRestricted = silo.is_restricted(); // boolean

// Check if user can perform action
const { allowed, reason } = canPerformAction(userId, 'live_trade');
if (!allowed) {
  console.log(`Action blocked: ${reason}`);
}

// Apply TDS-based weight to a signal
const adjustedSignal = applyTdsWeight(baseSignal, userId);
```

### TDS Thresholds

| Level | TDS Range | Effect |
|-------|-----------|--------|
| **OBSERVE** | 0 - 14.99 | Normal operation |
| **RESTRICT** | 15 - 39.99 | Reduce signal weight, require confirmations |
| **ISOLATE** | 40 - 69.99 | Paper trading only |
| **TERMINATE** | 70+ | Account locked |

---

## Content Redaction

Before sending content to SILO, apply redaction to remove sensitive data:

```typescript
import { redactContent, containsSensitiveData } from '@agencio-predict/be/lib/silo';

// Check if content contains sensitive data
if (containsSensitiveData(userInput)) {
  console.warn('User input contains potential credentials');
}

// Redact before hashing/sending
const safeContent = await redactContent(rawContent);
const contentHash = hashContent(safeContent);
```

### What Gets Redacted

- AWS keys and secrets
- API tokens (GitHub, Slack, Stripe, Google)
- JWT tokens
- PEM private keys
- Home directory paths
- Private IP addresses
- Environment variable values

---

## Admin API Reference

### Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/predict/v1/admin/silo/config` | Get enrollment status |
| `POST` | `/api/predict/v1/admin/silo/config` | Enroll with SILO |
| `PUT` | `/api/predict/v1/admin/silo/config` | Update enrollment |
| `DELETE` | `/api/predict/v1/admin/silo/config` | Unenroll |
| `GET` | `/api/predict/v1/admin/silo/diagnostics` | Run diagnostics |
| `POST` | `/api/predict/v1/admin/silo/diagnostics` | Test connectivity |

### GET /config - Enrollment Status

```json
{
  "success": true,
  "status": {
    "configured": true,
    "enabled": true,
    "webhookUrl": "https://cortex.example.com/***",
    "websocketUrl": "wss://cortex.example.com/***",
    "label": "Production Predict",
    "lastCheck": "2026-05-08T12:00:00Z",
    "lastCheckOk": true,
    "enrolledAt": "2026-05-01T00:00:00Z",
    "enrolledBy": "admin@example.com"
  }
}
```

### POST /config - Enroll

```typescript
// Request
{
  "webhookUrl": "https://cortex.example.com/api/v1/predict/webhook",
  "webhookSecret": "your-32-char-minimum-secret",
  // OR
  "generateSecret": true,

  // Optional
  "websocketUrl": "wss://cortex.example.com/ws/predict",
  "label": "Production Predict"
}

// Response
{
  "success": true,
  "message": "Successfully enrolled with SILO Cortex (latency: 45ms)",
  "diagnostics": { ... },
  "generatedSecret": "...",  // Only if generateSecret: true
  "secretNote": "..."
}
```

### PUT /config - Update

```typescript
// Request (all fields optional)
{
  "enabled": true,
  "webhookUrl": "https://new-cortex.example.com/...",
  "webhookSecret": "new-secret",
  "websocketUrl": "wss://new-cortex.example.com/...",
  "label": "Updated Label"
}

// Response
{
  "success": true,
  "message": "SILO enrollment updated successfully"
}
```

### DELETE /config - Unenroll

```json
{
  "success": true,
  "message": "Successfully unenrolled from SILO"
}
```

### GET /diagnostics - Run Diagnostics

```json
{
  "success": true,
  "diagnostics": {
    "enabled": true,
    "config": {
      "webhookUrl": "https://cortex.example.com/***",
      "hasSecret": true,
      "websocketUrl": "wss://cortex.example.com/***"
    },
    "connectivity": {
      "ok": true,
      "latencyMs": 45,
      "lastCheck": "2026-05-08T12:00:00Z"
    }
  },
  "enrollmentStatus": { ... }
}
```

---

## WebSocket Real-Time TDS

For real-time TDS updates, connect to SILO's WebSocket:

```typescript
import {
  initializeSiloWebSocket,
  onSiloUpdate,
  isSiloWsConnected,
  disconnectSiloWebSocket,
} from '@agencio-predict/be/lib/silo';

// Initialize (automatically called if websocketUrl is configured)
await initializeSiloWebSocket('wss://cortex.example.com/ws/predict');

// Subscribe to updates
onSiloUpdate((update) => {
  if (update.type === 'tds_update') {
    const { user_id, tds_feedback } = update.payload;
    // Update local TDS cache, adjust UI, etc.
  }

  if (update.type === 'alert') {
    const { severity, message, recommended_action } = update.payload;
    // Handle alert
  }
});

// Check connection status
if (isSiloWsConnected()) {
  console.log('Connected to SILO real-time feed');
}

// Cleanup
disconnectSiloWebSocket();
```

---

## TDS Response Types

### TDS Feedback

```typescript
interface TdsFeedback {
  tds_delta: number;      // Change from previous TDS
  current_tds: number;    // Current TDS value (0-100)
  previous_tds: number;   // Previous TDS value
  threat_level: 'OBSERVE' | 'RESTRICT' | 'ISOLATE' | 'TERMINATE';
  dimensions: {
    filesystem?: number;
    network?: number;
    process?: number;
    llm_behavior?: number;
    trading?: number;
  };
  threshold_warning?: {
    next_threshold: 'RESTRICT' | 'ISOLATE' | 'TERMINATE';
    distance: number;     // Points until next threshold
  };
}
```

### Webhook Response

```typescript
interface SiloWebhookResponse {
  received: boolean;
  event_id: string;
  timestamp: string;
  tds_feedback?: TdsFeedback;
  alerts?: SiloAlert[];
  recommended_action: 'allow' | 'monitor' | 'restrict' | 'block';
}
```

---

## Configuration Reference

### Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `SILO_WEBHOOK_URL` | No* | - | SILO Cortex webhook URL |
| `SILO_WEBHOOK_SECRET` | No* | - | HMAC-SHA256 shared secret |
| `SILO_WEBHOOK_TIMEOUT_MS` | No | 5000 | Request timeout |
| `SILO_WEBHOOK_ENABLED` | No | true | Enable/disable integration |

*Required if enrolling via environment. Recommended to use admin API enrollment instead.

### Database Storage

When enrolled via admin API, configuration is stored encrypted in:

```sql
-- Platform settings table
SELECT value FROM trading.platform_settings WHERE key = 'silo_agent';
```

The webhook secret is encrypted with AES-256-GCM using `CREDENTIALS_ENCRYPTION_KEY`.

---

## Error Handling

### Graceful Degradation

All SILO functions gracefully handle failures:

```typescript
// This won't throw if SILO is unavailable
emitAssistantTurn(userId, payload);  // Logs warning, continues

// Blocking calls return null on failure
const result = await sendAssistantTurn(userId, payload);
if (!result) {
  // SILO unavailable - proceed with default behavior
  console.warn('SILO unavailable, proceeding without TDS check');
}
```

### Circuit Breaker

The client implements a circuit breaker pattern:

- **Closed**: Normal operation
- **Open**: After 3 consecutive failures, stops sending for 30s
- **Half-Open**: After 30s, tries one request to test recovery

---

## Security

### Webhook Signing

All webhooks are signed with HMAC-SHA256:

```typescript
const signature = crypto
  .createHmac('sha256', webhookSecret)
  .update(JSON.stringify(payload))
  .digest('hex');
```

### Secret Storage

- Secrets are never logged
- Database storage uses AES-256-GCM encryption
- Secrets are rotated by updating enrollment

### Content Hashing

Sensitive content is hashed before sending:

```typescript
const contentHash = hashContent(content); // SHA-256
```

---

## Troubleshooting

### SILO Not Receiving Events

1. Check enrollment status:
   ```bash
   curl -H "Authorization: Bearer $TOKEN" \
     https://predict.example.com/api/predict/v1/admin/silo/config
   ```

2. Run diagnostics:
   ```bash
   curl -X POST -H "Authorization: Bearer $TOKEN" \
     https://predict.example.com/api/predict/v1/admin/silo/diagnostics
   ```

3. Check Predict logs for SILO errors:
   ```bash
   grep "silo" /var/log/predict/*.log
   ```

### 401 Unauthorized from Cortex

1. Verify secrets match on both sides
2. Check signature calculation
3. Ensure secret is at least 32 characters

### High TDS Blocking Users

1. Check user's event history in SILO Dashboard
2. Review specific events causing TDS increase
3. Consider adjusting TDS scoring rules in Cortex

### WebSocket Not Connecting

1. Verify websocketUrl is configured
2. Check firewall allows WSS connections
3. Ensure Cortex WebSocket server is running

---

## Related Documentation

- [Security Overview](./11-security-overview.md)
- [API Specification](./04-api-specification.md)
- [SILO Cortex Documentation](https://docs.silo.red/integrations/predict)
- [SILO TDS Engine](https://docs.silo.red/tds-engine)