← The Store

Enrichment Orchestrator

Five-agent pipeline that writes SEO metadata for every asset.

PipelinesLiveNode.jsClaude APISupabase
5agents

What it is

The most-used custom workflow on the site: five sub-agents (auditor, blog, content, linker, metadata) that audit, draft, and cross-link content, then write it to Supabase. Token usage and cost are tracked per run.

Take it with you

The real, committed source behind this system. Copy it or download the file. Plus a portable spec of everything on this page.

usage-tracker.mjsjavascript92 lines

The per-run token + cost tracker the enrichment pipeline records usage with — self-contained and reusable on any Claude pipeline.

// ---------------------------------------------------------------------------
// Usage Tracker — track Claude API token usage and estimated cost per run
// ---------------------------------------------------------------------------

// Pricing per million tokens (USD) as of 2025-05.
const MODEL_PRICING = {
  'claude-sonnet-4-20250514': { input: 3, output: 15 },
  // Fallback for unknown models
  default: { input: 3, output: 15 },
};

export class UsageTracker {
  constructor() {
    this.calls = [];
  }

  /**
   * Record a single API call's token usage.
   * @param {{ agent: string, inputTokens: number, outputTokens: number, model: string }} usage
   */
  record({ agent, inputTokens, outputTokens, model }) {
    this.calls.push({
      agent,
      inputTokens,
      outputTokens,
      model,
      timestamp: new Date().toISOString(),
    });
  }

  /**
   * Get aggregated usage summary.
   */
  getSummary() {
    const byAgent = {};
    let totalInput = 0;
    let totalOutput = 0;
    let totalCalls = 0;

    for (const call of this.calls) {
      if (!byAgent[call.agent]) {
        byAgent[call.agent] = { calls: 0, inputTokens: 0, outputTokens: 0 };
      }
      byAgent[call.agent].calls++;
      byAgent[call.agent].inputTokens += call.inputTokens;
      byAgent[call.agent].outputTokens += call.outputTokens;
      totalInput += call.inputTokens;
      totalOutput += call.outputTokens;
      totalCalls++;
    }

    const estimatedCost = this._estimateCost(totalInput, totalOutput);

    return {
      byAgent,
      totals: { calls: totalCalls, inputTokens: totalInput, outputTokens: totalOutput },
      estimatedCost,
    };
  }

  /**
   * Persist usage data to usage.json in the enrichments directory, appending to a runs array.
   * @param {import('./store.mjs').EnrichmentStore|{readEnrichment: Function, writeEnrichment: Function}} store
   */
  persist(store) {
    const existing = store.readEnrichment('usage') || { runs: [] };
    const summary = this.getSummary();
    existing.runs.push({
      ...summary,
      timestamp: new Date().toISOString(),
    });
    store.writeEnrichment('usage', existing);
  }

  /**
   * Create a usageCallback function bound to this tracker for a specific agent.
   * Pass this to callClaude/callClaudeJson.
   */
  callbackFor(agentName) {
    return ({ inputTokens, outputTokens, model }) => {
      this.record({ agent: agentName, inputTokens, outputTokens, model });
    };
  }

  _estimateCost(inputTokens, outputTokens) {
    // Use default pricing — all calls currently use the same model
    const pricing = MODEL_PRICING.default;
    const inputCost = (inputTokens / 1_000_000) * pricing.input;
    const outputCost = (outputTokens / 1_000_000) * pricing.output;
    return Math.round((inputCost + outputCost) * 10000) / 10000; // 4 decimal places
  }
}

Where it lives

  • scripts/enrichment/orchestrator.mjs
  • scripts/enrichment/agents/
  • scripts/enrichment/lib/usage-tracker.mjs

See it in action

Architecture map →

FAQ

What does the enrichment orchestrator do?

It runs five specialized sub-agents — auditor, blog, content, linker and metadata — that generate and cross-link content fields, then write the results to the main Supabase project.

How is usage tracked?

Each run records input/output token counts per agent and an estimated cost in a usage ledger, so the cost of every pass is known.

Part of these stacks

Related systems

CRSS Exam-Prep PipelineCorpus checks, snapshots, edit digests and SME queues for an exam bank.HF Open-Model Research CrewA CrewAI crew that scouts, benchmarks and writes up open Hub encoders.Content SeedersThe bridge that pushes committed content into the live database.

Explore the full catalog →

Want a system like this built for you?Work with me →