AI Cost Optimization

AI Cost Optimization

The AI plugin includes a built-in cost optimization engine (AA09) that analyzes your request history and automatically routes tasks to cheaper models when quality is not at risk.

How It Works

1. Task classification: each request is classified into one of eight task types using lightweight keyword heuristics (no extra API call required for common cases).
2. Retroactive analysis: a nightly job (or on-demand via nself ai cost report) reads np_ai_requests from the last 30 days and computes what each request would have cost on cheaper models.
3. Recommendations: the engine produces per-task-class recommendations showing current cost, projected cost, savings percentage, and estimated quality risk.
4. Routing rules: you can apply a recommendation as a routing rule with one click in the admin dashboard or via CLI. Subsequent requests of that class are sent to the cheaper model automatically.
5. Quality sampling: 5% of routed requests are still sent to the original model for comparison. If quality delta exceeds 0.15 (cosine similarity), the rule is flagged for review.

Task Classes

Class	Description	Can use cheaper model
code_generation	Write functions, classes, scripts	No
summarization	TL;DR, bullet points, key points	Yes
qa_factual	Simple factual questions	Yes
qa_reasoning	Multi-step logic, math, proofs	No
translation	Language translation	Yes
chat_casual	Casual conversation	Yes
structured_extraction	Parse JSON, CSV, tables	Yes
analysis_complex	Evaluate, compare, research	No

CLI Commands

# View cost analysis for the last 30 days
nself ai cost report

# Specify a time window
nself ai cost report --last 30d

# Show projected monthly savings if all recommendations applied
nself ai cost savings

# Apply a routing rule from a recommendation
nself ai cost apply <rule-id>

# List active routing rules
nself ai cost rules list

# Disable a routing rule
nself ai cost rules disable <id>

# Run a quality benchmark for a task class
nself ai cost benchmark <class>

Admin Dashboard

Navigate to AI > Cost Optimization in the admin panel to:

• See your Optimization Score (0–100): how much of the available savings you are currently capturing.
• Review the top 5 recommendations with one-click Apply rule buttons.
• Manage active routing rules (disable any rule that degrades quality).

Model Tier Order

Requests are routed from cheapest to most capable based on task class:

Haiku / Flash / GPT-4o-mini  (cheap)
Sonnet / Flash               (mid)
Opus / GPT-4o / Pro          (capable)

Local Ollama models are always $0 and are used when available and configured.

Database Tables

Table	Purpose
np_ai_optimization_recs	Generated recommendations (upserted by each analysis run)
np_ai_routing_rules	User-applied model override rules
np_ai_quality_samples	5% quality sampling results
np_ai_requests.task_class	Per-request task class label (added by AA09 migration)

Quality Risk Levels

• Low: summarization, translation, factual QA on a medium-tier model. Well-studied; cost savings reliable.
• Medium: structured extraction on the cheapest model tier. Generally fine; spot-check outputs.
• High: code generation or complex reasoning on a cheaper model. Not recommended; rule creation is blocked for these classes.