How I Control AI Costs Before Launching to Real Users

Problem

AI MVPs fail quietly on the invoice. A link shared in a Discord, a runaway agent loop, or a verbose system prompt can burn a month of runway before you have product signal. I needed a pre-launch cost posture that assumed curiosity traffic and mild abuse — not only happy-path demos.

Why I built it

After repeating cost mistakes across experiments (write-up), I turned “cost control” into a launch gate: no public users until metering, caps, and a kill switch exist. This post is the playbook I now reuse, including on the parenting copilot.

Initial hypothesis

Most early AI spend is not “model too expensive” — it is unbounded product design:

• Open chat with high max tokens
• Retries without idempotency
• Giant contexts “just in case”
• No per-user ceilings

Hypothesis: if I fix those before launch, I can learn from real users without needing a big cloud credit cushion.

MVP

Minimum cost-control slice I now ship before inviting strangers:

1. Single server-side AI module (one place to meter)
2. Per-action token ceilings
3. Per-user daily/monthly generation caps
4. Global kill switch
5. Usage rows: user, action, model, input/output tokens, estimated cost, prompt version

UI can be ugly. The gate is instrumentation.

Technology stack

• App host: Vercel (or equivalent)
• DB: Supabase (usage table + RLS; service role only on server)
• Models: configurable IDs; cheaper default, explicit upgrade path
• Optional: queue/timeouts so hung calls do not pile up
• Cursor prompts for rate limiting and cost reduction when wiring the module

Supporting reading: Reduce OpenAI API cost, How much does it cost to run an AI app?, cost calculator.

AI architecture

Request → authz → quota check → build bounded context
       → model router (cheap default / escalate rules)
       → call with timeout + max tokens
       → persist usage + result metadata
       → return (or degrade with cached/partial answer)

Controls that matter:

• Quota check before the provider call
• Model router — do not use the largest model for classification or short rewrites
• Context budget — last N items, summarised older history
• Idempotency keys on “Generate” to stop double-submit spend
• Agent step caps if tools are enabled at all

What worked

• Treating cost as a product surface (users see remaining daily generations)
• Defaulting to a smaller model and documenting when escalation is allowed
• Storing estimated cost on every run — made prompt changes accountable
• Kill switch tested once on purpose (muscle memory beats theory)

What didn’t

• Relying only on the provider’s monthly budget email (too slow, too coarse)
• Caching nothing because “every prompt is unique” — many prefixes are not
• Building fancy analytics before a simple CSV/SQL of usage rows
• Forgetting that support staff regenerating outputs can rival user spend

Cost lessons

Numbers I track and will publish when stable:

| Metric | Value | | --- | --- |

Qualitative lessons already clear:

• Output tokens are the silent killer — short structured JSON/markdown wins
• “One more clarifying question” in the system prompt can multiply cost
• Eval runs belong on a separate key/budget from production

User feedback

When caps are visible, early users complain less than when the app randomly fails with provider 429s. Still waiting on verified cohort stats:

• Whether the daily cap message is understood (qualitative notes only for now)

No conversion or revenue claims here.

What I would change

1. Add anomaly alerts (10× user baseline) earlier
2. Separate dogfood, eval, and production API keys from day one
3. Pre-compute cost of the onboarding path (first session often the fattest prompt)
4. Document model router rules in-repo next to the prompt files

Next experiment

Open a tiny public beta only after:

• Kill switch drill completed
• Caps enforced in production
• p50/p95 estimates filled from dogfood (replace the placeholders above)

Then measure whether users hit the free-tier cap before they experience value. If they do, I will shorten the workflow — not silently raise the cap and hope the invoice behaves.