Last week I shipped For German, a translation tutor on this site's lab page. Anyone on the internet can type, dictate, or photograph English or German text and get a translation with the grammar explained, every word highlighted by its role in the sentence, and IPA for the hard words. It runs on my personal Anthropic API key.
That last sentence is the interesting engineering problem. A public endpoint attached to your own key is an invitation: one person in a loop can spend your month's budget before breakfast. The feature isn't the model call — Claude does translation beautifully with one cached system prompt. The feature is everything wrapped around it.
Four layers, cheapest first:
A deterministic gate that refuses for free. Input in a script that can't be English or German (a Unicode-range check), URL-only input, empty input — all rejected with a polite sentence before any API call happens. Saying no costs €0.00 and a few microseconds. Only plausibly-translatable text gets to spend money.
Per-IP allowance. Twenty queries per IP per day, reset at midnight UTC. Enough for real use; useless for abuse.
A global daily budget with reserve-and-commit accounting. The tool as a whole may spend 200 cents per day. Each request reserves a worst-case amount before calling the API and commits the actual cost after the stream finishes. The distinction matters under concurrency: with naive check-then-spend, ten simultaneous requests all see "budget available" and collectively blow past the cap. Reservations can't.
Prompt caching and a bounded model. The system prompt is one cached
block, identical for every visitor, so most requests are cache reads. A
capped max_tokens, and Haiku rather than a frontier flagship — a tightly
scoped task doesn't need one.
The result: the worst-case request (an uncached image translation) reserves 4 cents. A typical cached text request commits well under one. The whole tool costs less per day than the coffee I drink while checking its logs, and if it ever goes viral the failure mode is a friendly "come back tomorrow", not a bill.
The transferable lesson, which holds just as well inside a company as on a portfolio site: treating spend as a first-class resource — reserved, committed, reconciled, per-caller and global — is what turns "we tried an LLM feature" into "we run an LLM feature". The model is the cheap part.