Build a shortlist from the requirement
The OpenRouter quickstart documents its API access. The LiteLLM documentation describes its SDK and self-hosted proxy, including virtual keys and cost tracking. Portkey’s overview documents its gateway and visibility features. Verify the exact edition and configuration rather than assuming all deployments offer the same controls.
Compare bills under matching conditions
Request a comparison for the same model version, input/output mix, cache behavior, and currency. Include funding or platform charges where applicable, as well as your own infrastructure and maintenance costs. Read each service’s current terms instead of assuming a fixed markup or discount. For Rock API, start with the live catalog and pricing and billing. Base rates, group multipliers, top-up costs, and historical settlements are separate pieces of evidence. Check whether a displayed price already includes a group multiplier before applying it. No universal saving applies to every model or request.Check caching without a yes/no shortcut
OpenRouter explicitly documents prompt caching for supported providers and models. Cache support is therefore not a reason to describe it as universally absent. For any candidate, verify prefix requirements, cache writes and reads, expiry, provider routing, and usage reporting on the model you need. Provider prompt caching and a gateway returning a cached response solve different problems. Check freshness and privacy requirements before using either. Compare repeated and unique prompts separately so an optimistic cache-hit assumption does not distort the budget.Plan a reversible migration
- Inventory the endpoints, model IDs, tools, output schemas, headers, timeouts, and usage fields your application actually uses.
- Check each candidate against that inventory; an OpenAI-compatible client does not prove full parameter compatibility.
- Use isolated credentials and a bounded test budget. Replay representative, redacted requests and evaluate correctness alongside latency.
- Test unavailable models, rate limits, malformed output, and stream interruptions. Confirm who performs retries and whether an alternate route changes privacy or cost conditions.
- Move a limited traffic slice, review completed usage records, and keep the previous route available for rollback.
