Skip to main content
A successful test request proves connectivity. Production integration also needs predictable behavior when the upstream slows down, a response disappears, traffic exceeds a limit, or a request is repeated. Agree on those behaviors before increasing traffic, because failed calls and repeated operations can add both cost and user confusion.

​
Verify one request and its boundaries

Confirm the selected service’s base URL, endpoint, authentication scheme, required fields, response format, and billing unit. Marketplace services can have their own contracts; do not assume that every API uses the AI gateway’s endpoint or request format. Follow the online subscription and testing guide for the marketplace workflow. Keep credentials on your server in a secret manager or environment configuration. Separate development and production credentials, and redact authorization headers in application logs. Save a sanitized working request and the fields your application relies on as an integration reference.

​
Set a deadline and a retry budget

Set connection, response-read, and overall operation deadlines. A streaming request also needs an idle timeout. Choose values from measured behavior and the time your user can wait. For example, a 5-second connection timeout, 30-second total deadline, and at most two retries can be a starting experiment for a short operation; these are example settings, not Rock API defaults. Use exponential backoff with jitter, and honor Retry-After when the service supplies it. Bound the total attempts and elapsed time. Check SDK, proxy, and worker retry settings together so nested retries do not multiply traffic.

​
Treat uncertain outcomes explicitly

A timeout does not prove that the server rejected the operation. A repeated POST may create a second resource, generation, or charge. Use an idempotency key only when the endpoint documents its support, scope, and retention period; adding an arbitrary header does not create that guarantee. Otherwise, keep a local operation ID and reconcile against a documented status endpoint or usage record before repeating sensitive work. For streamed responses, test partial chunks, disconnects, cancellation, and completion markers. Replaying a partial generation can produce different content and additional usage.

​
Test limits and release gradually

Use a bounded queue and per-service concurrency limits. Separate request rate from token, character, daily, and monthly allowances. Monitor available quota and balance rather than treating every 429 as a reason to buy a larger plan. Test invalid credentials, malformed input, quota exhaustion, delayed responses, interrupted streams, and duplicate submission in a safe environment. Start with a small rollout and define rollback criteria. Record request IDs, status, duration, retry count, and settled usage; avoid logging private payloads. Compare cost and P95 latency against your acceptance thresholds before increasing traffic.

Review AI API security and privacy

Evaluate the data path, retention terms, credential handling, and incident process before production use.