Investigate an AI cost spike: retries, model mix and longer requests

1. Make the comparison fair

Compare the same projects, timezone and completed periods. Confirm that both periods contain all expected sources. A newly connected source, delayed import or currency conversion can change the reported total without changing the workload.

Keep a change log beside the chart: deployments, traffic campaigns, evaluations and background jobs. Start with the largest contributors in absolute money, then inspect relative changes. A large percentage on tiny spend can distract from the main cause.

2. Count completed work and attempts separately

Choose a business outcome such as an accepted extraction or resolved request. Record its stable internal identifier, completion state and number of API attempts. Do not put raw prompts, secrets or customer content in a cost investigation export.

If attempted calls increase while completed outcomes remain stable, inspect retry policies, timeouts, duplicate jobs and tool loops. Provider billing alone may not identify the business operation behind each call. Verify the retry behavior of your actual SDK and application rather than assuming every retry has the same cost.

3. Inspect the workload mix

Group by model and project where the provider supplies those dimensions. A move toward a more expensive model may be intentional. Then examine input length, output length, cache usage and background evaluation volume using the telemetry actually available.

Compare representative tasks before and after the change. An average can conceal a small number of very long requests. Keep the distributions or a bounded sample, with privacy controls appropriate to the data.

• Check changes in active users and completed tasks.
• Separate foreground requests, evaluations and batch jobs.
• Look for repeated history, oversized retrieved context and excessive tool cycles.
• Verify model and pricing changes against current provider documentation.

4. Change one cause at a time

Define the hypothesis, expected behavior and rollback condition. For retries, inspect which failures are transient and where idempotency is needed. For request length, evaluate whether the removed context affects answer quality. For model selection, compare outcomes before routing more traffic.

A lower bill obtained by failing more jobs is not an improvement. Track accepted outcomes, error rate and latency with the cost during the evaluation window.

5. Reconcile the result

Wait for the relevant billing data before declaring a saving. Compare equivalent scopes and document traffic differences. If there are no accepted outcomes, leave the ratio unavailable instead of reporting zero.

Save the finding as a short incident note: symptom, evidence, cause, action, owner and follow-up. Use the finding to improve the next alert rather than setting a universal threshold from one event.