AI

Inference

Running a trained model to get an output — the actual 'thinking' step you pay for.

Inference is what happens every time you call a model: it takes your prompt and computes a response. Training is building the model (done once, staggeringly expensive); inference is using it (done constantly, priced per token). This split is where all your practical concerns live — latency, rate limits, and your monthly bill are all inference. When people talk about 'inference cost' or switching to a smaller, faster model, they're optimizing this step.