Prompt Caching

Prompt caching is a technique that stores the processed form of a repeated prompt prefix so the model can skip recomputing it on later requests, cutting latency and cost.

When many requests share a long, stable prefix — a system prompt or a large document — caching that prefix avoids reprocessing it each time, reducing both latency and token cost.

It is a practical optimization for high-volume agents, lowering the inference cost component of AI labor.