Model Cascade

A model cascade is a strategy that tries a cheaper, faster model first and escalates to a more capable model only when the cheaper one’s answer fails a confidence or quality check.

By resolving easy cases with a small model and reserving the expensive model for hard ones, a cascade lowers average cost while preserving quality on difficult inputs.

Cascades and routers are complementary techniques for managing the inference cost of AI labor.