Regression Testing (AI)

Regression testing for AI is the practice of re-running a fixed set of evaluations after a change — a new model, prompt, or version — to confirm that quality has not dropped on cases that previously passed.

Because model and prompt changes can silently degrade some behaviors while improving others, a held-out test set catches regressions before they reach production.

It pairs with an eval harness and a golden dataset, and for agents it guards against the quiet quality loss that causes drift.