Model Distillation

Model distillation is a technique that trains a smaller "student" model to reproduce the behavior of a larger "teacher" model, retaining much of its capability at lower inference cost.

The student learns from the teacher’s outputs or internal signals, yielding a compact model that runs faster and cheaper while approximating the larger model’s quality.

Distillation is one way to cut the inference cost of AI labor, and its quality trade-off is best measured on task outcomes rather than assumed.