The typical situation
A sample-volume forecast, built by a former vendor, set staffing and reagent orders for every centre. It ran every night. Nobody had checked it since it went live, and centres had started ignoring it.
This is common: the model still produces numbers, so nobody notices when the numbers stop being right.
What the data looked like
The model's code and its nightly outputs, three years of actual sample volumes by centre and test, and the staffing and reagent order records.
What we built
- An audit comparing two years of forecasts with what actually happened, centre by centre.
- Monitoring that tracks forecast error every week and alerts when a centre drifts past an agreed limit.
- A retrained model that accounts for the new centres and test types added since the original build.
- A runbook and a monthly review, so the lab's own team can see the model's health without calling anyone.
Typical outcome
| Measure | Before | After | Period |
|---|---|---|---|
| Average weekly forecast error | about 25% | under 10% | typical outcome |
| Time to notice a drifting centre | never noticed | within a week | typical outcome |
What usually trips these projects up
The audit usually finds the problem started with one change in the business, such as a new centre or a price change, months before anyone noticed. Monitoring needs to watch for those changes, not only for the error.