Service · Data and platform

ML operations and monitoring

Keep the models, pipelines and assistants you already have running, honest and cheap, in your own environment.

Relevant to Manufacturing, Service businesses, Education, Growing businesses

The problem

A model was built, by us or by someone else, and then the person who understood it left. Nobody knows whether it is still right, what it costs, or how to retrain it.

Pipelines fail silently after a power cut. A vendor's model changes under you. Accuracy drifts as the business changes, and nobody is measuring it.

What it costs today

Typical: a system that was accurate at launch and is quietly wrong a year later, and cloud and API bills nobody has reconciled against the value they deliver.

This is for you if
  • One or more models, pipelines or AI assistants in production, built by anyone.
  • You want to know, every month, whether they still work and what they cost.
  • Your IT team is small and should not have to become an ML team.
What we build
  • Monitoring: data drift, prediction drift, accuracy against outcomes as they arrive, and a page that says when the last run succeeded.
  • Retraining and release: scheduled, tested against the held-out set, rolled back if worse.
  • Cost control: per-model and per-conversation cost, cheaper models where the harness allows, caching and batching.
  • Runbooks and a monthly review, for your IT team or for ours on retainer.
How we build itIn this order
  1. 01

    Audit what is running

    Every model, pipeline and prompt: who owns it, what it costs, when it last ran, and whether anyone can retrain it. This is usually the first surprise.

  2. 02

    Add the missing measurements

    Outcomes joined back to predictions, drift metrics, a health page. Nothing improves until it is measured.

  3. 03

    Make it restartable

    Idempotent pipelines, a scheduler that survives reboots, and alerts that reach a phone when a run is missed.

  4. 04

    Set up the release path

    A new model or prompt is tested against the held-out set and the harness, and ships only if it scores better. Rollback is one command.

  5. 05

    Review monthly

    Accuracy, cost, incidents, and the one change worth making next month. One page, read by someone who can act on it.

Data you already haveThis is usually enough to start
  • The systems themselves, wherever they run
  • Prediction logs, if any exist
  • Outcomes: what actually happened after each prediction
  • Cloud and API bills
In practice

Illustrative example

Catching a demand forecast that had quietly stopped working

A sample-volume forecast, built by a former vendor, set staffing and reagent orders for every centre. It ran every night. Nobody had checked it since it went live, and centres had started ignoring it.

Read the case study
Typical engagement
Duration2-week audit, then 4 to 8 weeks to set up; monthly retainer optional
TeamML engineer, platform engineer
You receiveAudit report, monitoring in place, release and rollback path, runbooks
PriceFixed price for the audit and setup; retainers priced monthly
What you own at the end
  • Monitoring, scheduling and release tooling in your environment, built on standard tools.
  • Runbooks written for your IT team.
  • A monthly report format and, if you want it, our team on retainer to run it.

Everything runs in your environment and is handed over with documentation. Nothing stops working if we part ways.

Questions we are usually asked

Someone else built our models. Can you take them over?

Yes, and it is a common starting point. The audit tells you what is maintainable and what should be rebuilt, with the reasons.

Do we need Kubernetes or an MLOps platform?

Almost never at this scale. A scheduler, a database, a dashboard and disciplined scripts cover most businesses, and your IT team can run them.

How do we know a model has drifted?

We compare the inputs it sees now to the inputs it was trained on, and its predictions to the outcomes as they arrive. Both are on the health page and both raise an alert.

What does the retainer include?

Agreed hours, a named engineer, the monthly review, and retraining and releases as needed. It is the retained engineering option on the How we work page.

Can you monitor an AI assistant or agent the same way?

Yes. For those, the harness is the held-out set: a sample of real conversations is re-scored every month, and cost per conversation is tracked alongside.

Often combined with

Not sure if your data is ready?

Take the free 20-minute readiness check. You get a short scorecard and a plain answer about what, if anything, is worth doing next.

Start the readiness checkTalk to us