Clients arrive with one of two beliefs: that they need a large language model for everything, or that they need a model trained from scratch on their data. Both are sometimes right. The choice is a cost and accuracy question, and here is how we make it.
Train on your data
For predictions on tabular business records, which machine fails, which customer drifts, which batch is at risk, a model trained on your own history wins. Gradient-boosted trees on a few thousand rows regularly beat anything general-purpose, cost almost nothing to run, and explain themselves. This is our default for operations problems and we are rarely talked out of it.
Prompt a large model
For reading and writing language, classifying emails, extracting fields from documents, answering from a manual, a large model with a good prompt and your documents in front of it is the right first attempt. The work is in the test set and the guardrails, not the prompt. If it clears your accuracy bar on your cases, stop there.
Fine-tune
Fine-tuning a model on your examples sits between the two. We reach for it when prompting is almost good enough but not quite, when the volume is high enough that a smaller fine-tuned model saves real money, or when the data cannot leave your premises and an open model on your server has to match a hosted one. It needs a few thousand labelled examples and it is a maintenance commitment.
The rule
Start with the cheapest option that could work, measure it on a held-out set from your own records, and only move up when the number says to. Write the number down before you start so that nobody moves the bar afterwards.
Two things we will not do
Train a model where a rule can be stated in a sentence. And fine-tune before prompting has been tried and measured. Both waste the client's money to make the engineer look busy.