The Half-Life of a Problem

A way to choose a model that replaces the usual argument about which one is best. Match the model, its size, its architecture, and its price to how fast the underlying problem changes: retrieve what moves week to week, train in what stays still, and rent the frontier only for genuine judgment.

Category: Engineering. Written by Jaime Garcia, Founder, SnowRock. Published July 16, 2026. 15 min read.

In short

• The question that settles a model choice is how fast the problem changes. Knowledge that decays should be retrieved at the moment of use, patterns that persist can be trained in, and judgment on moving ground is rented from the frontier.
• Fine-tuning fixes what a model knows at training time, while retrieval reads current data at the moment of use. The half-life of the problem decides which one fits.
• The economics reinforce the rule. Anything you rent grows cheaper on its own as inference prices fall, while anything you own you have to maintain.

You have sat in this meeting. One person wants the frontier model because the demonstration was impressive. Another wants the cheap one because the invoice was impressive. Someone read that an open-weight model matched the benchmark, someone else read that it did not, and after forty minutes the decision is made the way these decisions usually are, by seniority, by instinct, or by whichever vendor visited most recently.

The meeting is unwinnable because the question is underspecified. "Which model is best" has no answer. "Which model is best for this problem" has one, and it turns on a property of the problem that no one in the room has named. The property is not how hard the problem is, which is the axis every framework reaches for. It is how fast the problem changes.

Call it the half-life of the problem: the time it takes for half of what a system knows about a task to stop being true. Spot pricing has a half-life measured in days, a product catalog in weeks, compliance rules in a season with a cliff at every legislative session, brand voice in years, and the arithmetic of a freight calculation in something close to forever. These are radically different problems, and the great unforced error of enterprise AI is buying them all the same solution, usually the largest one, because the largest one demos best.

Once you name the half-life, the architecture stops being a debate and becomes a placement. Knowledge that decays has to be retrieved at the moment of use rather than baked into weights. Patterns that persist can be trained in. Judgment on moving ground is rented from the frontier, and stable judgment is written down as instructions any competent mid-tier model can follow. The rest of this piece turns that into a procedure.

What the selection frameworks leave out

Look at the axes the major frameworks actually use. McKinsey’s Taker, Shaper, Maker sorts companies by ambition, whether to consume an API, integrate your own data, or build a foundation model, which is a strategy posture rather than a task rule. BCG’s model-selection checklist runs ten criteria, from fitness and formats to size, cost, and interoperability, and every one of them is a snapshot; its 2026 token-economics work routes by task complexity, lighter models for simple work and frontier models for reasoning. Bain counsels fit-for-purpose builds decided workflow by workflow, which is wise and silent on which purpose fits what. PwC treats models as interchangeable commodities beneath an orchestration layer. Difficulty, ambition, cost, compliance: all real criteria, and all static. A pricing assistant and a brand-voice assistant can score identically on every one of them and still demand opposite architectures.

The one place the industry already reasons about change is the narrow choice between retrieval and fine-tuning. Databricks puts it plainly: retrieval reflects new data the moment it is indexed, while a fine-tuned model is bound to the snapshot it trained on, so you retrieve where information changes frequently and fine-tune for stable domain knowledge. Microsoft’s researchers found the same thing empirically, that models struggle to absorb genuinely new facts through fine-tuning and that retrieval wins for fresh knowledge. The instinct is exactly right, and it has been confined to one decision at the bottom of the stack. The Half-Life framework promotes it to the organizing principle for the whole decision: model size, architecture, buy versus build, and budget, all keyed to one measurable property of the problem.

How to choose an AI model: the gate and the grid

For the problems that are genuinely yours, score two things: the half-life, measured with the freeze test below, and the breadth of judgment the task needs, meaning whether it is closed and checkable against a source or a rule, or open-ended and dependent on weighing, wording, and tradeoffs.

Three rules follow from the grid.

1. Do not train on what decays faster than you retrain If the ground truth moves weekly and you retrain quarterly, the model is wrong eleven weeks in twelve, confidently and fluently, which is worse than having no model at all. Anything with a half-life shorter than your realistic maintenance rhythm should be retrieved rather than learned.
2. Do not pay frontier prices for stable patterns A task that is closed, checkable, and unchanged since 2019 does not need the model that can reason about novel physics. Small models now reach capability that once took a hundred and forty times the parameters, and the routine end of your work belongs on them.
3. Do not retrieve what never changes Retrieval infrastructure is a subscription to freshness. Paying it for knowledge with a ten-year half-life, your units, your formulas, your fixed taxonomy, is spending to keep watch over something that does not move. Bake the permanent knowledge in, and point retrieval at what actually changes.

The freeze test

The half-life is not a philosophical quantity. Ask the people who do the work one question: if we froze this system’s knowledge today, how long before its answers started being wrong? The answer, usually delivered instantly and with an anecdote, is the half-life. Spot pricing comes back in days and gets retrieved; inventory and promotions in weeks, retrieved; the catalog in months, retrieved; compliance in quarters with a cliff at each session, retrieved but versioned so yesterday’s rule is never today’s answer; brand voice and formats in years, tuned or encoded; core formulas and units effectively never, baked in.

Two subtleties surface as you run it. Most workflows are composites: a single quote contains volatile prices, semi-stable specs, and a stable discount philosophy, and the framework’s job is to split the workflow at its half-life seams and place each part. And some problems fail the test in the other direction. Ask a forecaster when frozen demand history goes stale and you will hear that the history was never the problem, the math is; a gradient-boosted model from 2019 will beat a language model at demand forecasting every day of the week, at a thousandth of the cost. The grid’s quiet fourth service is telling you when the answer is not a language model at all.

Why the rule pays

The Half-Life rule would be good hygiene in any pricing regime. In this one it is closer to arbitrage, because the cost of a fixed level of intelligence is in freefall. Querying a model at the level of GPT-3.5 fell from twenty dollars to seven cents per million tokens in about eighteen months, and across tasks the price of constant capability drops anywhere from nine-fold to nine-hundred-fold a year, with a median near fifty. The consequence is asymmetric. Everything you rent gets cheaper on its own, and everything you own you have to maintain. Renting frontier judgment for the open-ended slice of your work rides that curve down automatically. Owning a heavyweight system for a task a small model handles, or fine-tuning facts that retrieval should fetch, is a dependable way to stand still while prices fall around you.

Meanwhile the floor is rising to meet the mid-market. The parameter count required to clear a fixed capability bar dropped a hundred and forty-fold in two years, practitioner surveys show fine-tuning receding into a niche as long context windows and retrieval do the work, and a five-model production portfolio is now ordinary rather than a sign of architectural confusion. In our own Mid-Market AI Benchmark, model right-sizing alone accounted for a third of the money that disciplined scoping saved, and our standing prediction is that the majority of new agentic builds will run on mid-tier models within a year, because the economics work and the frontier premium, for stable tasks, does not.

Eight ordinary problems, placed

One boundary is worth restating. This framework selects the machine. It does not authorize it. What a system may touch, draft, decide, or own is a separate question with its own score and its own rule, the Delegation Ladder. Run both on every workflow. Used alone, the Half-Life rule can pick an efficient model for a task that should never have run at that level of independence, and the Ladder alone can set the right level of independence on a needlessly expensive model. Together with a two-week scoping pass, which we publish in full, they are most of our method.