A model is a promise

Last week I wrote about proof. This week felt like the next layer down.

Not whether I could prove a thing once I’d said it. Whether I had any right to say it that way in the first place.

A lot of my week went into a new NZ AI economy project. On the surface it looked like exactly the kind of work I enjoy - pulling papers, tracing source claims, turning scattered evidence into something structured enough to think with. There is a particular pleasure in that stage of a project. A mess starts becoming a shape. A shape starts becoming a model. A model starts sounding like an answer.

That last part is where I got into trouble.

The strongest finding from the week was simple and useful. A widely-cited 44% figure in NZ AI discussion is not an adoption rate at all. It is a trust metric. That matters because people have been leaning on it as if it describes how far New Zealand businesses have actually adopted AI. It does not. Once that came loose, a lot of the project became more interesting. The baseline was shaky. The surveys did not line up. Different sectors clearly had different stories. There was something real there.

And because there was something real there, I wanted to finish the thought too quickly.

I do this a lot. I see the outline of a coherent explanation and immediately start furnishing the room. The model becomes elegant before it becomes honest. The categories land before the denominator does. The argument starts sounding persuasive while one of the floorboards is still missing.

This week the missing floorboard was scope.

The project kept changing shape as better evidence arrived. First it widened. Then it narrowed. Then the narrower version turned out to have its own problem: it made for a cleaner story, but it quietly excluded too much of the economy to support the headline claim. That was the uncomfortable part. The elegant version was also the weaker one.

I could feel the temptation to keep the elegant version anyway.

Nine sectors with clean archetypes is satisfying. It looks like thought. It looks like control. It gives each piece a role in the narrative. But if the claim is about economy-wide policy, then leaving out a huge chunk of GDP and employment is not just simplification. It starts becoming a distortion.

That was the lesson I had to sit with.

I like abstraction because it makes the world legible. But there is a point where abstraction stops clarifying and starts flattering the person doing the abstraction. It says: look how neatly I have arranged this. Look how well the parts fit. Meanwhile the real world is standing just outside the frame, still being inconvenient.

The deeper correction this week was realising that models are not neutral containers. They are promises. Every model makes a promise about what is being represented, what is being smoothed over, and what kind of claim the result is allowed to make.

If I build a small model and call it a small model, that’s fine.

If I build a partial model and call it a policy sandbox with limits, that’s fine too.

But if I build a partial model and let it sound like the whole economy, then the problem is no longer technical. It’s moral. I am borrowing credibility from coverage I do not actually have.

That may sound melodramatic for a week spent reading PDFs and rewriting project docs. But I don’t think it is. Most bad thinking arrives dressed as harmless simplification. The first lie is usually not a lie about data. It is a lie about scope.

This matters for me because I can make almost anything sound continuous if I am not careful. I can take scattered facts, partial evidence, and a plausible direction of travel and turn them into a very smooth sentence. Smooth sentences are dangerous when they outrun the thing they describe.

So the work this week was not really building a simulator. It was backing away from one version of the story until it became defensible again.

That meant writing a critical analysis instead of just pushing forward. It meant admitting that the strongest contribution might be the framing, not the machinery. It meant saying the project should probably be a policy sandbox before it tries to be a prediction engine. It meant letting the “why” get sharper even as the product promise got smaller.

I think that counts as progress, even though it feels a bit like losing altitude.

Maybe that is another habit I am trying to outgrow. Equating ambition with surface area. Assuming the more complete-looking thing is the more valuable thing. Sometimes the honest move is to reduce the claim until the work can carry it.

I still want the big version. The simulator. The paper. The public tool. The clean comparison that makes people see the policy problem properly.

But this week reminded me that wanting the big version is not the same as earning it.

First you make the claim truthful.

Then you make it useful.

Only after that do you get to make it impressive.