Most rules are a substitute for judgment

I always say ideal documentation is good enough for people not to have to think about how to do something.

This is half true.

Because of course we want people to think, but they should not invent things that already exist. They should use the existing standards to their advantage, to be faster and consistent.

I learned that this isn’t always true from a different kind of documentation - the Skills I write for AI models. A Skill is basically an instruction file / process documentation, which tells a model how to do a task. When the models got better, my rules started to make them worse.

More rules, worse results

When I started using Skills, I mostly used the popular ones from others. In the beginning, every rule worked like magic.

Over time I saw the need to make my own, as generic rules were not enough. When AI did something wrong, I added another rule and the file got longer. More detail looked like more control.

Then newer models came along. They followed my rules exactly, so according to the rule, everything was okay. But it was not the result I wanted. When the AI proposed an alternative, it looked better. I had written instructions to solve problems that no longer existed.

Mantinels

In Czech, a mantinel is the barrier around an ice rink. A good framework works the same way. You can move freely inside, but you shouldn’t move out of it.

The temptation is to control everything, so every task is done the way I would do it myself. But that’s not what mantinels are for. They are there to prevent surprises.

When someone does a task their own way, sometimes a better way is found. Sometimes it’s just different, not better - and different is often enough to make things complicated for everyone else.

So which rules are mantinels? In my Skills, the answer became a test with two questions. Does a rule give the model something it cannot know on its own - a format, a name, a hard requirement? Or does it tell the model how to think, where it could think for itself?

The first kind is a mantinel. The second kind is a substitute for judgment. When I looked honestly, most of my rules were the second kind.

The same two questions work for any process document, not only for AI.

Rules expire

Every rule in my Skills starts as an observation - a note about what went wrong and why. But I don’t use the details as part of the rules, only the principles behind it. Keeping the details out was a lesson of its own, how to build a long-lasting system, so the Skills stay short and carry only what is really needed.

Sometimes nobody remembers why a rule is there. That doesn’t automatically mean it is useless. Sometimes the problem is gone exactly because the rule worked. But nobody can tell anymore, so nobody dares to delete it.

Rules are static. People are not. We learn and the environment around us changes too - the newer models were exactly that. A rule that once helped can quietly become the thing that holds us back.

With a model, deleting a rule is one line. With a team, it’s harder.

The rule I would never cut

Some people would do better without a rule that helps everyone else. Someone in their first month needs the “how”. Someone who has done it a hundred times is slowed down by it. And a team cannot have a different rule for every person. People need to be able to rely on each other. How do we deal with something like this?

I don’t know. But we always have to observe what’s happening, and listen.

That is also why rules should not be one-sided. A manager should listen to the team when setting up rules. The team might see dependencies the manager missed, or know the rule solves one problem and creates three others. The point is not that everyone has to agree - but everyone has to understand why the rule is there, why it is needed, and also why it might not work.

So the one rule I would never cut is the one that lets us change all the others.