Why pilots die in meetings

We hear the same sentence in a lot of first conversations: "We ran a pilot. It impressed everyone and changed nothing."

The people saying it are not naive. They ran a real pilot with a real team on a real problem. The demo was good. Leadership nodded. Then the pilot ended, the slides went into a shared drive, and six months later the work is done the same way it always was.

This is not a failure of the technology and it is rarely a failure of the people. It is a predictable outcome of how pilots are built. Once you see the mechanism, you can design around it.

Pilots are built to be safe to abandon

A pilot is scoped so that if it fails, nothing important breaks. That is sensible. It also means that if it succeeds, nothing important changes either, because the pilot was never wired into anything important.

Look at where a typical pilot lives. It runs against a copy of the data, not the live system. It has one enthusiastic owner, not a team. It is measured on whether the output looks right, not on whether a process got faster or a decision got better. It has no on-call rotation, no budget line, and no place in anyone's roadmap.

So the pilot ends, and the meeting where it is presented is the last meeting it will ever be in. Nobody killed it. There was simply no next step that anyone owned.

The demo optimizes for the wrong reaction

A demo is designed to produce a specific feeling: "that's impressive." Impressive is a fine reaction for a product launch. It is a terrible signal for an operating decision, because the questions that matter are boring.

What happens when the input is malformed? Who reviews the output before it reaches a customer? How do we know it is still working next quarter, after the model vendor ships an update? What does it cost per run, and who is watching that number?

None of those questions get asked in a demo, because a demo is a performance and those questions ruin the performance. But those are exactly the questions whose answers turn a demo into a system. A pilot that cannot answer them has not proven anything about production. It has proven something about a lab.

Nobody has to change how they work

The deepest reason pilots die is that they never required anyone to change a default. The pilot was optional. Using it was a favor to the pilot team. The moment it stopped being novel, the path of least resistance reasserted itself, and the old way came back without anyone deciding to bring it back.

Real change means a process now runs through the AI system by default, and running it the old way is the exception that needs a reason. That is uncomfortable, and it is why most pilots are designed to avoid it. It is also the only thing that makes the work stick.

What we do differently

When we build the first system for a company, we refuse to build it as a pilot. We build it as the first production system, with the scope of a pilot and the plumbing of production.

Concretely, that means it runs on live data with a real integration, even if the blast radius is deliberately small. It has an evaluation harness from day one, so "does it work" is a number that gets recomputed automatically rather than an opinion from a meeting. It has an owner on your side who is not us, and that person is in the room while it is being built. It has a defined place in a process, and the process now routes through it.

It also has an ending. We plan the handoff before we write code, because a system that only works while we are around is a pilot with a longer invoice.

What to do with the pilot you already ran

If you have a pilot that impressed everyone and changed nothing, do not throw it away. It is evidence that the capability is real, and that is genuinely valuable. The gap is not capability. The gap is everything around it.

Ask the boring questions. Who owns this? Where does it live in the process? What is the eval? What does it cost? Who gets paged? If you can answer them, you have a production candidate. If you cannot, you know exactly what the next engagement is for.

Pilots do not die because the idea was wrong. They die because the meeting was the finish line. Move the finish line to production and the same idea usually lives.