What building an "AI wizard" cost me — and it wasn't in tokens

For the last month or so, I've been building a library of templates that an AI assistant reads and acts on. Using the tool, you point your AI at a template, it reads your standards, and it designs tables, or an audit logging system, or an error logger, and then it builds them once you approve. Work that used to be an afternoon of careful, error-prone typing is now a conversation, with the AI taking over the drudgery of turning an idea into a usable artifact.

That part has been everything I hoped. A complex job is becoming a manageable one. I say "becoming" because there's still a lot of work to do. The shape of it, though, is now apparent.

As you know from previous blogs, I pay attention to the costs of using AI, not because I pay them all, but because it's important to me to be able to say I'm working efficiently. Today I realized there was an invoice in front of me that I didn't see coming, and it has nothing to do with compute time or token burn.

The invoice demanded that I go back and re-examine many of the basic things I've done for nearly thirty years. Not the coding logic, not interface design, not data normalization. Not even the way I communicated with clients. What I had to look at with a fresh eye is the way I ask a question and the way I respond to questions about what needs to be done. Even the words I've used so long I stopped thinking carefully about what they mean to me and to others. This is, in fact, yet another aspect of shifting my primary role from Access developer to project manager.

Here are three examples of what that realization actually looked like.

Exhibit A: a question I was proud of, at first

One of my templates needs to know which tables to audit. Here's the actual question I wrote:

"Which of your tables should the scan consider for auditing? This is the one boundary decided in code — everything finer-grained is a switch you flip in a table afterwards."

I thought that was helpful. I was heading off a follow-up question. I was telling the user adopting the template about the flexibility I was so proud of in the template. My intent was to reassure them, so they wouldn't worry about being locked in by an early choice.

Today, troubleshooting a trial run, I was coerced into reading it again from the point of view of someone who has never seen my library.

My question merely generated more questions. What in the heck is "the scan"? What boundary am I asking about and between what things does it sit? Decided in whose code?

A newcomer can't learn anything from that sentence except that they are out of their depth and this template is not for them. And that person walks away with a vow not to use the tool. They don't file a complaint first, although that would be a kinder thing to do.

The rewritten question is five words:

"Which tables should be audited?"

That question is supported by similarly simple options such as "all tables" or "only tables with the tbl prefix".

In addition to the options, everything else the user needs to know is still available in the template. I just had the AI move it behind a "Tell me more" option they can click if they want it. I didn't take the information out of the template. Instead it only shows up for the person who needs, and asks, for it.

Another question, same faulty communication strategy:

"Should I run the read-only readiness check on your tables before anything is generated? It reads your table definitions and reports any table this system can't track as it stands — it changes nothing."

turned into

"Should I read your table definitions and report any table this system can't track as it stands?"

The AI template doesn't name the machinery or explain how it works. It only needs to say what it will do. And there's always the "Tell me more" if they need it.

Exhibit B: some words such as "default" can be a not-so-subtle trap

I've used the word "default" hundreds, maybe thousands, of times doing Access work without pausing to consider what it means. I'll bet I'm not alone in that.

In the course of today's trial run, I realized that, when the AI Agent is involved, "default" has two meanings, and one them I hadn't really had to think much about:

A. Default can mean the option we'd point at first, or recommend, or prefer. In this meaning, the AI has to offer it as the preferred choice, but it does have to offer it to the user.

B. Default can also mean what happens when nobody chooses. It's the fallback that fires on its own. Here's where it can burn you. When the AI finds something is specified as the default, it doesn't bother the user with a question; it just does it.

That's where we got into trouble. The AI was using definition B when I was expecting definition A.

When you write code, a default parameter value is the fallback, definition B, and nobody is confused because there's no 3rd party like the AI agent involved.

In the template, an agent will act on what you wrote exactly as you wrote, and definition B wins — because acting is what it's there to do. When the AI reaches a line labelled Default: it takes that as standing permission to skip the question. And lo and behold, the question doesn't get asked. The user is confused and the template gives off a faint odor.

That isn't hypothetical. It caused more than one trial run to go south on me, to my great frustration.

My own standards file said, in effect, " at this point emit option 3, and say so." The assistant did exactly that. It chose an error-handling approach, announced the choice, and never asked me if that was what I wanted for this current trial. I sat there thinking, "when the heck did I decide that?"

The defect was in a file written weeks ago, long before I figured this stuff out; it's not in the way the AI works. I had written a recommendation and an instruction-to-act in the same breath, with no way for the AI Agent to tell them apart, at least not in the way I intended.

So the word, "default" is gone from the templates. Instead every choice in the library now names a preferred choice, and there's a written rule about when it becomes the answer: a) when the developer declines to choose, or b) when they say "just build it." Never on the assistant's own initiative. Never because the answer looked obvious to the AI.

Exhibit C: making a statement of policy about using a tool or feature instead of just offering to use that tool or select that feature

I had written this into a build step:

"How should the four VBA modules get into the database? The library's default is to hand you the files — it won't assume an MCP just because one is connected."

That's not just a question. That's me also justifying a rule and going on to explain the philosophy of the library to somebody who just wants their modules installed. It also drops the term "MCP" on a reader with no definition, as if everybody knows. (Do you know, by the way, what an MCP Server does? There's a "Tell me more" in the template for that.)

That one turned into:

"I have a tool called an MCP that I can use to add the VBA modules. Should I use it?"

That's a question. What an MCP server is, and why you might not want one, live behind "Tell me more".

Exhibit D: the format and presentation of the question, not just the words

The first time I ran this, the assistant laid the choices out as a nicely formatted table and waited for me to type an answer in the chat box. Every word in the table was correct. It was still a failure because the template had promised me, the tester, a choice and handed me an essay question instead.

Rule of thumb for templates: If someone has to compose a sentence to answer you, you've made them work and making them work is the last thing a template needs to do.

Now every step is a predictble pattern consisting of two or three clickable options plus Tell me more as the last one. Tell me more is always last and always clickable. If you are not sure which option to choose, it will give you an explanation. Then you're asked the same question again, unchanged. Curiosity costs one click and loses you nothing.

So what is this thing I'm building?

I've taken to calling it an AI wizard, but I realize I should be careful with that word around Access people.

My AI Wizard approach is a wizard in concept only. It's not an Access Add-in. It installs nothing in your database. It builds no Wizard forms. It leaves nothing behind when it's completed, other than the artifacts you asked for. It's just the set-up decisions asked one short question at a time, each with a plainly named preferred choice, and the reasoning tucked one click away instead of fired at you before you've chosen anything.

It's a presentation device that I call a wizard only because of its similarity to the familiar Access concept.

The actual cost, and there's always a cost

The examples above are things I'd already written and thought were clear and helpful.

It turns out that they were not clear at all. Well, yes, they were clear to me, because I already knew all the underlying detail I was gesturing at; heck I'd built it. That framing language, the stuff that stages an action and explains where it fits, is genuinely useful to the person who built the thing. It's part of the process that created it and it has to be memorialized somewhere. That somewhere, though, is not in the template itself. To someone arriving fresh, it's white noise at best and a "you don't belong here" sign at worst.

Ironically working with an AI agent is precisely what made me see the flaws in the most uncomfortable way. It did exactly what I wrote, instantly, in front of me. There was no polite colleague in the loop smoothing over a sloppy sentence, rephrasing things for the crowd, asking clarifying questions. None of that. Just me and the AI literally doing exactly what I told it to do.

Where this stands at the end of today's work: three trial runs, three different failures, three different causes. The wizard still hasn't made it all the way to the last step in a trial run. That's because we're working out the methodology right alongside the building of the artifacts.

AI makes hard things manageable, but it will hand you the bill in a currency you didn't expect: finding out that your clearest writing was only ever clear to you. Pay it. It's worth it. But budget for it. It's in your next invoice.

--written with assistance from Claude Code