Why I Don't Mind That AI Makes Mistakes

Everyone who knows me is aware that I am not a perfect coder. I didn't come from a computer science background. I learned Access because I had a problem to solve and Access seemed like the best option. That forced me to learn on the job, and it necessarily involved a lot of trial and a lot of error.

That's why most of what I know about Access, SQL Server, and PowerApps I learned by trying something that seemed reasonable, watching it fail, and then figuring out why. I don't consider that to be incompetence — it's a description of the way I did my job. Try things to find out what's possible. Whether it actually works is the final evaluation, not how I got there.

A fellow Access MVP even coined a phrase that captures that spirit.

An Access developer is the one who, when presented a new challenge, replies, "Let me try. How hard can it be?"

I've learned two things over the course of nearly 30 years working with, and learning about, Access.

• I learn best by trial and error, not by reading manuals, documentation or books
• Like most people,I learn the right lessons from my mistakes, not from my successes.

When a query based on a correlated subquery returns the correct result set on the first set of sample data, that's a binary result. I've learned one thing — it worked. I don't know whether it worked because I correctly formulated the problem and solution or because the data just happened to fit the way I wrote it that day.

When something fails, though, I get the opportunity find out something real. For example I might learn the complicated query which was fine on ten rows turns out to be catastrophicly slow on a hundred thousand. More importantly though, I learn that using an inline VBA function in a query isn't a great idea, even though on a small set of sample data the problem wasn't apparent.

In other words, when I fail I learn why a different approach might be faster, not just that it is. Failure is specific. Success is ambiguous.

Why it matters when working with an AI agent

AI agents aren't human; the mechanism by which they "learn" is not identical. But the results do look awfully familiar. An agent proposes something plausible that turns out to be wrong — a method that doesn't exist, a T-SQL function that was never in the version I'm running, an assumption about PowerFX delegation that doesn't survive contact with a real data source. In my role as Project Manager, I step in and correct it. It tries again. The next pass is better.

Here's the key though: it's better for a reason I can articulate, because the correction forced the reason into the open. That feedback loop is the really productive part of the work. It's the same loop I've been running on myself for thirty years, just faster.

I'm not suggesting you become less skeptical of AI. It's your job as Project Manger to test and to verify. You still test against real data, and you still assume the AI's irritatingly over-confident answer might be wrong. Looking back over my career, that's the same posture I take toward my own first draft of anything. In fact, I used to think that I was less component than many of my colleagues who seemed to produce such brilliant results. After working with AI for a while, I've come to suspect there's a considerable amount of "trial and error" that never saw the light of day in other shops as well.

With regard to AI, I've stopped expecting a tool that's right the first time, every time. I've never met a developer who works that way, and I wouldn't trust the ones who claim to. I also don't really worry when the AI does have to throw away a try and start again. It's not a sign of failure. It's part of the learning process.

When I work with AI, I want something that gets corrected quickly and doesn't repeat the error. On that standard, the mistakes aren't a defect in the process. They are the process.