The $265 Query Fix: What Two Tools Couldn't Show Me About My Own Work

Why I'm publishing this Case Study

I've spent the last several weeks tracking every AI-assisted work session in my own projects — what got worked on, which model did it, how many tokens it took, what it cost*. I was able to do that because I recently enhanced my tracking tool, called DevPulse, to settle a question I kept asking myself:

How cost-effective is an AI assistant?

That's a broad question and it rests on the assumption that it truly is, at least to some degree, cost-effective to turn certain development tasks over to the AI. So, Claude and I looked for another way to address the question in a concrete way. And, because it's more objective, we rephrased the question to focus just on the economics.

I'm looking at things like AI cost*, revision cycles, and how much of the "elapsed time" recorded in timesheets was really work versus elapsed calendar time between when you clocked in and when you clocked out for the day. It's not concerned with things like code quality or adherence to design standards. Those are measured by different, albeit equally important, metrics. We can talk about that in a different place.

We have those economic metrics in DevPulse, and in a later article, I'll share some interesting examples with you.

First, though, Claude and I wanted to compare our results to other Project Tracking Tools. And that raised the question: Do those others even have the ability to track AI-related costs?

For comparison, we picked two professional Project Tracking Tools: Azure DevOps and Linear.

We wanted to see what, if anything, DevPulse could tell us about development costs using AI, and how that stacked up with other tools.

The cross-platorm comparison "failed"

Claude and I ran two small experiments: I took a real, closed DevPulse project and pushed its history into both commercial tools, side by side, and watched what survived the trip and what didn't.

Azure DevOps doesn't even have fields for AI-related economic metrics.

Neither does Linear.

I don't know if any of the many other available project tracking tools do; my hunch is that they'll probably be the same as these two. There are other kinds of work related metrics -- recorded hours worked in ADO and "points" in Linear. The ADO metric is a gross elapsed clock time, which we'll look at more closely below. The Linear metric is a point system indicating how many points a task is expected to consume, or did consume. What a "point" is and how it's determined is not clear to me yet, although I surmise it's explained somewhere on their website.

That lack of comparable data left us to examine what DevPulse can tell us. I hope you find it as interesting and revealing as I do.

In particular, one session in the DevPulse history turned out to say more than I expected — not just about the tools, but about my own discipline. That's the one I want to walk through.

The $265 query fix session

Session 3099 shows up in my records as a small thing: "Fix lookup-edit-form cache-binding anti-pattern (3 queries)." You may be asking yourself, "Do what, now?"

Let me explain. The Access FE links to a SQL Server back end. For lookup tables, we cache the records in local cache tables. Combo and list boxes load faster from the local cached values than from the linked tables, a minor, but worthwhile perfomance enhancement.

The problem we remediated was that three forms in an Access application were writing edits to their local cached lookup tables instead of the live database lookup tables; that's a real bug, the kind that quietly loses data until someone notices. The fix itself was straightforward: repoint three queries from local cached to linked tables, verify the rows come back correctly, done. Not more than a few minutes of work for a developer or for the AI.

By the numbers most tools, including Azure DevOps and Linear, would show you the cost of this session was minimal. When I pushed this project's history into a test instance, Azure DevOps logged the task at 0.55 hours — half an hour, rounded. Linear, in the same experiment, showed it as "1 point" — its smallest unit of relative size. Both entirely reasonable-looking numbers for a three-query bug fix.

My guess was maybe 15 minutes of work, but I can see a legit half an hour, including post-fix testing.

DevPulse's own record for that same session shows $265.03 in AI equivalent cost — 5.7 million tokens of output, the second-most-expensive single session in everything I've tracked so far. That's an eye-popping divergence.

Half an hour of work. $530 an hour for an AI assistant that's supposed to save me time and money?

Something doesn't add up, and I want to be honest about what that something is, because it isn't what I expected to find, and it reveals the power of the DevPulse tool's economic metric tracking.

What actually happened

The $265 isn't the cost of fixing those three queries. It's the cost of an entire working session of which the query fix was one part. It included a full review of 35 forms and their control names, which I'd asked for in the same sitting, presented back to me before I closed anything out. When it came time to log the work, I cheaped out, recording the whole session's token total against this last small task, because at the time, splitting the cost precisely across everything done in that sitting wasn't yet something I'd become disciplined about. I lazily approved logging it as a blob of work, rather than spending more tokens on sorting things out properly. In fact DevPulse has it right there in my own notes from that day: the totals reflect the entire session, not just this task.

That was a subtle trap I'd set for myself, exposed because DevPulse also tracks economic metrics other project tracking tools don't.

I could have spared myself some embarrassment and picked a cleaner example for this piece — a session where the numbers and the task line up exactly, no asterisk required. I decided not to take the easy way out because this one teaches the more useful lesson.

The tool built specifically to catch this blind spot still needs the discipline to use it correctly.

I've since strengthened our discpline regarding checkpoints and closing tasks. Without a checkpoint — a deliberate pause to say "this task is done, log it, start the next one clean" — cost tracking degrades into the same fog it's supposed to clear up.

That's not a flaw in the concept itself. It's what happens the moment you stop being deliberate about it.

I know, because I did exactly that, in my own data, on a day I didn't think to stop and split the session accurately.

Did I ever mention I have a tendancy to be a "Cowboy Coder"? This is just another illustration of how that can go bad.

What this says about the other tools

Here's the part that matters beyond my own workflow discipline: with my bookkeeping mistake (attributing an entire session's cost to one task instead of splitting it accurately across everything that happened in that session) DevPulse is still the only one of the three project tracking tools that clearly shows you anything is wrong at all.

Azure DevOps and Linear both reported a small, plausible-looking number for this task — 0.55 hours in the first case, 1 point in the second, and neither one had any way to ask "does this actually reflect what happened?" There's no field in either tool for AI cost, token volume, or how many things really got bundled into one sitting. A number that's quietly, catastrophically wrong looks identical, in both tools, to a number that's simply small. Our covering note stating that the entire session, not just one task, was included didn't carry enough semantic weight to raise an alarm to a casual reviewer.

"Three queries in an entire session"? Not much to be concerned about in that phrase. A cost several times larger than similar sessions is something to explore more carefully.

This isn't a criticism of either product — they were built to track human-estimated work, and they do that well. It's a gap, not a flaw, and it's the exact gap I built DevPulse to fill.

The useful version of this story is this: DevPulse caught something neither other tool could have caught, and then showed me that I was the remaining weak link in getting it right. Both things are true, and I think that's a more useful lesson to publish.

What can this concept change about how you work

If you're doing AI-assisted development and tracking your work in Azure DevOps, Linear, or anything like them, here's the concrete takeaway: A task's logged duration and its actual cost are two different axes. Nothing stops them from silently diverging, the way they did here, unless someone, or something, is specifically watching for it.

• The human initiated checkpoint is the discipline. Before you close out a task, actually asking "did I just do only this, or did I let three other things ride along?" is a habit, not a software feature. I didn't have that habit on 2026-06-15. Three weeks later I do, and it's the direct reason I caught this at all — the record created by DevPulse tattled on me.
• If a tracked number ever looks suspiciously convenient — too round, too small, too easy — it's worth asking what actually produced it, the same way I went back and asked what produced "0.55 hours at a cost of $265" here instead of just accepting it.

Where this goes from here

In an upcoming article, I'll deliver on the promise to tell you more about the specific metrics we track, and what we found in our test push from DevPulse to Azure DevOps and Linear.

• In DevPulse, Cost is an estimate of the equivalent API usage would cost, even though I'm using Claude Code Pro with a fixed monthly license cost.