AI made producing work cheap - what it didn’t make cheap is knowing if that work is right

There’s some version of this quote hanging out across the forums I frequent on the internet. Reddit, LinkedIn, various digital media. They all agree in some way. Hackernoon says “Building got cheap. Thinking didn’t”, and since slopification has taken over my life as a product manager, I can’t think of a more true statement.

What is slopficiaiton? It’s AI’s tendency to hallucinate, plus the way it writes, combined with its frustrating level of confidence. Here’s how a few of my days have gone recently. 

‍
Battling The Basics

By around 10 a.m., I had handed my workplace AI a handful of the small, vaguely annoying tasks that when delegated to AI are supposed to make a product manager's day easier: Write a stakeholder update in the same format I’ve used for the past 10 weeks, summarize what this person is talking about in this slack thread, give me a read on this data you’ve had access to for a while. None of them felt high effort, or even major stakes.

By 4 p.m., I was still reading. 

I was opening tickets, re-reading the original thread, checking numbers against sources, and looking for the small changes that had disappeared between one revision and the next, even without me asking. 

I had produced more than I would have alone, sure, but a lot of it felt untrustworthy. Part of that is because of little details missing, and part of it is because out of the box, AI writes in a way that feels like convoluted car salesman speak dressed up with a CS degree (yes, this is my own comparison). 

That is the trade-off I did not understand fully when I started using AI as heavily as I do today. Producing has become almost instant, but now I am my AI’s QA, editor, and fact checker. It’s exhausting.

Checking is still work, and I don’t think we fully understand the costs yet 

I am not saying AI is useless, or that every output needs 100% confidence that would hold up in a courtroom. I’m saying as we keep moving forward into the great next chapter in knowledge work, we aren’t accounting for the cost properly.

‍

‍

The statement “practice makes perfect” exists because when you do a task yourself, you build confidence and knowledge as you go. You know where statements like, “We grew our market share in Europe by 21.5% this summer” came from, because you sourced it, pulled it, and probably scrubbed the dirty data. You know why a sentence is in the narrative you wrote to your stakeholders, because you put it there. To verify it, you know where to start.

When AI does a first pass, it is acting like someone new to the job. You often have to remind it where things are (even if that's baked into your setup), why you’re asking for something, and even, very frustratingly, remind it of your preferred style choices. What you get out of the first pass looks great - inished artifact, formatted nicely, great headers. 

But hen you start digging in…ou find a flaw, and all of a sudden you’re on a new crusade for understanding. 

In practice, that means opening the ticket. Reading the thread one more time. Checking the source-of-truth document. Clicking the link instead of trusting that it has the right title. Then doing it again after a revision, because a new version often loves to undo something you already approved.

When companies started going “AI first” we didn’t announce this trade-off, or arguably really think about it. We focused on how quickly our AI tools could create output, and make us faster. The flaw is, in a lot of knowledge work, the time hasn’t been “given back”. It’s been eaten up by reviewing, fact checking, and sleuthing, and it’s burning people out.

The most dangerous answer is often the one that sounds right

One of the first moments that made this concrete for me was through an AI-generated status update that I run with a custom-built skill. The claim sounded confident, current and precise. A PR (pull request) had been opened, and work was moving along. It looked exactly like something I would normally write after looking at my team’s project. 

Except it was completely false. The ticket had not been touched. There was no pull request.

That’s one of the more egregious errors I’ve seen, among other wildly confident statements - like when I asked, “We haven’t done that right?", and AI responded, "We haven’t built that feature yet”.

Reader:… we had built that feature yet.

‍

The problem is that claims like the PR example above fit my mental model. It’s specific enough that it feels like a fact, current enough it seems useful, and plausible enough to slide past scrutiny.

It’s important to remember that the outputs that agree with you are often the ones worth checking. AI doesn’t have to make an outrageous mistake to send you down a rabbit hole, only a believable one. 

‍

Approval is not a one-time thing, and AI has been forgetting that

Another type of task that’s eating time, are the ones where initial output is mostly fine, but the revision process turns into its own kind of work.

I had a stakeholder brief outlining some changes we were making to our system that took five rounds to get into shape. I gave actionable feedback, just like you’d want in your own feedback from a supervisor, things like “change the tone, preserve this requirement, get rid of this claim”. 

The next version AI spat out would make the fixes, and then drop something else I had already corrected two rounds ago.

After ample frustration I had to accept I wasn’t editing prose anymore; I was straight up regression-testing it. That was exhausting.

People often review AI outputs as though each draft gets steadily better, since that’s how we operate as professionals. The truth is, it’s not ready to work that way consistently. 

Many tools out there regenerate from scratch vs truly taking compounding feedback and editing. Each time you revise, it can be a new artifact, with AI storing a new memory of that conversation that overwrites the prior, without reliably preserving the details you approved. Every revision starts things off fresh. 

If you have ever read a document for the fourth time because you no longer trust that a surgical edit was the only edit, you know how quickly the “time savings” disappear.

‍

The cost increases exponentially when the error reaches someone else

At first, I thought this was entirely a “me” problem. AI was saving me time on first drafts, but I was stuck in a constant edit loop. Annoying, but manageable, especially as I tuned my system. 

Then I had an AI help me write a meeting recap, including a recording link. Who’d have thought putting in a link would be so frustrating? 

I sent the meeting recap to a cross-functional groom with the Zoom recording link AI had dropped into the draft. It looked good, and I took my connections for granted. After the email went out, someone said they couldn’t access the recording. Turns out AI used the wrong URL, which didn’t embed a passcode directly into it.

Anyone who clicked hit a wall, and I owed the group a follow-up email with the working link.

Was this catastrophic? No. The point is that details matter. 

My time to write my own summary was cheaper than me having to apologize, figure out what was wrong with the link, and then send a new one. When a wrong claim, a bad link, or a distorted priority reaches another person, the work costs more because you now have to correct in public. This could mean confusion, lost trust, and sometimes if you aren’t careful, even a decision made on false information.

‍

This is where we need to deploy the “Definition of Done” when using our AI systems. 

If brainstorming? Sure let’s riff on what we have here. If answering questions from the head of sales? Well, the stakes got much higher. The same output may be a useful starting point in one situation, and a risky liability in another.

The easiest way to wrap my head around this has been, unfortunately for me, to do some math and develop a mental model. For whatever task I am doing, when it starts to make me frustrated, think about how long it usually takes to do it myself, and then compare that with the time I have spent on prompting, inspecting, correcting, reinspecting, and fixing whatever hasn’t held up. 

This quick mental model is something I think about at the beginning of engaging AI in a task now, especially one that requires writing. 

‍

When is the math not “mathing”?

When the math comes out negative, you should have done it yourself. That’s a fact in my story using AI. 

Here’s a great example. 

I built a detailed walkthrough and wanted to convert it from what I had to a Google Doc.

Me to my already connected to Google AI: “Ok, let’s upload this to my Google drive”

AI: *tries in machine for 20 minutes*

What happened?

Well, the browser extension could only send files from a few shared folders, and mine was not in one of those. Instead of stopping, AI kept trying to complete the directive, and things got weird.

It injected a file input into the page, spun up a local web server, faked a drag-and-drop event, hit a browser security prompt, and asked me to click, “Allow”. 

Then, FAIL. 

I dragged the file into Drive myself in about 30 seconds.

The 30-second manual version was sitting there after four workarounds and a  prayer to the Prophets of Crustafarianism (AI agent-created religion). This is the part that makes working with AI feel exhausting, even when output is “up”. 

The exhaustion isn’t that we don’t have enough stamina as human beings, or even evidence that your prompts need work. You’ve become more than just an AI-Powered product manager. You’re a QA layer for work that appears faster than it can even be read, let alone safely reviewed. 

What I’ve learned, and how I’ve changed because of it

1. Define done  before anything is generated.
   ‍
   Checking AI output is work, and personally, I lose the plot easily past around 2 pm. That is when the bad link or the inflated risk gets through. I fixed this by being extra clear. Not "write a status update," but what has to be true for the update to be useful to me. These facts have to survive; source claims from this document, not that one.
   
   For anything new, I write the context myself before asking for a draft. For standing jobs, I have skills, context docs, etc. that I lean on to speed me up. Deciding what matters is the part of the job where we still need our product sense.
   
   
2. The claim I most want to be true is the one I check first.
   ‍
   The PR that never existed got past me because it matched what I expected to see. That’s a dangerous pattern I’ve had to reconcile. Now, if an output contains a number, a status, a link, or a confident assertion I was already excited about, that is my cue to dig deeper and ask questions.
   
   The outputs that surprise me already have me asking for the receipts. The ones where I agree the most, that’s where I have to force myself to dig in.
   
   
3. If the AI has to remember it, it isn't a reliable control.
   ‍
   Where a rule matters, I attach a check. I keep a list of banned words, punctuation habits, and framing patterns. An automated scan runs against every draft and blocks delivery until the violations are fixed. That is not a reminder the AI reads. It is a test the output has to pass.
   
   I got here by looking back at two months of my own sessions. Clear patterns showed up, and rules were broken right after I stated them to my AI. "Always include X" is not a control if the only enforcement mechanism is hoping the next response remembers it. The moment a fix I requested two rounds ago goes missing, I run the scan again before I read another word.
   
   
4. I set a “retry limit” before I start working on something.
   ‍
   Generally, mine is three. After that, I stop and do the task myself.
   
   The next retry feels like it’s the one that’s going to work, especially when we’re this close to what we need. Deciding the limit in advance makes the decision less emotional when I am already 20 minutes in.
   
   

None of this is perfect, and slopification still creeps in. I still miss things, AI still does it’s classic “load bearing, smoking gun” setup, and mechanical checks only cover the things they’re designed to catch. But it has made the remaining review more deliberate instead of turning every task into a full night of babysitting. 

Sometimes you have to fire AI

This one is hard to accept, especially since I am lucky enough to work somewhere that encourages experimenting with AI. But sometimes, it’s not the right tool for the job.

One thing I have fired AI from is drag and drop tasks. After watching it spend 20 minutes writing code to simulate me dragging a file into Google Drive, a job I then did myself in 30 seconds, I won’t be  using AI for that again. 

The second thing I have fired it from is ever sending a message on my behalf. This article has outlined how much reading, editing, and proving is still involved in working with AI for content creation. My sound judgement as a PM is important to me, and having an unreliable digital intern send messages in my voice isn’t something I am willing to risk. 

Third, is strategic decision making. There’s a lot of literature on this, but the big through line is that product sense is clearly a person’s job, not AI’s. Sure, you can feed all the context and research to your agent, but in the end, humans know the subtleties best.

My list is still growing, but I pay closer attention to signals than I did in the past. If a believable mistake would cost me more time in correction than the first draft saves, I am not trusting AI as the author.  AI can still take the first swing at it, but I am the authoritative voice. It doesn’t get to tell me the job is finished, or that it’s finished well. That’s for me to decide.

We talk about AI as though it is a teammate: something that can multiply people, take on work, and free us up for more strategic thinking. If we want to think of these tools this way, we need to be better bosses. 

When someone is repeatedly unreliable at a job, you do not keep assigning them that job because the next attempt might finally be different; you provide actionable feedback, or change the assignment to fit their skillset. Or, ultimately you fire them.

Sometimes you need to fire AI. Sometimes you should never have hired it for that task in the first place.

The work that remains is not diminished by that conclusion. It is the part product managers have always been responsible for: deciding what problem is worth solving, defining what “done” means, validating claims, and knowing when the price of being wrong is too high.

AI may make building cheaper, but that doesn’t come without its costs.