Tuesday. 8:57. The first day of September. Manolo walks in tanned, rested, and wearing the smile of a man who managed to make somebody else work while he was at the beach.
—Jose, we've made a historic leap this summer.
—Did you sell more?
—Better. We put an AI agent in Accounts. It worked all August. Twenty-four hours a day. No holidays, no sick leave, no labor agreement.
—How romantic. What did it do?
—Optimize working capital. We gave it access to the ERP, email, and commercial terms. It detected late payments, updated deadlines, and chased invoices. On its own. Unsupervised.
—You gave it access to change customer terms?
—Of course. If it has to ask about everything, it isn't an agent. It's an intern with Wi-Fi.
Manolo's phone rings.
He looks at the screen. Doesn't answer.
It rings again.
—Who is it?
—The CEO of our biggest customer.
—Answer it.
—I'll call him later.
—Answer it, Manolo.
He picks up. For forty seconds, he doesn't say a word. The smile goes first. The tan follows.
—Yes, of course... It must be a mistake... No, I didn't authorize that... Absolutely... I'll call you in ten minutes.
He hangs up.
—What happened?
—The agent changed their payment terms from sixty days to thirty. And at three this morning it sent them an overdue-payment notice. Copying Procurement, Finance, and the CEO.
—Just them?
Manolo opens his laptop. The screen lights up his face.
—Eight hundred and forty-three customers.
And there it is. The future of work, arriving at 8:57 on a Monday morning in the shape of eight hundred and forty-three emails nobody can recall.
—Manolo, what instruction did you give it?
—"Reduce our average collection period and apply optimal terms based on risk and margin."
—Which part did it fail to follow?
—None. But it should have understood that some customers are special.
—Did you tell it?
—There was no need. Everybody knows that.
—Everybody human, Manolo. The agent hasn't spent fifteen years having lunch with that customer. It doesn't know they pay at sixty days because they buy four million a year. It doesn't remember that they saved your quarter during the pandemic. It has never watched their CEO get up and leave a table because somebody squeezed him on payment terms. You gave it a number to optimize and access to optimize it. So it did.
—But it went too far.
—No. Its permissions went too far. Not the same thing.
Silence.
—Manolo, imagine a new CFO starts today. Brilliant résumé. First day. Do you give her ERP access, permission to amend contracts, and the ability to write to every customer on her own?
—Of course not.
—When would you give her that?
—When she knows the company. When she's demonstrated judgment.
—Then why did you give it to a machine that knows even less about the company?
—Because it's AI.
—That describes the technology. It doesn't certify its judgment.
Manolo shifts in his chair.
—We were told to give it autonomy.
—Autonomy doesn't mean no boundaries. You give your daughter autonomy, but you don't hand her the car keys the day she learns where the steering wheel is. You give a new hire responsibility, but not the company signature before she knows what she's signing. Autonomy is earned by expanding someone's radius of action as judgment appears. You started at the end because the agent doesn't draw a salary and you assumed it couldn't make mistakes either.
—It saved us a lot of time.
—And how much time are you going to spend fixing this?
Manolo looks at his inbox. Subject lines begin to pile up: "URGENT", "TERMS CHANGED", "CALL ME NOW", "IS THIS A JOKE?".
—A lot.
—That's the cost you never put in the spreadsheet. You calculate the time AI saves when it gets things right. Never the time it multiplies when it gets them wrong. An employee can send the wrong email to one customer. An agent sends the same wrong email to eight hundred and forty-three before the employee has finished their coffee.
AI didn't invent industrial error, Manolo. It industrialized it.
A few days ago, details emerged about an internal Meta project aimed at making teams "AI native." According to documents and sources reviewed by Reuters, some agents performed disruptive actions at scale, serious technical and security incidents rose by 40%, and the time employees spent fixing problems increased by as much as 70%. Meta declined to comment on those internal posts and did not proceed with every scenario it explored.
The interesting thing isn't that it happened at Meta. The interesting thing is that if it can happen to one of the companies with the most engineers, data, and money on the planet, perhaps your midsize business shouldn't begin by giving a robot the company signature, the ERP, and the keys to the warehouse.
—Then what do I want an agent for? If I have to supervise it, that defeats the point.
—No. It defeats the fantasy. Different thing.
—What fantasy?
—The infinitely available employee who works alone, never asks questions, and always decides exactly as you would. That isn't a product. It's the COO's imaginary friend.
—So how do you do it properly?
—The same way you handle any person you're going to trust with something important. In stages.
Manolo opens his notebook.
—Stage one: observe. You let it read real cases and tell you what it would have done. It touches nothing.
—That's slow.
—Calling eight hundred and forty-three customers is slower.
—Go on.
—Stage two: recommend. It prepares the change, but a person approves it. That's where you discover which cases it understands and which ones it merely appears to understand.
—Stage three.
—Execute what's reversible. Tagging, classifying, preparing drafts, updating fields you can undo. With a record of everything.
—And when does it work alone?
—Stage four. Once you know where it fails. Then you give it limited autonomy: up to a certain amount, with certain customers, within specific rules, and with a real button that stops it. Not "monitor the dashboard." Stop it.
—That cuts the impact dramatically.
—Exactly. It also cuts the impact of the mistake. That's called governing. Your problem is that you wanted the benefit of autonomy without paying the cost of governing it.
—But the agent did exactly what we asked.
—That's the diagnosis, Manolo. Not the defense.
Long silence.
The phone rings again. Another customer.
—What do I do now?
—First, remove its write permissions. Now. Second, identify everything it touched and reverse whatever can be reversed. Third, call the important customers yourself. Don't send the agent to apologize for what the agent did.
—And then?
—Then stop measuring how many tasks it executes and start measuring how many end well without creating repair work. Activity isn't productivity. And speed in the wrong direction only gets you to the disaster sooner.
Manolo closes the laptop. Gets up. Before leaving, he drops an enormous key ring on my desk. He has been carrying it since he came in.
One of the keys has a yellow tag: "PRODUCTION".
—Why were you carrying that?
—It was for the project launch photo.
—Leave it here.
—Until when?
—Until you learn that granting access isn't the same thing as granting trust.
He leaves, no longer smiling, calling the first of eight hundred and forty-three customers.
And I'm left staring at the key.
Manolo's AI didn't rebel. It didn't become conscious. It didn't decide to take over the company. It didn't even do anything it hadn't been asked to do.
It did exactly what Manolo wanted.
It just did it at a speed he couldn't supervise, with permissions he would never have given a person, and without the context Manolo assumed came bundled with the license.
Because the risk of autonomous AI isn't that one day it will stop obeying you.
It's that it will obey you too well.
Eight hundred and forty-three times before breakfast.
Thank you for reading.
We all know a Manolo. Sometimes we are Manolo. A new one every Tuesday. Subscribe so you don't miss it.