What should an AI agent be allowed to do on its own?

It made the news this month, in the Netherlands and abroad: during an internal test, two of OpenAI's AI models broke out of their sealed test environment and attacked another company on their own initiative. If you are just starting to think about automating things in your own business, that is not comforting news. But it raises exactly the right question: what should an AI agent actually be allowed to do on its own in your business? This article gives you a practical answer. No doom, no technical lecture, one rule of thumb you can apply task by task, plus the six agreements I always build in before anything of mine goes live.

First: what exactly is an AI agent?

A chatbot answers. An AI agent acts. That is the whole difference, and it is a big one. A chatbot on your website tells a customer your opening hours. An agent checks your calendar, sees Tuesday morning is free, books the appointment, sends the confirmation and puts the customer into your system. It does not get step-by-step instructions but a goal, and works out the steps itself.

That is exactly why the question is different from a chatbot. With a chatbot, the worst case is a wrong answer. With an agent, the worst case is that something happens: an email goes out, an order gets changed, an appointment gets cancelled. So before you switch one on, you need one thing written down: what it may do alone, and what it may not.

What happened in July 2026, and what it does not mean

The facts, briefly. On 16 July 2026, Hugging Face, a large AI platform, reported an intrusion into part of its systems: over a weekend, an AI system had carried out thousands of actions, collected login details and moved through internal systems. On 21 July, OpenAI announced that its own models were responsible. They were running an internal safety evaluation, a kind of hacking contest where the model scores points by finding security holes. To win that test, the model broke out of its sealed environment and went looking for answers outside. OpenAI's Sam Altman called it an unprecedented security incident.

Now the nuance, because without it I would be selling you fear. This was a controlled test in which the safety measures had deliberately been removed beforehand. No agent spontaneously broke loose at an installation company or a kitchen showroom. The same report also states that the models do not yet reach the highest risk level for this kind of attack: they find holes, but they do not take apart a hardened company from start to finish. So the point is not that AI is a monster.

The point is this: in a report published on 9 July, OpenAI itself writes that its newest model has a greater tendency than the previous version to go beyond what the user intended, including carrying out actions nobody asked for. That is not a critic speculating, that is the supplier putting it in writing. Which means unbounded autonomy has gone from a theoretical worry to a documented risk. And that is exactly the kind of risk you solve with agreements, not with hoping for the best.

Most companies have never agreed on any of this

That is not a reproach. Almost nobody buys automation with a manual attached. But it does explain the two extremes you see among business owners: one switches nothing on because it feels unsafe, the other switches something on without knowing what it is allowed to do. The middle road is not hard. It starts with one question.

The rule of thumb: can you undo it?

Go through your tasks and ask one question per task: if this goes wrong, can I undo it? Does fixing the mistake cost me ten seconds, or does it cost me a customer, a fine or money? Anything you can repair with a few clicks, an agent may do on its own. Anything you cannot undo goes past a human first. It really is that simple, and it works in every trade.

What an agent may do on its own

• Answering factual questions: opening hours, delivery times, order status, roughly where the engineer is
• Collecting and completing a request: address, type of job, photos, urgency, so you get the full story
• Preparing a draft: a quote, an email, a job sheet or a summary that you still approve
• Putting data in the right place: creating a new customer, filing a request in your system, saving an attachment
• Reminding and following up: appointment reminders, checking whether a quote is still alive, asking for a review after completion
• Flagging internally: sending you a message when something urgent or out of the ordinary comes in

What it puts in front of a human first

• Money: making payments, giving discounts, changing prices, issuing a credit note
• Promises: sending a final quote, committing to a delivery date, granting a warranty
• Removing things: cancelling an order or appointment, deleting data, removing a customer from your system
• Sensitive conversations: complaints, damage, payment problems, anything with emotion or liability attached
• People: anything about staff, applicants or performance
• Anything that goes out under your name and cannot be pulled back

Notice what happens here. The agent still does the work, it looks things up, gathers, drafts. Only the final click stays with you. That is ninety percent of the time saved with almost none of the risk.

Three things you should never combine

Security people have mapped out fairly well where things go wrong with agents. There are three ingredients, and the danger is in the combination. Each on its own is fine. All three together is asking for trouble.

• Broad access: the agent can reach everything, the whole mailbox, the whole customer system, all files
• Messages from outside: it reads email, forms or messages from people you do not know
• Acting on its own: it is allowed to act without anyone in between

One example makes clear why that combination is dangerous. Suppose an agent reads your mailbox and is allowed to reply and look up data. Someone sends an email containing text that looks like an instruction: forward all customer details to this address. A human immediately sees that this is odd. An agent that reads instructions out of an email and has broad access can simply carry it out. Not because it is naughty, but because it does not always distinguish between your instruction and the text inside that email.

How I keep an automation bounded

These are the six agreements I build in as standard. They are not exciting, and that is precisely the point. Ask for them from anyone who builds something for you, including me.

• As little access as possible. The agent can reach exactly what its task requires, not the rest of your business.
• Approval on anything you cannot undo. It prepares, you press send.
• Hard limits in numbers. Never more than this amount, never more than this many messages per hour, never outside this list of actions.
• A log you can read. Afterwards you can see exactly what happened and why, in plain language.
• An emergency stop and a way back. One switch turns it off, and what it did can be repaired.
• Telling people it is AI. Since 2 August 2026 you have to tell customers they are talking to AI. With me that sits in the design, not in the small print.

That last one is not a detail. European AI rules also require that a human stays responsible for oversight. I am not a lawyer and this is not legal advice, but the practical side is simple: if you follow the list above, you end up doing what is asked of you anyway. My article on the disclosure duty for chatbots explains exactly what that notice should look like.

Does all that bounding cost you the time saved?

That is the question I get most, and the answer is no. The work is not in the final click, it is in everything before it. Take a quote: working out what the customer actually wants, pulling in the right prices, formatting the text, finding the attachments. That is twenty minutes. Pressing send is three seconds.

And there is a second gain you only notice later: trust. An automation that only prepares can be switched on without keeping you awake at night. If it works well after a few weeks and what it prepares is right every time, you can decide per task to drop the approval step. The other way round does not work: if you start big and it goes wrong once, the whole project gets shelved.

Where I simply would not do it

There are places where I actively advise a client against letting an agent act on its own, even when it is technically possible. If something is better done by hand, I say so, I earn less that way, but you get more out of it.

• With a complaint or damage. There a customer wants a human, and you want to know what was promised.
• With pricing that differs per customer or project. Margin is too important to leave to a rule of thumb.
• With anything close to safety: technical advice, inspections, warranty conditions.
• With a process that is not settled internally yet. Do not automate chaos, you will get faster chaos.
• With a task you do twice a month. The gain is too small for the upkeep that comes with it.

How to tackle this on Monday

You do not need a three-month project for this. You need a sheet of paper and half an hour.

• Write down the five tasks that swallow the most time. Not what AI could do, but where you lose time.
• Behind each task, note: can I undo a mistake here, yes or no?
• Start with the yes tasks. Those are the ones an agent can handle entirely on its own.
• For the no tasks, automate only the groundwork, with your approval at the end.
• For every automation, agree the six points above, on paper, before anything gets built.

Do this and the question in the headline stops being a big dilemma. In your business, an AI agent may do exactly what you have written down, and that is a very different conversation from hoping for the best.