4 questions before rolling out AI in your company

Most AI projects get approved out of enthusiasm and cancelled out of surprise. Someone shows a demo that works, the room lights up, and it gets the green light. Months later a failure nobody had foreseen shows up and the project is cancelled in a hurry. Between those two moments, four questions were missing. If you have an AI proposal on the table and you are not technical, these are the four questions you should ask before approving anything: what happens if it gets it wrong, what measures limit the damage, whether the benefit is worth the risk, and who takes responsibility.

You do not need to understand how the technology works under the hood to make this decision. You need to weigh risk and return, which is exactly what you already do with any other investment.

Why a demo is not a decision

A demo shows the best case, not the average case or the worst. The vendor prepares the examples that go well and presents them in a controlled setting. Your business runs in the real world, with customers who write in odd ways, incomplete data, and situations nobody prepared for.

One detail changes the whole evaluation: generative AI does not fail like normal software. A traditional program, given the same input, always returns the same output. If it works today, it works the same tomorrow. AI systems like the ones behind a text assistant are different. Given the same question they can return different answers, and sometimes they invent an answer that sounds perfectly believable but is false. That invention with the appearance of truth is called a hallucination. It is not a bug you patch. It is a property of the system, and you have to manage it, not eliminate it.

That is why a demo is not enough. The question is not “does it work?” but “what happens when it does not work, because it will?”. The four questions that follow take you from the excitement of the demo to a decision with judgment. And that decision has three possible outcomes: approve, approve with conditions, or reject.

Question 1: What happens if it gets it wrong?

The first thing is not how often it fails, but how much each failure costs. A system that is right almost every time is excellent for suggesting products and catastrophic for approving loans. The same error rate, opposite consequences. What decides is the cost of being wrong, not the frequency.

To measure that cost, ask yourself three concrete questions. Is the error reversible or irreversible? A wrong recommendation can be corrected; an offensive email sent to ten thousand customers cannot. Who suffers it? It is not the same for an internal employee to notice it as for a customer to suffer it or for it to trigger a legal penalty. Is it caught in time or does it go unnoticed? A visible error gets stopped; a silent one piles up.

Impact level	Example	What it requires before approval
Low	Drafting an internal document that someone reviews anyway	Little oversight. Good candidate to automate
Medium	Answering frequent customer questions	Sampling review and a path to escalate to a person
High	Decisions about money, health, contracts, or personal data	Mandatory human review on every sensitive case

If you want to go deeper into how to score the impact of each use case, we develop it in the risk analysis of an AI project.

Question 2: What measures minimize the impact?

Once you know what is at stake, the next question is what controls put a limit on the damage. In safety we talk about guardrails: a good system does not trust the driver to stay on the road, it puts up a barrier so that, if they leave it, they do not fall off the cliff. In AI the guardrail is any measure that reduces the consequences of a failure.

The most useful ones do not require any technical knowledge to demand them:

• A person reviews before the result takes effect. In anything that affects a customer or money, the system proposes and a person approves.
• The scope is bounded. You start with a small, low-impact case, not the whole process at once.
• You can roll back. If something goes wrong, there is a button to undo it and switch it off without drama.
• There is a record. You know what the system decided and why, so you can audit it later.

The goal of these measures is that the worst possible case is bearable. If, with the guardrails in place, the worst scenario is still unacceptable, no demo fixes that. The general framework for these controls is in the guide to AI risks for companies.

One new concept every week

Subscribe

Question 3: Is the risk worth it?

This is where many projects fall, and rightly so. The real cost of an AI system is almost never the license they show you in the proposal. On top of that figure you have to add the integration with your systems, the ongoing oversight, the maintenance when the vendor changes the model, and the time of the people who review the results.

That last point is the one that produces the most surprises. If a use case requires a person to review every answer, the savings from automating can evaporate into the cost of supervising. Sometimes it is more expensive to watch the machine than to do the work directly. It is not a failure of the technology, it is a case where it does not fit.

The honest comparison is this: expected benefit against total cost plus the expected cost of the failures. If the benefit does not comfortably exceed both of those combined, the answer is no, and saying no in time is as valid a business decision as saying yes. AI does not have to be everywhere. It has to be where the return justifies the risk. This criterion of when to automate and when not to is exactly what we work through with real cases in the IA sin hype course.

Question 4: Who takes responsibility?

Responsibility is not delegated to the vendor or to the system. If your company makes a decision backed by AI and that decision harms someone, the one who answers for it is your company. The vendor sells you a tool; the use you make of it and its consequences are yours. “The AI said so” is not a defense before a customer, a judge, or a regulator.

This has two sides. The reputational one: if the system treats a customer badly, the customer gets angry at your brand, not at the model’s maker. And the legal one, which in Europe is worth looking at closely. The General Data Protection Regulation makes clear that responsibility over personal data rests with whoever decides how it is used, that is, your company as the data controller. And the European Union’s Artificial Intelligence Regulation, known as the AI Act, classifies systems by their level of risk and demands more obligations the higher that risk is, with reinforced requirements for uses considered high-risk.

Before approving, put in writing who the internal owner of the system is, what legal obligations apply to your case, and who answers if something goes wrong. Question 4 in detail, with the map of responsibilities, is in legal responsibility for AI in the company.

This is not legal advice. For specific obligations in your sector, consult a legal professional.

The verdict: approve, condition, or reject

The four answers do not add up, they combine into one of three decisions. Approve when the impact of an error is low, there are reasonable controls, and the return pays off. Condition when the project makes sense but only with mandatory control measures: human review, bounded scope, or a pilot with a review date. Reject when the worst case is unacceptable, when the cost of supervising eats the benefit, or when nobody wants to sign as the owner.

The goal of this framework is not to hold AI back. It is to approve it with your eyes open, knowing what you control and what you do not. Most failed projects did not fail because of the technology. They failed because nobody asked these four questions in time.

Checklist before approving an AI project

• I know what happens if the system gets it wrong and who suffers that error
• The worst possible error is reversible or bounded
• There is human review on every sensitive decision before it takes effect
• The initial scope is a small case, not the whole process at once
• I have calculated the total cost, including oversight, not just the license
• The benefit comfortably exceeds the cost plus the risk of the failures
• There is an internal person who signs as the owner of the system
• I have reviewed the data and AI legal obligations that apply to my case

Preguntas frecuentes

Do I need to know about technology to evaluate an AI project?

No. Evaluating an AI project is a decision about risk and return, not a technical exam. The four questions before rolling out AI, what happens if it gets it wrong, how you limit the damage, whether it pays off, and who answers for it, are answered in business language. The technical part is the job of the vendor or your team; the decision is yours.

What if the vendor guarantees that the system does not fail?

Be suspicious. Generative AI systems can give plausible but incorrect answers, what is called a hallucination, and that behavior is not fully eliminated. A serious vendor explains how the error is managed and what controls they put in place, not that it does not exist. A guarantee of zero failures is a warning sign, not a reassurance.

How long should a trial run before deciding?

Long enough to see the system fail with real data and situations, not just the prepared examples. A useful trial has a bounded scope, a review date, and clear success and failure criteria defined before starting. A pilot that drags on without a decision is usually a way of postponing a no.

Does AI remove my legal responsibility?

No. Responsibility over the data and over the decisions stays with your company, not the vendor or the system. In Europe, you are the controller of the personal data, and the European Union’s AI framework adds obligations according to the risk level of the use. This is not legal advice; for your specific case, consult a professional.

When is the right answer simply no?

When the worst possible error is unacceptable and you cannot bound it, when the cost of supervising the system eats the savings, or when nobody in your organization wants to sign as the owner. Rejecting an AI project in time is as valid and as professional a decision as approving the one that does pay off.