How to give AI your company's data: context and tools

The first time an executive opens ChatGPT and asks it about their own sales, they get a letdown: the AI has no idea about their company. And it is right not to. A generative AI model knows absolutely nothing about your business. The only thing it knows in each conversation is what you put in front of it. That is the context, and understanding it clears up half the questions a decision-maker has about this technology.

In this article I explain, without jargon, what AI really knows about your company, why it doesn’t remember yesterday, and the three real ways to give it your data. None of them is magic, and each one has its cost and its risk.

What does AI really know about your company?

Nothing. A language model (LLM, for large language model: the engine behind ChatGPT, Gemini or Claude) is trained by reading an enormous amount of public text from the internet. It learns to write well, to reason, to summarize. But your margins, your clients, or the email your sales rep sent yesterday were never in that public text, so the model has never seen them.

There is also a time limit worth knowing about: the cutoff date. It is the point up to which the model read data during its training. Anything that happened afterward, the model ignores. If you ask about recent news or a product you launched last month, it doesn’t know it, even though it speaks with total confidence. That confidence without knowledge is exactly one of the reasons AI gets things wrong.

The practical takeaway: if you want AI to work with your data, you have to give it to it yourself. It is not going to guess it.

What is context and why is it the only thing that matters?

Context is all the information the model has in front of it at the moment of answering. It is its workbench. Whatever is on the bench, it uses. Whatever is not there, for the model does not exist.

Think of a new employee with an unusual kind of amnesia: brilliant and knows about almost everything, but every morning shows up remembering nothing of what you discussed yesterday. So each day, so the work gets done, you leave on the desk the folder with what is needed: the order, the client’s terms, the returns policy. With that folder in front of them, they work wonderfully. Without it, they improvise. AI works the same way. The context is that folder.

This explains why the same tool gives excellent results at one company and disappointing ones at another. It depends less on the model than on what you put in front of it. Good context turns a generic answer into a useful answer for your case. If you want the foundation underneath, I cover it in the guide on how generative AI works.

Why doesn’t it remember what I told it yesterday?

By default, AI keeps no memory between conversations. Each new chat starts from scratch, with an empty bench. You explained in detail yesterday how your business works, and today it doesn’t remember a thing.

This surprises a lot of people and has real business consequences. It means you cannot “train” AI by chatting with it for a while: the moment you close the conversation, that knowledge is lost. Some products add a memory layer on top that jots things down and puts them back on the bench next time, but that is a feature of the product, not of the model. When someone sells you an AI that “learns from your company”, ask exactly what it stores, where, and who can see it. Often the answer is simpler and more fragile than the pitch promises.

The three ways to give it your data

There are three ways to put your company’s information onto that workbench. They go from least to most automatic, and from least to most setup effort.

Paste it by hand. You copy the document, the email, or the table into the conversation and ask it to work with that. It is instant and requires no setup. It works for one-off tasks. The problem is that little fits (the bench has a size limit) and it doesn’t hold up if you have to do it twenty times a day.

Let it search your documents. You connect the AI to a store of documents (contracts, manuals, product sheets) and, when you ask, the system first searches for the relevant fragments and puts them on the model’s bench before answering. This is RAG, which in plain terms means “search before answering”. It is what lets an assistant answer about your manuals without you pasting anything. It requires setup and maintenance, but it scales well.

Give it tools. Instead of only text, you give the AI permission to use tools: search your website, read a system, query a database, send an email. When an AI can use tools to act, it stops being a chat and becomes an agent. It is the most powerful and also the most delicate, because now it doesn’t just read: it does things.

Way	What it is	When it makes sense	Main risk
Paste by hand	You copy the information into the chat	One-off tasks, no setup	Little fits; doesn’t scale
Automatic search (RAG)	The system searches your documents and gives them to the model	Repeated queries over your own documentation	If it searches poorly, it answers poorly
Agent with tools	The AI uses tools to read systems and act	Automating tasks that touch several systems	Poorly scoped access, unwanted actions

An “agent” without the hype

An agent is simply an AI with permission to use tools. Nothing more. You give it the keys to certain doors (your catalogue, your calendar, your order system) and it decides when to open them to finish what you asked. The word sounds like science fiction, but the idea is just that: model plus permissions.

Here is the nuance they don’t tell you in the demo. In the demonstration everything goes perfectly because the scenario is set up. At your company, with real data and odd cases, the agent will sometimes open the wrong door or do something you didn’t expect. Giving it access to a system where it only reads is very different from giving it access to a system where it can modify or delete. A flawless demo is no guarantee of production, and confusing the two is expensive.

Deciding what it can access is security, not technical

The moment you connect AI to your systems, it stops being a technical decision and becomes a security decision. The question is no longer “can the AI read this?” but “who is exposed if it reads, copies, or shares it where it shouldn’t?”.

The sensible rule is least access: the AI should only be able to see and touch what is strictly necessary for the task, and nothing more. If an agent that answers customer questions also has access to payroll, you have a problem waiting to happen, even if it works today. Also, when that data includes personal information about clients or employees, the GDPR comes into play, the European data protection regulation. Sending personal data to a model, especially one from an external provider, has legal implications worth reviewing beforehand, not afterward. What exactly happens to the data you send to a third-party model is something I develop in third-party data and AI models. This is not legal advice: check it with whoever handles data protection in your organization.

This criterion (what data goes in, who accesses it, what the agent can do) is exactly the kind of decision we work through calmly in the AI without the hype course, so you know what to ask before signing anything.

One new concept every week

Subscribe

What to ask whoever implements it for you

When someone proposes to “connect AI to your company’s data”, these questions separate a serious proposal from a smoke-and-mirrors sale:

• How do you give it the data: pasted, searched in documents, or with tools?
• Which systems does the agent access exactly, and can it only read or also modify?
• What personal data is sent to the model and under what legal basis (GDPR)?
• Does the model’s provider store or reuse what we send?
• What happens when the AI gets it wrong, and who reviews it before it reaches the customer?
• Is what works in the demo tested with our real data?

If the answer to any of these is vague, you don’t yet have a proposal ready for production.

Frequently asked questions

Does AI keep my data?

It depends on the provider and the plan you sign up for. Some services use what you send to improve their models unless you turn it off; others, especially enterprise plans, commit not to. It is not a minor detail: before sending sensitive information, read the specific terms of the service and, if there is personal data, review how it fits with the GDPR.

Does AI learn from what I give it in the chat?

Not permanently. Within a single conversation it uses what you have given it, but once you close it, it doesn’t retain it by default. You cannot “teach” it your business by chatting. For it to always work with your data, you need to set up one of the three ways described above, not repeat it every day.

Do I need my own AI model to use my data?

Almost never. The vast majority of companies work with existing providers’ models and give them context with their data, without training anything of their own. Building a custom model is expensive and rarely pays off. The real work is not in the model, but in giving it good context and properly scoping what it can access.

What is the model’s cutoff date?

It is the date up to which the model read information during its training. It knows nothing after that point, nor anything that was private. That is why, for current or internal company data, you always have to give it to it yourself in the context: the model on its own does not have it.

Does giving context to AI comply with the GDPR?

It can, but not automatically. The moment you send personal data about clients or employees to a model, data protection rules apply: minimize what you send, have a legal basis, and know where that data ends up. It is not an insurmountable obstacle, but it is a decision to make with awareness and, if needed, with legal advice.