Published: 28 June 2026 · Roy Morken, Datafolka

AI governance for European businesses: a practical framework for using AI without leaking data

AI governance is how a business decides where AI may be used, what data it may touch, and who is accountable when it goes wrong. For European businesses the question stopped being optional in 2026. The EU AI Act became broadly applicable on 2 August 2026, GDPR already governs every byte of personal data your AI tools see, and staff are pasting customer data into free chatbots faster than any policy can keep up. This article lays out a practical framework an SMB can actually run.

The hard part is rarely the law in the abstract. It is the gap between a free ChatGPT tab open on every desk and the contracts, customer records, and personal data those tabs quietly receive. Good AI governance closes that gap without banning AI outright, because a blanket ban just pushes usage underground. The goal is to match each AI tool to the sensitivity of the data it handles, and to keep the sensitive cases inside a boundary you can document.

Everything here is based on the EU AI Act, GDPR, the Schrems II ruling, and public guidance from Datatilsynet, the Norwegian Data Protection Authority. No anonymised client cases and no "we have seen" anecdotes. Just the rules and the practical engineering of keeping AI useful and compliant at the same time.

What AI governance actually means for an SMB

AI governance is how an organisation sets the rules for AI use and decides who owns them. It is the same discipline as IT governance or data governance, applied to AI tools. For a company of five to fifty people, it does not need to be a thick framework. It needs to answer four questions clearly.

• Where is AI used? Every tool, including the ones staff adopted on their own.
• What data goes into it? Public copy, internal notes, customer data, or special-category data.
• Which rules apply? GDPR for the personal data, the EU AI Act for the AI system itself.
• Who is accountable? One named person who owns the policy and the review.

That is the whole of it for most SMBs. AI compliance, the question of whether a specific use meets a specific legal rule, falls out of governance as a by-product. When every tool is classified and documented before it goes into use, compliance stops being a fire drill the day a customer or Datatilsynet asks.

The two rulebooks: GDPR and the EU AI Act

Two separate bodies of law sit over business AI in Europe, and they answer different questions. GDPR governs the personal data your AI processes. The EU AI Act governs the AI system itself, based on what it does. You need both, and confusing them is a common mistake.

The EU AI Act entered into force on 1 August 2024 and phases in over several years. It sorts AI systems into risk tiers, and the obligations follow the tier. Here is the timeline that matters for a business deciding what it must do and when.

For most SMBs the practical reading is simple. You are almost never a provider of a high-risk AI system, so the heaviest obligations do not land on you. You must avoid the prohibited practices, which is easy because no normal business uses them. And you must meet the transparency duty: tell people when they are talking to AI or reading AI-generated content. The fines are what give the Act teeth, up to 35 million euro or 7 percent of global annual turnover for prohibited use, and up to 15 million euro or 3 percent for most other breaches.

GDPR is the rulebook that bites more often for AI. The moment an AI tool processes a customer name, an email, or a contract, it processes personal data, and the full weight of GDPR applies: a legal basis, a data processing agreement with the vendor, and a valid transfer basis if the data leaves the European Economic Area. That last point is where AI compliance most often breaks.

Where AI compliance actually breaks: the data leaves Europe

The most common AI compliance failure in European SMBs is mundane: an employee pastes customer data into a free US chatbot. Datatilsynet has put it plainly. When you paste text into ChatGPT, you send information to OpenAI servers in the US, which is a transfer of personal data to a third country and requires a legal basis under GDPR. Establishing that basis is far from trivial.

The reason is the Schrems II ruling of 2020, in which the Court of Justice of the EU struck down the EU-US Privacy Shield. US law, including FISA 702 and the CLOUD Act, lets US authorities demand data held by US companies in a way that conflicts with GDPR. The replacement, the EU-US Data Privacy Framework from 2023, gives a basis for transfers to certified US companies, but it sits under legal challenge and could fall as its two predecessors did.

For AI specifically, this hits hardest in regulated sectors. Law firms, accountants, and healthcare providers handle exactly the data that GDPR guards most tightly, and they are the ones most tempted to drop a sensitive document into a chatbot to save an hour. AI governance is the mechanism that catches this before it happens, by classifying the data and routing the sensitive cases away from public tools.

The practical answer: keep the data in-house with a private LLM

For sensitive use cases, the cleanest governance answer is to remove the transfer question rather than paper over it. A private LLM is a language model dedicated to your organisation, where your prompts are never shared or used to train anyone else. A self-hosted LLM goes further: the model runs on infrastructure you control, so the data never leaves a boundary you can document. A local LLM means the same thing run on local or on-premise hardware.

This is not exotic any more. Open-weight models such as Llama, Mistral, and the Norwegian NorMistral are free to run on your own servers without asking a US vendor for permission. The inference layer is typically vLLM or Ollama. For ten to fifty concurrent users, a single NVIDIA H100 instance is usually enough. The quality gap with public ChatGPT has closed to the point where everyday tasks like drafting and document analysis run comfortably on a self-hosted model.

The hosting choice carries half the governance weight. A model is only as private as the infrastructure under it. If a US-owned cloud runs the server, the CLOUD Act question returns even when the data sits physically in Europe. For Norwegian businesses that means hosting with a Norwegian provider under Norwegian jurisdiction, such as Terakraft, Bulk Infrastructure, or Green Mountain. Datafolka builds and operates private AI on Norwegian servers for exactly this reason: the benefits of AI without sending customer data outside the EEA.

Self-hosted AI is not the right answer for everything. If you only draft marketing copy or write spreadsheet formulas, a public tool gives better value. The point of governance is to make that call per use case, not once for the whole company.

Public AI versus private or self-hosted AI

The choice is not all-or-nothing. Most businesses run both, matching the tool to the data. This is how the two models compare on the points that matter for governance.

The legal difference looks small on paper. The practical difference is large. With a self-hosted model you do not have to persuade a compliance officer, a board, or Datatilsynet that the risk is acceptable. You have documentation showing the risk is not there.

A six-step AI governance framework

The six steps below take a typical SMB from no policy to a documented, reviewable setup in a few days of work spread over a couple of weeks. Each step is listed in full in the how-to summary at the top of this page.

1. Map where AI is already used, including tools staff adopted on their own.
2. Classify each use by data sensitivity and EU AI Act risk tier.
3. Write a one-page AI use policy that names the tools and the rules.
4. Choose a deployment model per use case: public for low-sensitivity, private or self-hosted for the rest.
5. Document the legal basis and processing for each tool that touches personal data.
6. Review on a schedule and whenever a tool or the law changes.

The work is mostly mapping and decisions, not technology. The one technical step, standing up a private LLM, only applies to the use cases your classification flags as sensitive.

Next steps

Start with the mapping in step one. Most SMBs are surprised by how much AI is already in use and by how much of it touches data that should not leave the building. Once you have the map, the policy and the deployment decisions follow quickly. The data processing agreements for your AI vendors are part of the same exercise, and our guide to data processing agreements covers the eight clauses each one must contain.

If you want a sparring partner to run an AI governance and data review over one to two days, Datafolka can help. It is usually a combination of IT advisory and IT security assessment : we map your AI use, classify it by data sensitivity, and set up a private LLM for the cases that need one.

Send an email to Roy.Morken@Datafolka.no , and we will arrange a call.

Related reading on datafolka.no: Private AI without data leakage , Datatilsynet 2026 for small businesses and data processing agreements for SMBs .

Sources

• EU AI Act, regulatory framework and timeline (European Commission): digital-strategy.ec.europa.eu/regulatory-framework-ai
• EU AI Act implementation timeline and risk tiers: artificialintelligenceact.eu/implementation-timeline
• GDPR Articles 28, 30 and Chapter 5 (transfers): lovdata.no/personopplysningsloven
• Datatilsynet on AI and personal data: datatilsynet.no/kunstig-intelligens
• Schrems II (C-311/18) and the EU-US Data Privacy Framework: commission.europa.eu/eu-us-data-transfers

Roy Morken, co-founder of Datafolka. This article is built on the EU AI Act, GDPR, the Schrems II ruling, and Datatilsynet's public guidance on AI, translated into practical language for European SMBs. No customer data or anonymised cases have been used. Last updated 28 June 2026.

Claude AI helped me with phrasing and proofreading in this article.