Running AI on your own server: when it makes sense and what it takes
· By Bee Flow · 3 min read
Hosting the software yourself is not hard. Running the language model yourself asks for a graphics card and some patience. And there is a middle road: open models in European data centres.

Picture a small law firm of about twelve people. They want to use AI to summarise case files and draft letters. The partners do not like the idea of those files going to a data centre in the United States, and one of them asks whether it can simply run here, on a server of their own.
It can. But "on your own server" means two different things, and it helps to keep them apart.
The software and the model
The first is the software: the chat, the knowledge bases, the conversations and the users. You can host Bee Flow yourself with Docker Compose and one install script. It needs no special hardware. Our documentation gives 2 processor cores and 4 GB of memory as the minimum and recommends 4 cores and 8 GB. Your conversations and documents then sit in your own database.
The second is the language model, the part that writes the answers. That can run on your own hardware too, with open models through programs such as Ollama, vLLM or llama.cpp. A question then never leaves your network. Self-hosting with Ollama shows the install step by step.
You can do the first without the second. Host the software yourself and send questions to a model in the cloud, with Privacy Shield in between to hold back personal data.
What a local model asks of your hardware
This is where the real work is. Our documentation says that models above roughly 8 billion parameters need a graphics card to reach a usable speed. Smaller models can do without. When a model does not fit in the card's memory, everything slows down a great deal. As a reasonable starting point for a set of mid-sized open models, the documentation assumes a single card with 24 GB of memory.
You have to buy that card, and it has to sit somewhere it is cooled and looked after. In our experience an open model you run yourself is also less capable than the largest cloud models. For summarising, translating, drafting an e-mail or asking questions of your own documents you will hardly notice. For hard reasoning you will.
The bill looks different. A model on your own hardware counts as zero euros in Bee Flow's usage overview, because you pay nothing per question. You do pay for the card and the electricity.
When it pays off
A local model is worth it when you work with data that really must not leave the building, such as medical records or documents under professional secrecy. Also when your installation has to keep working without an internet connection. And sometimes simply because usage is so high that the cost per question starts to add up.
For most small companies we would not start there.
The middle road: models hosted in Europe
Between your own hardware and an American cloud service there is a third option: open models in European data centres. Bee Flow works with Scaleway (France and the Netherlands), IONOS (Germany), Mistral's European service and EU GPT, compared in EU-hosted LLM providers. These are often the same open models you would otherwise run yourself. You buy no hardware and pay for what you use, while your questions stay in the EU.
You can mix as well. In Bee Flow you choose per level which model does the work. A small local model for short jobs, such as coming up with a title for a conversation, and a larger model in Europe for the heavy work is a setup that works well.
How to start
Start with the software on your own server and a model in Europe. Watch for a few weeks what your team does with it. If some work turns out to involve data that really has to stay inside, add a local model for that work only.
If you would rather not manage the server yourself, we can run Bee Flow for you in European data centres. The product is the same; only who runs it differs.
What each setup involves is on self-host or EU cloud. The chat itself is on private AI chat, and if you are weighing Open WebUI, Bee Flow vs Open WebUI compares the two. Questions about your situation can go to contact.