Self-hosting an AI workspace with a local model on Ollama
· By Bee Flow · 4 min read
From a fresh Docker host to a first chat answered by a model on your own hardware: what selfhost.sh starts, which compose profiles exist, and what runs on CPU.

This walks through a Bee Flow install on one Docker host, with chat answered by an open-weight model running next to it. No API key from a model vendor is involved, and prompts stay on the machine. Whether a local model is worth it for you is covered in running AI on your own server.
You need a Linux or macOS host with Docker and the Compose plugin. The self-hosting docs give 2 cores and 4 GB of RAM as the minimum for Bee Flow itself and 8 GB as recommended. A local model needs memory of its own on top of that, and how much depends on the model you pull.
Install with the local-llm profile
git clone https://github.com/Bee-Flow/Bee-Flow.git
cd Bee-Flow
PROFILES="core local-llm" ./selfhost.sh
The script checks that Docker is running, generates fresh secrets into .env, pulls the images from ghcr.io/bee-flow (public, no login), starts the stack, waits until the server reports healthy and prints the URL with a generated admin password. Open http://localhost:5176, sign in and change that password.
Your profile choice is saved as PROFILES= in .env, so a later ./selfhost.sh --upgrade or --stop covers the same services without you typing it again.
What the stack consists of
The core profile is the part you always run:
| Service (image) | What it does |
|---|---|
postgres (pgvector/pgvector:pg15) |
All durable state, plus the vectors for knowledge-base search |
rustfs (rustfs/rustfs:1.0.0) |
S3-compatible object storage for files |
server (ghcr.io/bee-flow/server) |
The API, bound to 127.0.0.1:3001 by default |
agent-hub (ghcr.io/bee-flow/agent-hub) |
The web UI on port 5176, which also proxies the API |
browser (ghcr.io/bee-flow/pwt-runner) |
A Playwright browser runner for web tools, on its own network |
Because agent-hub already forwards /api and /auth to the server over the compose network, your reverse proxy only has to point at that one container. The API port itself stays on loopback.
Four optional profiles sit next to core: guard (personal-data detection), classify (a zero-shot topic classifier for the "is about" rule in automations), local-llm (Ollama) and analytics (Umami, for the website analytics tab). The Docker socket is not mounted unless you opt in with DOCKER_SOCKET=1, since whoever holds the socket holds the host.
Connect Ollama
The local-llm profile starts the container, and the server starts using it once you tell it where it is. Add this line to .env, then run ./selfhost.sh again so the server is recreated with it:
OLLAMA_URL=http://ollama:11434
On boot the server registers the provider by itself. The admin UI labels it "from environment", and it comes back on every restart, so removing the variable is how you disconnect it for good.
Then pull a model, either from the Add model panel in the admin UI or from a shell:
docker exec beeflow-ollama ollama pull qwen3:8b
Under Admin → AI Config → Chat Models each tier (Fast, Flow, Thinking, Deep Thinking) gets a model. Put the local one on the tier your chat uses, open a new chat, and the answer comes from your own hardware. Usage reporting counts it at zero cost. That is decided by the provider type stored in the database and never by a URL that happens to look local, so a paid endpoint cannot end up priced at zero by accident.
The container has defaults you can override in .env: OLLAMA_KEEP_ALIVE=15m keeps a model loaded between turns, OLLAMA_MAX_LOADED_MODELS=2 and OLLAMA_CONTEXT_LENGTH=8192. Its port is bound to 127.0.0.1 because Ollama has no authentication of its own.
What runs on CPU
Every service in this stack starts without a GPU. The GPU reservation for Ollama is commented out in the compose file so the profile runs on any machine, and the same file says you want a GPU for usable speed on anything above roughly 8B parameters. To use one, uncomment the deploy.resources block of the ollama service and install the NVIDIA container toolkit.
The other models are small and run on CPU by design:
- The guard runs a GLiNER model through ONNX Runtime to find names and addresses, with a default container limit of 3 CPUs and 4 GB.
- The classifier runs GLiClass with the model baked into the image, so it needs no volume and no outbound traffic.
- The server reranks knowledge-base results with an in-process cross-encoder (
Xenova/bge-reranker-base), which is on by default. - Without an embedding provider, the server falls back to
multilingual-e5-smallon CPU for knowledge-base vectors.
Other runtimes
Ollama is the easy start because it downloads models over its API. vLLM, llama.cpp, LM Studio, SGLang, LocalAI, TGI, Jan and KoboldCpp work too, all through one adapter (local.js) with one entry per runtime in localModels.js. Each has a variable of its own (VLLM_URL, LLAMACPP_URL, LMSTUDIO_URL, SGLANG_URL), and anything else that speaks the OpenAI chat completions API goes through LOCAL_LLM_URL with LOCAL_LLM_TYPE.
When the runtime runs on the host instead of in compose, localhost inside the server container is the container itself. Use http://host.docker.internal:11434 or the host's LAN address, and let the runtime listen beyond loopback (OLLAMA_HOST=0.0.0.0 for Ollama). Start llama-server with --jinja, or it does no OpenAI-style tool calling and agents lose their tools.
Where it falls short
Not every open-weight model calls tools reliably, and an agent without working tools answers in prose. The local models guide lists families that behave, and what to raise when answers get cut off because a tier asks for more tokens than the runtime's context window holds.
A core-only stack checks nothing for personal data. If people will paste customer text into it, add the guard profile; how the PII filter works covers what it does. Running it yourself and the hosted option are compared on self-host or hosted, and if Open WebUI is also on your list, Bee Flow vs Open WebUI compares the two.
If you want to talk through hardware for a larger model, or would rather have us run the stack, contact us.