Architecture Overview
OnVeris is made up of a Gateway, one or more Nodes, and the OnVeris Chat app. Understanding how these components relate to each other will help you plan your installation and diagnose any issues.
How The Components Connect
The diagram below shows how requests travel from people and applications to the models.
OnVeris Chat Applications
(browser, desktop, (OpenAI-compatible
mobile) API)
│ HTTPS │ HTTPS
└────────┬──────────┘
▼
OnVerisGateway
├── Administration pages
├── OnVeris Chat (web version)
├── Assistants, knowledge bases, tools
└── API
│ Encrypted WebSocket (port 8022)
▼
OnVerisNode (one or more)
└── AI models (GGUF, run by llama.cpp)
Requests from people and applications arrive at the Gateway over HTTPS. The Gateway signs them in and adds what the request needs — an Assistant's instructions, tools, passages from a Knowledge Base. It then sends the work to a Node that has the right model running. The Node runs the model and returns the result, which the Gateway delivers back to the caller.
OnVerisGateway
The Gateway is the front door to your OnVeris installation. It is the only component that needs to be reachable by your users and applications. It handles:
- Sign-in — People sign in with a username and password. Applications use an API Key.
- Routing — Requests are sent to a connected Node with the requested model running.
- Tools — When the AI calls a tool, the Gateway runs it (or, for tools that run on a person's own computer, asks the desktop app to), and feeds the result back to the model.
- Assistants, Knowledge Bases and Scheduled Tasks — all defined and stored on the Gateway.
- The administration pages — all configuration lives here.
- OnVeris Chat for the web — the Gateway serves the browser version of OnVeris Chat at
/chat.
Conversations, documents and Knowledge Bases are stored on the Gateway.
OnVerisNode
The Node runs the models. It has no API of its own for people or applications — it talks only to Gateways. Each Node:
- Finds GGUF model files in its models folder, and downloads new ones when an administrator asks.
- Runs each model it is asked to start in its own process, using its GPUs where it can.
- Answers inference requests from the Gateway.
- Reports hardware figures (CPU, GPU, RAM, VRAM) back to the Gateway in real time.
A Node can serve several Gateways at once.
OnVeris Chat
OnVeris Chat is the app people use to talk to the AI. The same app runs in a browser — served by the Gateway — and on Windows, macOS, Linux, iPhone, iPad and Android. Every version connects to the Gateway over HTTPS. No AI model runs in the app itself.
The desktop app can also run some tools on the person's own computer, such as tools that work with their files.
Deployment Patterns
You can arrange the Gateway and Nodes in several ways, depending on your hardware and how many people will use OnVeris.
Single machine Gateway and Node installed together on one server. This is the simplest setup, and is appropriate when one machine has enough hardware for both roles.
Separate machines Gateway on one server, one or more Nodes on other machines. This is useful when your inference hardware (a machine with GPUs, for example) is separate from the server you want to expose to your network.
Multiple Nodes One Gateway with several Nodes connected. Requests are spread across the Nodes running each model. This is useful for serving more people at once, or for running different models on different machines.
Several Gateways sharing a Node For example, a test Gateway and a production Gateway using the same GPU server.