Skip to content

Node Overview

OnVerisNode is the inference engine of OnVeris. It runs on the machine that hosts your AI model files, manages the processes that run them, and answers the requests the Gateway sends it. You can run a Node on the same machine as the Gateway, or on separate machines with their own GPUs.

What The Node Does

The Node does the work of running models, so that the Gateway can concentrate on users, Assistants and tools. It:

  • Finds GGUF model files in its models folder, and downloads new ones when you ask
  • Runs each model you start in its own process, using the machine's GPUs where it can
  • Accepts inference requests from the Gateway over an encrypted WebSocket connection
  • Reports live hardware figures (CPU, RAM, GPU, VRAM) back to the Gateway
  • Restarts model processes if they crash

The Node has no interface of its own. All configuration and monitoring is done through the Gateway — see Managing Nodes and Managing Models.

How The Node Connects To The Gateway

The Node runs as a background service and listens for connections from the Gateway on port 8022 (WSS — encrypted WebSocket). The Gateway makes the connection, so the Node does not need to know the Gateway's address.

The Access Key

When a Gateway connects, it must present the Node's access key — a shared secret that ensures only your Gateways can use the Node. A Gateway and a Node installed together on one machine are given the same key automatically.

A Node on another machine needs to be given the Gateway's key. The installer asks for it when you install a Node on its own, and you can see or change a key at any time from the installer's Manage page. See Installing A Node On Another Machine and Access Keys.

The Node's Certificate

The Node encrypts the connection with its own certificate. The first time a Gateway connects, it records that certificate, and from then on accepts only that one. If the Node is reinstalled and creates a new certificate, the Gateway will show Certificate not trusted until an administrator accepts the new one. See Managing Nodes.

One Node, Several Gateways

A Node can serve more than one Gateway at the same time — for example, a test Gateway and a production Gateway sharing one GPU server. Every connected Gateway sees the same models and the same live figures, and changes made from one Gateway will be seen by the others.

A Node has one access key, so every Gateway that uses it must have the same key.

Several Nodes, One Gateway

A single Gateway can connect to any number of Nodes. The Gateway sends each request to a Node that has the requested model running.

You can use several Nodes when:

  • you have several machines with GPUs and want to spread the load across them
  • different machines suit different models
  • you need more capacity than one machine provides

To answer more requests at once on a single machine with several GPUs, you can also run one model as several instances. See Hardware and GPU.

The Node As A Background Service

The Node is installed as a system service named OnVerisNode. It starts automatically when the machine boots, and does not need anyone to be logged in. When it starts, it will start the models that were running when it stopped.

You can start, stop and restart the service from the installer's Manage page, on any platform. See Starting And Stopping.

You can also restart it from the operating system:

Windows — Services (services.msc), or PowerShell:

Restart-Service OnVerisNode

Linux — systemd:

sudo systemctl restart OnVerisNode

macOS — launchd, where the service is named com.parkersoftware.onverisnode:

sudo launchctl kickstart -k system/com.parkersoftware.onverisnode

Note

On a machine first installed as Optima, the service is named OptimaNode, and com.parkersoftware.optimanode on macOS. See Machines First Installed As Optima.

The Node writes its log files to the Logs folder in its data folder. See Log Files.