afsharidevops/hermes-smart-router repository overview

⁠Hermes Smart Router

Self-hosted intelligent model routing, visual AI orchestration, governance, RAG, observability, and controlled AI operations for OpenAI-compatible infrastructure.

Hermes Smart Router sits between AI clients and an upstream OpenAI-compatible gateway. It handles model=auto routing, policy, provider health, retry/fallback, Knowledge/RAG, agents/workflows, visual graph orchestration, governance, and operator telemetry.

It is the routing and operations layer of Hermes Linux Stack and is shared across the supported 9router and OmniRoute branches.

⁠Current release

Hermes Smart Router 0.5.9

docker pull afsharidevops/hermes-smart-router:0.5.9

Rolling deployments may use:

docker pull afsharidevops/hermes-smart-router:latest

For reproducible deployments, pin the versioned tag or immutable OCI digest:

sha256:aae4f021cc11b774e4eb8e3afdd26c318d62930ba4ee233e598bb6c8a021206e

Supported runtime platforms:

linux/amd64
linux/arm64

0.5.9 and latest were published from the same release build.

⁠What Hermes provides

• capability-aware model=auto routing;
• explicit-model passthrough;
• provider health and circuit-state tracking;
• retry/fallback and load-balancing foundations;
• visual Router Pipelines;
• visual Workflow, Agent, and Knowledge Pipeline studios;
• port-aware graph persistence;
• request traces, latency, token, and cost telemetry;
• budgets, quotas, users, virtual keys, Groups, and ACLs;
• guardrail foundations;
• lexical, vector, and hybrid Knowledge/RAG;
• PostgreSQL + pgvector foundations;
• Memory, Agents, Teams, and Workflows;
• Skills, Plugins, Marketplace, prompts, datasets, and evaluations;
• Model Catalog and provider inventory;
• light/dark Operations Center and Flight Deck;
• optional separately approved Linux/Docker/SSH execution through Hermes Execution Broker.

⁠What is new in 0.5.9

⁠Visual Flow Connections

v0.5.9 adds a shared port-aware graph engine across four studios:

Workflow Studio
Agent Studio
Router Pipeline Studio
Knowledge Pipeline Studio

The editor supports:

• output-to-input drag connections;
• live connection previews;
• valid/invalid target states;
• typed ports;
• named branch outputs;
• edge selection and deletion;
• edge reconnection;
• drop-on-empty quick add;
• auto-connect for newly created nodes;
• undo/redo;
• pan, zoom, and fit-to-view;
• save/reload persistence;
• backward normalization of v0.5.8 graph edges.

Named branches include patterns such as:

Condition:
  true
  false

Approval:
  approved
  rejected
  timeout

Health:
  healthy
  unhealthy

Router graphs can represent retry/fallback and classifier paths without turning visual graph structure into execution authority.

⁠Agent graph persistence

Agent Studio now persists its graph representation while preserving managed Knowledge, Skill, and Plugin attachments.

Existing agents can synthesize a compatible graph when no saved graph exists.

⁠Router graph/runtime integration

Router Pipeline graphs persist named branches and derive runtime stage/transition definitions while preserving legacy pipeline compatibility.

⁠Knowledge typed pipelines

Knowledge Pipeline Studio validates typed direction across stages such as:

Source
  -> Extract
  -> Transform
  -> Chunk
  -> Embed
  -> Index
  -> Knowledge Base

Invalid reversed connections, duplicate edges, incompatible ports, and invalid graph structures are rejected.

⁠Reliability fix

v0.5.9 also fixes direct Users, Policies, and Plugins creation paths that could return HTTP 500 when audit logging accessed expired SQLAlchemy rows after the database session closed.

⁠Routing behavior

Automatic routing starts with:

model=auto

Conceptually:

Client
  |
  v
Hermes Smart Router
  |
  +-- model=auto
  |     |
  |     +-- authentication / ACL
  |     +-- guardrails
  |     +-- Knowledge/RAG
  |     +-- classification
  |     +-- capability checks
  |     +-- provider health
  |     +-- Router Pipeline / policy
  |     +-- cost / latency scoring
  |     +-- retry / fallback
  |     |
  |     v
  |   selected route
  |
  +-- explicit model
        |
        v
      pass through

Explicit upstream model requests remain explicit.

Typical routing profiles include:

FAST
STANDARD
STRONG
CODING
VISION

Routing modes:

observe
route

Policy modes:

heuristic
calibrated
learned

⁠Quick start

Generate secrets:

export SMART_ROUTER_SECRET="$(openssl rand -hex 32)"
export SMART_ROUTER_ADMIN_KEY="$(openssl rand -hex 32)"
export SMART_ROUTER_CLIENT_KEY="$(openssl rand -hex 32)"

Run:

docker run -d \
  --name hermes-smart-router \
  --restart unless-stopped \
  -p 127.0.0.1:8787:8080 \
  -v hermes-smart-router-data:/data \
  --add-host=host.docker.internal:host-gateway \
  -e SMART_ROUTER_UPSTREAM_BASE_URL=http://host.docker.internal:4000/v1 \
  -e SMART_ROUTER_HMAC_SECRET="$SMART_ROUTER_SECRET" \
  -e SMART_ROUTER_ADMIN_API_KEY="$SMART_ROUTER_ADMIN_KEY" \
  -e SMART_ROUTER_CLIENT_API_KEY="$SMART_ROUTER_CLIENT_KEY" \
  -e SMART_ROUTER_REQUIRE_AUTH=true \
  -e SMART_ROUTER_CONTROL_PLANE_ENABLED=true \
  -e SMART_ROUTER_DASHBOARD_ENABLED=true \
  -e SMART_ROUTER_PROVIDER_HEALTH_ENABLED=true \
  -e SMART_ROUTER_MODE=route \
  -e SMART_ROUTER_POLICY=heuristic \
  -e SMART_ROUTER_RAG_MODE=hybrid \
  -e SMART_ROUTER_GUARDRAILS_MODE=audit \
  afsharidevops/hermes-smart-router:0.5.9

Open:

Flight Deck:
http://127.0.0.1:8787/dashboard

Operations Center:
http://127.0.0.1:8787/control/

OpenAI-compatible API:
http://127.0.0.1:8787/v1

For remote access, put administrative interfaces behind a trusted private network or TLS reverse proxy.

⁠Docker Compose

services:
  smart-router:
    image: afsharidevops/hermes-smart-router:0.5.9
    restart: unless-stopped

    ports:
      - "127.0.0.1:8787:8080"

    extra_hosts:
      - "host.docker.internal:host-gateway"

    environment:
      SMART_ROUTER_UPSTREAM_BASE_URL: http://host.docker.internal:4000/v1
      SMART_ROUTER_HMAC_SECRET: ${SMART_ROUTER_HMAC_SECRET}
      SMART_ROUTER_ADMIN_API_KEY: ${SMART_ROUTER_ADMIN_API_KEY}
      SMART_ROUTER_CLIENT_API_KEY: ${SMART_ROUTER_CLIENT_API_KEY}
      SMART_ROUTER_REQUIRE_AUTH: "true"
      SMART_ROUTER_CONTROL_PLANE_ENABLED: "true"
      SMART_ROUTER_DASHBOARD_ENABLED: "true"
      SMART_ROUTER_PROVIDER_HEALTH_ENABLED: "true"
      SMART_ROUTER_MODE: route
      SMART_ROUTER_POLICY: heuristic
      SMART_ROUTER_RAG_MODE: hybrid
      SMART_ROUTER_GUARDRAILS_MODE: audit
      SMART_ROUTER_CONTROL_DATABASE_URL: sqlite:////data/control-v0.5.2.sqlite3

    volumes:
      - hermes-smart-router-data:/data

volumes:
  hermes-smart-router-data:

The control-v0.5.2.sqlite3 filename is intentionally retained for upgrade compatibility. Current schema state upgrades in place.

⁠Main endpoints

OpenAI-compatible API   /v1
Health                  /health
Readiness               /ready
Router information      /router/info
Prometheus metrics      /metrics
Flight Deck             /dashboard
Operations Center       /control/

⁠Knowledge / RAG

Retrieval modes:

lexical
vector
hybrid

Optional embedding configuration:

SMART_ROUTER_EMBEDDINGS_BASE_URL
SMART_ROUTER_EMBEDDINGS_MODEL
SMART_ROUTER_EMBEDDINGS_API_KEY
SMART_ROUTER_EMBEDDINGS_DIMENSIONS

For production semantic retrieval, use a real embeddings endpoint with PostgreSQL + pgvector.

SQLite remains supported for portable and single-node deployments.

⁠Guardrails

Modes:

off
audit
enforce

Guardrail foundations include prompt-injection indicators, PII indicators, deny patterns, tool allowlists, high-risk-tool confirmation policy, and traceable decisions.

Treat guardrails as one layer of defense in depth, not as a universal security boundary.

⁠Authentication and security

Common credentials include:

SMART_ROUTER_CLIENT_API_KEY
SMART_ROUTER_ADMIN_API_KEY
SMART_ROUTER_HMAC_SECRET
SMART_ROUTER_BOOTSTRAP_ADMIN_PASSWORD

Production should normally use:

SMART_ROUTER_REQUIRE_AUTH=true

Keep client and administrator credentials separate. Never publish API keys, HMAC secrets, provider credentials, database passwords, execution credentials, or signing keys.

⁠Secure execution

Privileged execution is intentionally separated from Smart Router.

Optional broker image:

afsharidevops/hermes-execution-broker:0.1.3

The separated design can support independently approved:

• local Sandbox execution;
• structured Docker operations;
• structured SSH operations.

Smart Router must not receive:

Execution Admin key
approval-signing private key
Docker socket
SSH private credentials

A visual Approval node or approved graph edge is orchestration only. It does not manufacture authorization.

Use the complete Hermes Linux Stack deployment for controlled host execution rather than mounting privileged host authority into Smart Router.

⁠Persistent data and upgrades

Persist:

/data

Do not delete the persistent data volume during a normal upgrade unless you intentionally want to reset Operations Center state.

Back up durable state before upgrades.

v0.5.9 preserves compatibility with earlier visual graphs and upgrades managed graph/control state in place.

⁠High availability

Hermes includes foundations/examples for:

• PostgreSQL;
• PostgreSQL + pgvector;
• Redis shared quotas/state;
• multiple Smart Router replicas;
• shared provider-health coordination;
• load/failover testing.

Validate HA under your own production workload before treating a deployment as production-HA certified.

⁠Supported gateway branches

Hermes Linux Stack maintains:

main                           -> 9router
hermes-omniroute-linux-stack  -> OmniRoute

Smart Router is intended to remain shared across both branches.

⁠Production recommendations

• pin afsharidevops/hermes-smart-router:0.5.9 or an immutable digest;
• enable authentication;
• persist and back up /data;
• keep Operations Center private;
• use TLS/reverse proxying for remote access;
• keep credentials outside public Compose files;
• verify provider health and request traces after deployment;
• use PostgreSQL/pgvector plus a real embeddings service for production semantic RAG;
• validate HA/load/failover behavior in the target environment;
• use the separate Execution Broker for controlled execution authority.

⁠Source

Repository:

https://github.com/Afsharidevops/hermes-linux-stack

Branches:

main
hermes-omniroute-linux-stack

Relevant source:

smart-router/
execution-broker/

⁠License

See the repository LICENSE file.

⁠Hermes Smart Router 0.5.9

Route intelligently. Build visually. Observe every decision. Govern your self-hosted AI infrastructure.

model=auto uses Smart Router policy.
Explicit models stay explicit.
Flight Deck explains routing behavior.
Operations Center manages the platform.
Visual graphs define orchestration.
Privileged execution remains isolated behind independent approval.