starlakeai/quack-on-demand repository overview

⁠Quack on Demand

Multi-tenant Arrow Flight SQL⁠ gateway for DuckDB. Per-tenant catalog, federated queries (iceberg, s3 ...), per-tenant node pools, a first-class RBAC graph (users · groups · roles · table permissions · pool grants), and pluggable identity (DB / JWT / OIDC). Single uber-jar, shipped as one image.

DuckDB's Quack⁠ protocol lets DuckDB instances talk over HTTP/2, but it is intentionally minimal: one static token, no multi-tenancy, no authorization, DuckDB-only clients. Quack on Demand is the infrastructure you put in front of it.

Any JDBC / ODBC / ADBC / PyArrow client connects to a single Flight SQL edge. The gateway authenticates the user, gates the connection at handshake (user-scope + pool-access), parses each statement and matches its table references against the user's cached EffectiveSet, then routes the statement to a compatible node in the tenant's pool.

⁠Supported tags

• latest — most recent published build
• pinned release tags (e.g. 0.2.x), set QOD_VERSION in Compose to pin

Images are multi-arch: linux/amd64 and linux/arm64. Runtime base is eclipse-temurin:17-jre.

⁠Requirements

A reachable PostgreSQL instance. It holds the control plane (qodstate_* tables in the qod database) and one DuckLake catalog database per tenant (${tenant}_${tenantDb}). The Compose file below bundles one.

⁠Quick start (Docker Compose — recommended)

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: change-me
      POSTGRES_DB: qod
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d qod"]
      interval: 5s
      timeout: 3s
      retries: 20

  quack:
    image: starlakeai/quack-on-demand:latest
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      QOD_PG_HOST: postgres
      QOD_PG_PORT: "5432"
      QOD_PG_USER: postgres
      QOD_PG_PASSWORD: change-me
      QOD_PG_DBNAME: qod
      QOD_ADMIN_USERNAME: admin
      QOD_ADMIN_PASSWORD: change-me
      PROXY_TLS_ENABLED: "false"
    ports:
      - "20900:20900"          # REST + admin UI
      - "31338:31338"          # Flight SQL edge
      - "21900-22500:21900-22500"  # per-tenant Quack node port range
    volumes:
      - ./ducklake:/app/ducklake
      - ./certs:/app/certs

docker compose up

Admin UI: http://localhost:20900/ui/⁠ (log in with the admin credentials above). The full repo ships a richer Compose stack with a seeded TPC-H tenant, S3/SeaweedFS, and a Prometheus + Grafana profile — see the project README and guides/QUICKSTART.md.

⁠Quick start (docker run)

Point it at an already-running PostgreSQL:

docker run --rm \
  -p 20900:20900 -p 31338:31338 -p 21900-22500:21900-22500 \
  -e QOD_PG_HOST=host.docker.internal \
  -e QOD_PG_PORT=5432 \
  -e QOD_PG_USER=postgres \
  -e QOD_PG_PASSWORD=change-me \
  -e QOD_PG_DBNAME=qod \
  -e QOD_ADMIN_USERNAME=admin \
  -e QOD_ADMIN_PASSWORD=change-me \
  -e PROXY_TLS_ENABLED=false \
  -v "$PWD/ducklake:/app/ducklake" \
  -v "$PWD/certs:/app/certs" \
  starlakeai/quack-on-demand:latest

⁠Ports

⁠Key configuration

Every config key is overridable via a QOD_* environment variable.

Persist ./ducklake (data files) and ./certs (TLS cert/key, generated on first boot) with volumes.

⁠Connecting a client

Each session scopes itself with tenant + pool routing parameters.

JDBC

jdbc:arrow-flight-sql://localhost:31338?useEncryption=true&disableCertificateVerification=true&user=admin&password=admin&tenant=<tenant>&pool=<pool>

ADBC (Python)

import adbc_driver_flightsql.dbapi as flight_sql
conn = flight_sql.connect(
    "grpc+tls://localhost:31338",
    db_kwargs={
        "username": "admin",
        "password": "admin",
        "adbc.flight.sql.rpc.call_header.tenant": "<tenant>",
        "adbc.flight.sql.rpc.call_header.pool": "<pool>",
        "adbc.flight.sql.client_option.tls_skip_verify": "true",
    },
)

Works with any Flight-aware client: DBeaver, PyArrow, Spark, ODBC via the Apache Arrow Flight SQL driver.

⁠Features

• Arrow Flight SQL edge with auto-generated self-signed TLS (swap in a CA-signed cert for prod)
• Pluggable auth: Postgres / JDBC (BCrypt), external JWT (HS256 / RS256 / PEM), OIDC (Keycloak, Google, Azure AD, AWS Cognito)
• First-class RBAC graph with two handshake gates (user-scope, pool-access) and per-statement table-ref matching against a cached EffectiveSet
• Multi-tenant pools of READONLY / WRITEONLY / DUAL nodes; the router classifies each statement and picks a compatible node
• Per-tenant DuckLake catalog DB — tenant isolation at the database boundary, not just row-level
• Observability built in: Prometheus /metrics, or push to CloudWatch / Azure Monitor / GCP; ships a Grafana dashboard
• Self-healing on restart: the registry is reconciled against running nodes; dead nodes are respawned before the edge accepts traffic

⁠Links

• Source & docs: https://github.com/starlake-ai/quack-on-demand⁠
• Quickstart / Running / API / Resilience guides: guides/ in the repo
• Maven Central: ai.starlake:quack-on-demand_3
• Community: Discord (invite in the repo README)

⁠License

Apache License 2.0.