floci/floci-duck repository overview

⁠Floci Duck

Any Cloud. Locally.
A lightweight DuckDB-powered SQL executor sidecar for Floci

floci-duck is the query engine behind Floci⁠'s Athena and Cost and Usage Reports support. It exposes a small HTTP interface that executes SQL through DuckDB, with S3-compatible storage wired in via the httpfs extension.

Written in Rust for minimal overhead. It is a sidecar, not a standalone emulator: Floci talks to it, you normally do not.

⁠Quick Start

The bundled docker-compose.yml brings up both Floci and floci-duck:

docker compose up --build

The executor is available at http://localhost:3000.

cargo build --release
./target/release/floci-duck

On startup floci-duck runs a preflight check that installs the httpfs DuckDB extension to local disk, so every later request loads it instantly with no network download. If preflight fails, for example on a first boot with no network, a warning is logged and the extension installs on the first request that needs it.

⁠Modes

Parquet, CSV, and JSON are read and written directly from and to S3.

⁠API

⁠GET /health

Returns 200 OK with body OK.

⁠POST /query

Executes a query and returns the result rows.

{
  "status": "success",
  "rows": [
    { "id": 1, "name": "Alice", "amount": 99.5 }
  ]
}

On error, status is "error", message carries the details, and rows is omitted.

⁠POST /execute

Executes a statement with no row output. Takes the same S3 fields as /query, plus:

{
  "sql": "SELECT name, count(*) FROM users GROUP BY 1",
  "s3_endpoint": "http://floci:4566",
  "output_s3_path": "s3://results-bucket/report.csv"
}

S3 credentials in the request body are optional when the matching FLOCI_DUCK_S3_* environment variables are set.

⁠Configuration

⁠Observability

Every request carries a correlation ID, taken from the incoming x-correlation-id header or generated as a UUID v4. It is propagated through every log line for that request, including those emitted deep inside the executor, and appears as a bare value inside the span context.

curl -X POST http://localhost:3000/query \
  -H "Content-Type: application/json" \
  -H "x-correlation-id: my-trace-id-123" \
  -d '{ "sql": "SELECT 1", "s3_endpoint": "http://floci:4566" }'

⁠Testing

duck-test is the integration test CLI. It needs a running server plus jq and curl, and the aws CLI for the init, parquet, and validate suites.

make dev-infra        # bring up infrastructure and create S3 resources
./duck-test all       # run every suite
./duck-test health query
./duck-test parquet --bucket my-bucket
./duck-test all -v    # print full JSON responses

All options can be set as flags or environment variables (FLOCI_DUCK_URL, FLOCI_DUCK_S3_ENDPOINT, and so on).

⁠Resources

• Source: github.com/floci-io/floci-duck⁠
• Main project: github.com/floci-io/floci⁠
• Website: floci.io⁠
• Slack: Join the Flock⁠

⁠License

MIT. Part of the Floci⁠ ecosystem.