beshultd/deltaglider_proxy repository overview

⁠DeltaGlider Proxy

S3-compatible proxy with transparent xdelta3 compression for versioned binary artifacts.

Clients see a standard S3 API — the proxy silently deduplicates using xdelta3 against a per-prefix reference baseline. Upload your JARs, ZIPs, firmware images, and ML models as usual; DeltaGlider stores only the differences.

⁠Key Features

• Transparent S3 API — Drop-in replacement for any S3-compatible storage. Clients never know compression is happening.
• Delta compression — Versioned binaries (.zip, .jar, .tar.gz, .whl, etc.) are stored as xdelta3 diffs against a shared reference, typically saving 80–95% storage.
• Passthrough for incompressible files — Images, video, and already-compressed formats skip delta entirely.
• Built-in web dashboard — Real-time Prometheus metrics visualization with cache health, compression stats, HTTP traffic charts, and auth monitoring.
• Reference cache — LRU cache (moka) for hot reference files with configurable size, utilization alerts, and per-response x-deltaglider-cache headers.
• SigV4 authentication — Optional AWS Signature V4 authentication with hot-reloadable credentials.
• Admin GUI — Browser-based S3 file browser with settings management, upload interface, and live metrics dashboard.
• Dual backend — Local filesystem (with xattr metadata) or upstream S3/MinIO.
• TLS support — User-provided or auto-generated self-signed certificates.
• Hot-reloadable config — Change compression ratio, cache size, credentials, and backend without restart.

⁠Quick Start

docker run -p 9000:9000 -p 9001:9001 beshultd/deltaglider_proxy

• Port 9000 — S3-compatible API
• Port 9001 — Admin GUI + metrics dashboard

⁠With persistent storage

docker run -p 9000:9000 -p 9001:9001 \
  -v /data:/data \
  -e DGP_CACHE_MB=2048 \
  beshultd/deltaglider_proxy

⁠With S3 backend (e.g., MinIO)

docker run -p 9000:9000 -p 9001:9001 \
  -e DGP_BACKEND=s3 \
  -e DGP_S3_ENDPOINT=http://minio:9000 \
  -e DGP_BE_AWS_ACCESS_KEY_ID=minioadmin \
  -e DGP_BE_AWS_SECRET_ACCESS_KEY=minioadmin \
  beshultd/deltaglider_proxy

⁠Environment Variables

⁠Endpoints

⁠Requirements

• xdelta3 must be installed in the container (included in this image)
• Filesystem backend requires xattr support (ext4, XFS, Btrfs, ZFS, APFS)

⁠Architecture

HTTP request
  → S3 handlers (GET/PUT/HEAD/DELETE/LIST)
  → SigV4 auth middleware (optional)
  → DeltaGlider engine (route, compress, cache, reconstruct)
  → Storage backend (filesystem or S3)

⁠Links

• Source: github.com/beshu-tech/deltaglider_proxy⁠
• Issues: GitHub Issues⁠
• License: See repository