scottgal/mostlylucid-nmt repository overview

⁠mostlylucid-nmt Docker Images

Production-ready FastAPI service for neural machine translation with multiple model family support.

⁠Quick Links

• GitHub Repository⁠
• Full Documentation⁠
• Docker Hub⁠

⁠Available Tags

All variants are available in this single repository as different tags:

Note: All images are built from the same source code, just with different configurations and base images.

⁠Pulling Images

# Pull specific tag (always latest)
docker pull scottgal/mostlylucid-nmt:cpu        # or :latest
docker pull scottgal/mostlylucid-nmt:cpu-min
docker pull scottgal/mostlylucid-nmt:gpu
docker pull scottgal/mostlylucid-nmt:gpu-min

# Pull specific version (pinned)
docker pull scottgal/mostlylucid-nmt:cpu-20250108.143022
docker pull scottgal/mostlylucid-nmt:cpu-min-20250108.143022

# Or just run (auto-pulls if not present)
docker run scottgal/mostlylucid-nmt:gpu

⁠Versioning

Each image includes two types of tags:

• Named tags: cpu (alias: latest), cpu-min, gpu, gpu-min (always point to most recent build)
• Version tags: Immutable snapshots with datetime format YYYYMMDD.HHMMSS (e.g., cpu-20250108.143022)

All images include OCI labels with version, build date, git commit, and variant information.

⁠Supported Model Families

• Opus-MT (Helsinki-NLP): 1200+ translation pairs, best quality
• mBART50 (Facebook): 50 languages, all-to-all, single model
• M2M100 (Facebook): 100 languages, all-to-all, single model

Switch between them using MODEL_FAMILY environment variable.

⁠Quick Start

⁠latest - CPU Production Image

Best for: Production CPU deployments with preloaded models

docker run -d -p 8000:8000 scottgal/mostlylucid-nmt

⁠min - CPU Minimal Image

Best for: Volume-mapped cache, switching model families without rebuilding

docker run -d -p 8000:8000 \
  -v ./model-cache:/models \
  -e MODEL_CACHE_DIR=/models \
  -e MODEL_FAMILY=opus-mt \
  scottgal/mostlylucid-nmt:cpu-min

⁠gpu - GPU Production Image

Best for: Production GPU deployments with FP16, 10x faster

docker run -d --gpus all -p 8000:8000 \
  -e USE_GPU=true \
  -e EASYNMT_MODEL_ARGS='{"torch_dtype":"fp16"}' \
  scottgal/mostlylucid-nmt:gpu

⁠gpu-min - GPU Minimal Image

Best for: GPU with volume-mapped cache for large multilingual models

docker run -d --gpus all -p 8000:8000 \
  -v ./model-cache:/models \
  -e USE_GPU=true \
  -e MODEL_FAMILY=mbart50 \
  -e MODEL_CACHE_DIR=/models \
  -e EASYNMT_MODEL_ARGS='{"torch_dtype":"fp16"}' \
  scottgal/mostlylucid-nmt:gpu-min

⁠Features

• ✅ Multi-model family support - Opus-MT, mBART50, M2M100
• ✅ Auto-fallback - Automatically tries other model families for maximum coverage
• ✅ Production-ready - Queueing, backpressure, graceful shutdown
• ✅ GPU accelerated - CUDA 12.6, FP16/BF16 support
• ✅ EasyNMT compatible - Drop-in replacement
• ✅ Model discovery - Dynamically query available models
• ✅ Persistent cache - Volume-mapped model storage

⁠Cache Overlay on Prepack Images

Both :cpu and :gpu images include a minimal set of preloaded Opus‑MT models inside the image under /app/models. You can still map a cache directory, and it works on top of the preloaded set:

# CPU prepack with external cache overlay
docker run -d -p 8000:8000 \
  -v ./model-cache:/models \
  -e MODEL_CACHE_DIR=/models \
  scottgal/mostlylucid-nmt:cpu

Behavior:

• Preloaded models are used directly from /app/models (zero first‑request latency for those pairs).
• Any additional models or updates are downloaded into /models (your mapped volume), persisting across restarts.

Minimal images (:cpu-min, :gpu-min) do not include preloaded models. Mapping a cache directory is optional; without it, models will be re-downloaded each run.

⁠Preloaded set and how to change it

By default, prepack images preload EN<->(es,fr,de,it) for Opus‑MT at build time. This is driven by a Docker build arg:

• ARG PRELOAD_LANGS="es,fr,de,it" — legacy convenience that expands to EN<->XX (two repos per language).
• Preferred: specify explicit direction pairs via ARG PRELOAD_PAIRS, since Opus‑MT models are per-direction.

Examples:

# CPU prepack: preload exact pairs (uses smart English pivot when a direct pair is missing)
docker build -t scottgal/mostlylucid-nmt:cpu \
  --build-arg PRELOAD_PAIRS="en->de,de->en,fr->en,en->it,ja->de" .

# GPU prepack with pairs
docker build -f Dockerfile.gpu -t scottgal/mostlylucid-nmt:gpu \
  --build-arg PRELOAD_PAIRS="en->de,en->fr,fr->en" .

# Legacy: preload by language list (expands to EN<->XX per language)
docker build -t scottgal/mostlylucid-nmt:cpu \
  --build-arg PRELOAD_LANGS="es,fr,de" .

Notes:

• Smart pivot: For Opus‑MT, if a non‑English direct pair (e.g., ja->de) is not available, the preloader will fetch ja->en and en->de instead, so the runtime can pivot via English.
• Minimal images (:cpu-min, :gpu-min) preload nothing by design.

⁠Key Configuration

# Model family selection
MODEL_FAMILY=opus-mt  # or mbart50, m2m100

# Auto-fallback (enabled by default)
AUTO_MODEL_FALLBACK=1
MODEL_FALLBACK_ORDER="opus-mt,mbart50,m2m100"

# Volume-mapped cache
MODEL_CACHE_DIR=/models

# GPU optimization
USE_GPU=true
EASYNMT_MODEL_ARGS='{"torch_dtype":"fp16"}'

# Preload models (optional)
PRELOAD_MODELS="en->de,de->en,fr->en"

⁠API Endpoints

• POST /translate - Translate text (batch supported)
• GET /lang_pairs - List supported language pairs
• GET /discover/opus-mt - Discover available Opus-MT models
• GET /discover/mbart50 - List mBART50 language pairs
• GET /discover/m2m100 - List M2M100 language pairs
• GET /healthz - Health check
• GET /readyz - Readiness check
• GET /cache - Cache status

⁠Example Request

curl -X POST http://localhost:8000/translate \
  -H 'Content-Type: application/json' \
  -d '{
    "text": ["Hello world", "Machine translation is amazing"],
    "target_lang": "de",
    "source_lang": "en",
    "beam_size": 1
  }'

⁠Performance Tips

GPU:

• Use FP16: EASYNMT_MODEL_ARGS='{"torch_dtype":"fp16"}'
• Increase batch size: EASYNMT_BATCH_SIZE=64
• Single worker: WEB_CONCURRENCY=1
• Preload hot models: PRELOAD_MODELS="en->de,de->en"

CPU:

• Lower batch size: EASYNMT_BATCH_SIZE=8
• More workers: MAX_WORKERS_BACKEND=4

⁠Documentation

Full documentation: https://github.com/scottgal/mostlylucid-nmt/blob/main/README.md⁠

⁠License

MIT License - Free for personal and commercial use