lj020326/llmfit repository overview

Find out which open-source Large Language Models (LLMs) your hardware can comfortably run. llmfit inspects your CPU, system RAM, GPU(s), VRAM, and accelerator configuration to recommend models across popular quantizations.

📊 New: benchmark & share — real numbers from your machine, better estimates for everyone. Download a model, serve it, and measure real tok/s on your hardware — then contribute the results back to the project as a PR, straight from the TUI. No gh CLI, no third-party account. Every run is saved locally first, your own measurements replace estimates in the fit table, and each merged submission ships in the next release: anyone on identical hardware gets measured ✓ numbers before they ever run a benchmark. Follow the step-by-step benchmarking guide →⁠

Previously: llmfit 1.0 — the release where the numbers became verifiable →⁠

⁠Features

• Hardware Auto-Detection: Detects CPU cores, system RAM, available discrete/integrated GPUs, VRAM, and unified memory architecture (NVIDIA CUDA, Apple Silicon, AMD ROCm, Intel OneAPI).
• Model Compatibility Engine: Analyzes model parameter counts, context lengths, and quantization formats (GGUF, AWQ, GPTQ, EXL2) to project memory footprints and tokens-per-second performance.
• Interactive TUI & Web Dashboard: Choose between a lightweight, zero-dependency terminal interface or a feature-rich web dashboard.
• REST API Endpoint: Exposes standard HTTP JSON endpoints (/api/v1/system, /api/v1/models) for integration into orchestrators, dashboards, and automated deployment pipelines.
• Multi-Platform Support: macOS (Apple Silicon & Intel), Linux (x86_64 & ARM64), and Windows (x86_64).
• Hundreds of models & providers. One command to find what runs on your hardware.

⁠Web UI & API Server

docker run -d -p 8787:8787 lj020326/llmfit web

⁠Docker Compose

services:
  # Rust Backend API Service
  llmfit-backend:
    image: lj020326/llmfit:latest
    container_name: llmfit-backend
    restart: unless-stopped
    command: ["serve", "--host", "0.0.0.0", "--port", "8787"]
    expose:
      - "8787"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8787/health"]
      interval: 15s
      timeout: 5s
      retries: 3
      start_period: 10s

  # Web UI Frontend Service
  llmfit-frontend:
    image: lj020326/llmfit:latest
    container_name: llmfit-frontend
    restart: unless-stopped
    command: ["web"]
    ports:
      - "8787:8787"
    depends_on:
      llmfit-backend:
        condition: service_healthy
    environment:
      - NODE_ENV=production
      - VITE_API_BACKEND=http://llmfit-backend:8787

⁠Github Repository

Find the github repo here⁠.