Sign inSign up

irfanuruchi/local-llm-service:1.0

Multi-platform
Machine learning & AI
Web servers
Databases & storage
Index digest

sha256:7c79e5aabe5a2aee52d738bf544919a316c70044d2e26d14f1ecaf0be451b42f

OS/ARCH

Compressed size

2.06 GB

Last pushed

7 months by irfanuruchi

Type

Image

Manifest digest

sha256:aec03e355544aa492f10aa241ed1027ebb1c757986c0bff9041b58dccaef96a5

Image Layers

Image Layers

1# debian.sh --arch 'arm64' out/28.74 MB
2ENV PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin0 B
3ENV LANG=C.UTF-80 B
4RUN /bin/sh -c set -eux;1.21 MB
5ENV GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D0 B
6ENV PYTHON_VERSION=3.11.140 B
7ENV PYTHON_SHA256=8d3ed8ec5c88c1c95f5e558612a725450d2452813ddad5e58fdb1a53b1209b780 B
8RUN /bin/sh -c set -eux;13.65 MB
9RUN /bin/sh -c set -eux;250 B
10CMD ["python3"]0 B
11ARG DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf0 B
12WORKDIR /app93 B
13RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c96 B
14RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c94 B
15ENV PYTHONUNBUFFERED=10 B
16ENV MODEL_PATH=/models/model.gguf0 B
17ENV MODEL_NAME=local-llm0 B
18ENV N_GPU_LAYERS=350 B
19ENV N_CTX=40960 B
20ENV CHAT_DB_PATH=/data/chat_history.db0 B
21COPY /wheels /tmp/wheels # buildkit33.12 MB
22RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c48.7 MB
23RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c2.64 MB
24RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c1.93 GB
25COPY app /app/app # buildkit4.03 KB
26COPY web /app/web # buildkit2.57 KB
27EXPOSE [8000/tcp]0 B
28VOLUME [/models /data]0 B
29CMD ["uvicorn" "app.server:app" "--host" "0.0.0.0"0 B

Command

# debian.sh --arch 'arm64' out/ 'trixie' '@1771804800'