Sign inSign up

irfanuruchi/local-llm-service:latest

Multi-platform
Machine learning & AI
Web servers
Databases & storage
Index digest

sha256:15563f622d79e40e01e12f842a04c4861d466fde55961e6648418aff67cb4a1a

OS/ARCH

Compressed size

2.06 GB

Last pushed

7 months by irfanuruchi

Type

Image

Manifest digest

sha256:5d3929012054643dc1e4b83316c56992dff48503988cb9614207ec77a3b2e358

Image Layers

Image Layers

1# debian.sh --arch 'arm64' out/28.74 MB
2ENV PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin0 B
3ENV LANG=C.UTF-80 B
4RUN /bin/sh -c set -eux;1.21 MB
5ENV GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D0 B
6ENV PYTHON_VERSION=3.11.150 B
7ENV PYTHON_SHA256=272179ddd9a2e41a0fc8e42e33dfbdca0b3711aa5abf372d3f2d51543d09b6250 B
8RUN /bin/sh -c set -eux;13.65 MB
9RUN /bin/sh -c set -eux;250 B
10CMD ["python3"]0 B
11ARG DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf0 B
12WORKDIR /app93 B
13RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c96 B
14RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c94 B
15ENV PYTHONUNBUFFERED=10 B
16ENV MODEL_PATH=/models/model.gguf0 B
17ENV MODEL_NAME=local-llm0 B
18ENV N_GPU_LAYERS=350 B
19ENV N_CTX=40960 B
20ENV CHAT_DB_PATH=/data/chat_history.db0 B
21COPY /wheels /tmp/wheels # buildkit33.12 MB
22RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c48.7 MB
23RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c2.64 MB
24RUN |1 DEFAULT_MODEL_URL=https://huggingface.co/second-state/Phi-3-mini-4k-instruct-GGUF/resolve/main/Phi-3-mini-4k-instruct-Q4_0.gguf /bin/sh -c1.93 GB
25COPY app /app/app # buildkit3.93 KB
26COPY web /app/web # buildkit2.53 KB
27EXPOSE [8000/tcp]0 B
28VOLUME [/models /data]0 B
29CMD ["uvicorn" "app.server:app" "--host" "0.0.0.0"0 B

Command

# debian.sh --arch 'arm64' out/ 'trixie' '@1771804800'