unfor19/nexus-ops repository overview

⁠nexus-ops

Provision Nexus Repository Manager⁠ (NXRM) Docker container.

All scripts are written in Bash⁠, and the Nexus REST API⁠ calls are made with curl⁠. Check the provision/entrypoint.sh⁠ to learn more about the provisioning process.

⁠Goals

• Hands-on experience with Nexus Repository Manager
• Provisioning a ready-to-go Nexus Repository Manager container
• Use Nexus Repository Manager as part of a CI/CD workflow

⁠Requirements

• Hardware
  • CPU⁠: Minimum 4, Recommended 8+
  • Memory⁠: Minimum 8GB, Recommended 16GB+
• Docker⁠ (Windows users should use WSL2 backend⁠)
• Set Nexus's Repository as a trusted Docker repository (insecure registry⁠) - Either edit $HOME/.docker/config.json or the Docker Engine, and then restart the Docker Daemon.

  "insecure-registries": [
     "localhost:8081"
  ],
  

  Copy
  

⁠Quick Start

The provisioning script provision/entrypoint.sh⁠ performs the following tasks

1. Changes the initial random password⁠ that is in /nexus-data/admin.password to admin
2. Enables anonymous access⁠ - allows anonymous users to access localhost:8081 with READ permissions
3. Adds Docker Bearer Token Realm⁠ - allows anonymous pulls from local Nexus registry localhost:8081
4. Creates two Docker repository of type proxy⁠
   1. docker-hub - DockerHub
   2. docker-ecrpublic - AWS ECR Public
5. Creates a Docker repository of type group⁠
   1. docker-group - The above Docker repositories are members of this Docker group

⁠Docker Run

# ulimit - https://help.sonatype.com/repomanager3/installation/system-requirements#SystemRequirements-Docker
# 8081 - Nexus
# 8082 - docker-group
docker run -d \
   --ulimit nofile=65536:65536 \
   -p 8081:8081 \
   -p 8082:8082 \
   --name nexus "unfor19/nexus-ops"

⁠How It Works

Nexus's Repository will serve as a "cache server", here's the logic -

1. Each docker pull goes through localhost:8082/repository/docker-group and gets the and then redirected to the relevant docker-proxy.
   • DockerHub - http://localhost:8081/repository/docker-hub/v2/
   • AWS ECR Public - http://localhost:8081/repository/docker-ecrpublic/v2/
2. If the Docker image exists, Docker Client pulls it from NXRM
   • It is recommended to re-tag the image from localhost:8082/nginx to nginx, so the references in your Dockerfile can remain nginx.
3. If the Docker image doesn't exist, NXRM pulls it from DockerHub and saves it in Nexus's Repository. Following that, the Docker Client pulls the image from NXRM.

⁠Run Nexus Locally (UI)

This is the exact same process that was done in the Quick Start section, only now we're going to do it manually in the UI. The purpose of this section is to help Nexus newbies (like me) to get familiar with the UI.

⁠Test Locally

1. Pull relevant Docker images via localhost:8082

   docker pull localhost:8082/unfor19/alpine-ci:latest && \
   docker pull localhost:8082/ubuntu:20.04
   

   Copy
2. Tag relevant images with docker.io or omit the prefix (like I did), that will mitigate the need to rename images to localhost:8082

   docker tag localhost:8082/unfor19/alpine-ci:latest unfor19/alpine-ci:latest && \
   docker tag localhost:8082/ubuntu:20.04 ubuntu:20.04
   

   Copy
3. docker build - build the application; the Docker Daemon already pulled the required images to build the app.

   git clone https://github.com/unfor19/nexus-ops.git
   cd nexus-ops
   docker build -f Dockerfile.example -t unfor19/nexus-ops:example .
   

   Copy
4. test image

   docker run --rm unfor19/nexus-ops:example
   # Miacis, the primitive ancestor of cats, was a small, tree-living creature of the late Eocene period, some 45 to 50 million years ago.
   
   # ^^ A random cat fact for each build
   

   Copy

⁠Workflow (CI/CD)

I've added the GitHub Action⁠ - docker-release.yml⁠. If you check it out, you'll see the following code snippet⁠

jobs:
  docker:
    name: docker
    runs-on: linux-self-hosted # <-- A label that I chose

As implied from the label self-hosted, I intend to run this workflow on my local machine. Adding your local machine as a self-hosted runner is quite simple.

1. GitHub Repository > Settings > Actions > Runners
2. Add Runner > Operating System: Linux (or any OS), Architecture: X64
3. Follow the steps in Download
4. Follow the steps in Configure > Follow the prompts and add a custom label when prompted

   Enter any additional labels (ex. label-1,label-2): linux-self-hosted
   

   Copy
   A successful configuration should look like this.

   ~/actions-runner $ ./config.sh --url https://github.com/unfor19/nexus-ops --token YOUR_TOKEN
   ...
   √ Settings Saved.
   
   ~/actions-runner $ ./run.sh
   
   √ Connected to GitHub
   
   2021-06-01 21:18:04Z: Listening for Jobs   
   

   Copy

Go on and initiate a workflow; add a file, commit and push.

touch some-file                   && \
git add some-file                 && \
git commit -m "added some file"   && \
git push

⁠Pull From Different Repositories

So far, the examples showed how to use DockerHub, though the process is the same for AWS ECR or any other container registry.

• Pull from DockerHub - nginx:1.19-alpine⁠

  docker pull localhost:8082/nginx:1.19-alpine
  

  Copy

• Pull from ECR - nginx/nginx:1.19-alpine⁠

  docker pull localhost:8082/nginx/nginx:1.19-alpine
  

  Copy

⁠Same Tag Precedence

But what happens if both repositories hold images with the same tags? owner/repository:tag

We'll use bitnami/kubectl:latest which is named the same both in DockerHub⁠ and AWS ECR Public⁠

docker pull localhost:8082/bitnami/kubectl:latest

Navigate to http://localhost:8081/service/rest/repository/browse/docker-hub/v2/bitnami/kubectl/tags/⁠, as the link implies, the image was pulled from DockerHub.

To change this behavior, go to the docker-group⁠'s settings and change the order of Members

After changing the order of Members -

1. Invalidate cache⁠

   

2. Remove the existing image from the local Docker Daemon cache

   docker rmi localhost:8082/bitnami/kubectl
   

   Copy

3. Pull the image again; this time, it will be from ECR

   docker pull localhost:8082/bitnami/kubectl
   

   Copy

4. Check results at http://localhost:8081/service/rest/repository/browse/docker-ecr/v2/bitnami/kubectl/tags/⁠

⁠Development

⁠Flow Of Work

1. Build the image

   docker build -t unfor19/nexus-ops .
   

   Copy
2. Run the image and mount the code

   docker run -d -it --name nexus -p 8081:8081 -p 8082:8082 -v "$PWD"/:/code/ unfor19/nexus-ops
   

   Copy
3. Exec to container

   docker exec -it nexus --workdir="/code/provision" bash
   

   Copy
4. Change the code of entrypoint.sh in local IDE (VSCode, PyCharm, etc.), and then execute the code in the container

   # Exploring current dir /code/provision
   bash-4.4$ ls
   entrypoint.sh  repositories  wait_for_endpoints.sh
   
   # After changing the code
   
   # Execute entrypoint.sh
   bash-4.4$ ./entrypoint.sh 
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Healthy endpoint - http://localhost:8081/service/rest/v1/status/writable
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Nexus API is ready to receive requests
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Password was set
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Repository exists - proxy docker-hub
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Repository exists - proxy docker-ecrpublic
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Repository exists - group docker-group
   [LOG] Sun Jun  6 22:58:46 UTC 2021 :: Finished executing - /nexus-data/nexus-ops/entrypoint.sh
   

   Copy

⁠Exploring Nexus API

The easiest way to get familiar with Nexus's REST API⁠, is to run Nexus locally and check the API page http://localhost:8081/#admin/system/api⁠.

From there, you can execute commands with Swagger UI⁠, and get the relevant curl commands. The following example demonstrates how I learned about the [GET] repositories/docker/proxy/{repositoryName} API.

The provided curl commands do not include -u admin:admin, so make sure you add this argument when you use curl. Here's an example of a curl command

curl -u admin:admin -X GET "http://localhost:8081/service/rest/v1/repositories/docker/proxy/docker-ecrpublic" -H "accept: application/json"

⁠Known Caveats

• It is required to pull all relevant images before the build step. Re-tagging the images tricks the Docker Daemon to use existing images (--pull missing). This helps to avoid hitting DockerHub for each pull, though it means you need to prepare a list of images that should be pulled.

⁠References

• Cover image - Hackerboy Emoticon⁠
• Cover image - Nexus Logo⁠
• Cover image - Docker Logo - https://draw.io⁠

⁠Authors

Created and maintained by Meir Gabay⁠

⁠License

This project is licensed under the MIT License - see the LICENSE⁠ file for details