Sign inSign up

deserializeme/pxeless

By deserializeme

Updated over 2 years ago

An automated system install tool for when PXE is not an option, or is not an option yet.

Image
4

10K+

deserializeme/pxeless repository overview

What is PXEless? GitHub Release

It's an automated system install and image-creation tool for situations where provisioning machines via a PXE server is not an option, or is not an option yet. It's ideal for small-scale greenfielding, proofs-of-concept, and general management of on-prem compute infrastructure in a cloud-native way without the cloud.

PXEless is based on covertsh/ubuntu-autoinstall-generator, and generates a customized Ubuntu auto-intstall ISO. This is accomplished by using cloud-init and Ubuntu's Ubiquity installer - specifically the server variant known as Subiquity, which itself wraps Curtin.

PXEless works by:

  1. Downloading the ISO of your choice - a daily build, or a release.
  2. Extracting the EFI, MBR, and File-System from the ISO
  3. Adding some kernel command line parameters
  4. Adding customised autoinstall and cloud-init configuration files
  5. Copying arbitrary files to / running scripts against the squashfs (Optional)
  6. Repacking the data into a new ISO.

The resulting product is a fully-automated Ubuntu installer. This serves as an easy stepping-off point for configuration-management tooling like Ansible, Puppet, and Chef or personalization tools like jessebot/onboardme. Please note that while similar in schema, the Autoinstall and Cloud-Init portions of the user-data file do not mix. The user-data key marks the transition from autoinstall to cloud-init syntax as seen HERE

Application Flow

Diagram showing the flow of information through the PXEless process. 1. Downloading the ISO. 2. Extracting the EFI, MBR, and File-System from the ISO. 3. Adding some kernel command line parameters. 4. Adding customised autoinstall and cloud-init configuration files. 5.Repacking the data into a new ISO.

Docker Quickstart

It is advised to run PXEless in a docker container due to it's reliance on Linux-only packages.

Skip 1 and 2 if you already have a cloud-init file

  1. Clone the rpos

    git clone https://github.com/cloudymax/pxeless.git
    
  2. Change directory to the root of the repo

    cd pxeless
    
  3. Run in a Docker container:

    • Basic Usage:

      docker run --rm --volume "$(pwd):/data" \
        --user $(id -u):$(id -g) deserializeme/pxeless \
        --all-in-one  \
        --user-data user-data.basic \
        --code-name jammy \
        --use-release-iso 
      
    • Adding static files to the ISO

      Take note that we do not specify a user here. Adding extra files to the ISO via the -x or --extra-files flag requires root access in order to chroot the squashfs.

      The contents of the extras directory will be copied to the /media dir of the image's filesystem. The extra-files are mounted as /data/<directory> when running in a docker conatiner because we mount $(pwd) as /data/

      docker run --rm --volume "$(pwd):/data" deserializeme/pxeless \
        --all-in-one \
        --user-data user-data.basic \
        --code-name jammy \
        --use-release-iso \
        --extra-files /data/extras
      
    • Offline Installation

      Running an offline installer script to customize image during build procedure. Adding a bash script as content of the extras directory. The script should be passed to image-create using -o or --offline-installer.

         docker run --rm --volume "$(pwd):/data" deserializeme/pxeless \
          --all-in-one \
          --user-data user-data.basic \
          --code-name jammy \
          --use-release-iso \
          --extra-files /data/extras \
          --offline-installer installer-sample.sh
      
  4. Writing your ISO to a USB drive

    • On MacOS I reccommend using Etcher

    • On Linux use dd.

      # /dev/sdb is assumed for the sake of the example
      
      export IMAGE_FILE="ubuntu-autoinstall.iso"
      
      sudo fdisk -l |grep "Disk /dev/"
      
      export DISK_NAME="/dev/sdb"
      
      sudo umount "$DISK_NAME"
      
      sudo dd bs=4M if=$IMAGE_FILE of="$DISK_NAME" status=progress oflag=sync
      
  5. Boot your ISO file on a physical machine for VM and log-in. If you used my user-data.basic file the user is vmadmin, and the password is password. You can create your own credentials by running mkpasswd --method=SHA-512 --rounds=4096 as documented on THIS page at line 49.

Command-line options

ShortLongDescription
-h--helpPrint this help and exit
-v--verbosePrint script debug info
-n--code-nameThe Code Name of the Ubuntu release to download (bionic, focal, jammy etc...)
-a--all-in-oneBake user-data and meta-data into the generated ISO. By default you will need to boot systems with a CIDATA volume attached containing your autoinstall user-data and meta-data files. For more information see: https://ubuntu.com/server/docs/install/autoinstall-quickstart
-e--use-hwe-kernelForce the generated ISO to boot using the hardware enablement (HWE) kernel. Not supported by early Ubuntu 20.04 release ISOs.
-u--user-dataPath to user-data file. Required if using -a
-m--meta-dataPath to meta-data file. Will be an empty file if not specified and using the -a flag. You may read more about providing a meta-data file HERE
-x--extra-filesSpecifies a folder with files and folders, which will be copied into the root of the iso image. If not set, nothing is copied. Requires use of --privileged flag when running in docker
-k--no-verifyDisable GPG verification of the source ISO file. By default SHA256SUMS- and SHA256SUMS-.gpg files in the script directory will be used to verify the authenticity and integrity of the source ISO file. If they are not present the latest daily SHA256SUMS will be downloaded and saved in the script directory. The Ubuntu signing key will be downloaded and saved in a new keyring in the script directory.
-o--offline-installerRun a bash script to customize image, including install packages and configuration. It should be used with -x, and the bash script should be avilable in the same extras directory.
-r--use-release-isoUse the current release ISO instead of the daily ISO. The file will be used if it already exists.
-s--sourceSource ISO file. By default the latest daily ISO for Ubuntu 20.04 will be downloaded and saved as script directory/ubuntu-original-current date.iso That file will be used by default if it already exists.
-t--timeoutSet the GRUB timeout. Defaults to 30.
-d--destinationDestination ISO file. By default script directory/ubuntu-autoinstall-current date.iso will be created, overwriting any existing file.

Sources

This project is made possible through the open-source work of the following authors and many others. Thank you all for sharing your time, effort, and knowledge freely with us. You are the giants upon whos shoulders we stand. :heart:

ReferenceAuthorDescription
ubuntu-autoinstall-generatorcovertshThe original project that PXEless is based off of. If the original author ever becomes active again, I would love to merge these changes back.
Ubuntu Autoinstall DocsCanonicalOfficial documentation for the Ubuntu Autoinstall process
Cloud-Init DocsCanonicalThe official docs for the Cloud-Init project
How-To: Make Ubuntu Autoinstall ISO with Cloud-initDr Donald KinghornA great walkthrough of how to manually create an AutoInstall USB drive using Cloud-Init on Ubuntu 20.04
My Magical Adventure with Cloud-InitXe IasoExcellent practical example of how to manipulate cloud-init's execution order by specifying module order
Basic user-data exampleCloudymaxA very basic user-data file that will provision a user with a password
Advanced user-data exampleCloudymax

Need something different?

PXEless currently only supports creating ISO's using Ubuntu Server (Focal and Jammy). Users who's needs ae not met by PXEless may find these other FOSS projects useful:

Project NameDescription
TinkerbellA flexible bare metal provisioning engine. Open-sourced by the folks @equinixmetal; currently a sandbox project in the CNCF
Metal³Bare Metal Host Provisioning for Kubernetes and preferred starting point for Cluster API
Metal-as-a-ServiceTreat physical servers like virtual machines in the cloud. MAAS turns your bare metal into an elastic cloud-like resource
PackerA tool for creating identical machine images for multiple platforms from a single source configuration.
Clonezilla Live!A partition or disk clone tool similar to Norton Ghost®. It saves and restores only used blocks in hard drive. Two types of Clonezilla are available, Clonezilla live and Clonezilla SE (Server Edition)

Testing your ISO with QEMU

Click to expand

You will need to have a VNC client (tigerVNC or Remmina etc...) installed as well as the following packages:

    sudo apt-get install -y qemu-kvm \
        bridge-utils \
        virtinst\
        ovmf \
        qemu-utils \
        cloud-image-utils \
        ubuntu-drivers-common \
        whois \
        git \
        guestfs-tools
  • You will need to replace my host IP (192.168.50.100) with your own.
  • Also change the path to the ISO file to match your system.
  • I have also set this VM to forward ssh over port 1234 instead of 22, feel free to change that as well.
  1. Do fresh clone of the pxeless repo

  2. Create the iso with

    docker run --rm --volume "$(pwd):/data" --user $(id -u):$(id -g) deserializeme/pxeless -a -u user-data.basic -n jammy
    
  3. Create a virtual disk with

    qemu-img create -f qcow2 hdd.img 8G
    
  4. Create a test VM to boot the ISO files with

    sudo qemu-system-x86_64 -machine accel=kvm,type=q35 \
    -cpu host,kvm=off,hv_vendor_id=null \
    -smp 2,sockets=1,cores=1,threads=2,maxcpus=2 \
    -m 2G \
    -cdrom /home/max/repos/pxeless/ubuntu-autoinstall.iso \
    -object iothread,id=io1 \
    -device virtio-blk-pci,drive=disk0,iothread=io1 \
    -drive if=none,id=disk0,cache=none,format=qcow2,aio=threads,file=hdd.img \
    -netdev user,id=network0,hostfwd=tcp::1234-:22 \
    -device virtio-net-pci,netdev=network0 \
    -serial stdio -vga virtio -parallel none \
    -bios /usr/share/ovmf/OVMF.fd \
    -usbdevice tablet \
    -vnc 192.168.50.100:0
    
  5. Select "Try or install Ubuntu" from the grub pop-up Screenshot 2022-12-29 at 06 57 01

  6. Connect to the VM using VNC so we can watch the grub process run.

    Screenshot 2022-12-29 at 07 01 06
  7. After the install process completes and the VM reboots, select the "Boot from next volume" grub option to prevent installing again

    Screenshot 2022-12-29 at 06 58 50
  8. I was then able to log into he machine using vmadmin and password for the credentials

    Screenshot 2022-12-29 at 07 00 01
  9. Finally i tried to SSH to the machine (since the vm I created is using SLIRP networking I have to reach it via a forwarded port)

    Screenshot 2022-12-29 at 07 05 58

The most common issues I run into with this process are improperly formatted yaml in the user-data file, and errors in the process of burning the ISO to a USB drive.

In those cases, the machine will perform a partial install but instead of seeing pxeless login: as the machine name at login it will still say ubuntu login:.

Contributors
License

MIT license.

This spin-off project adds support for eltorito + GPT images required for Ubuntu 20.10 and newer. It also keeps support for the now depricated isolinux + MBR image type. In addition, the process is dockerized to make it possible to run on Mac/Windows hosts in addition to Linux. Automated builds via github actions have also been created.

Tag summary

Content type

Image

Digest

sha256:84e9560bd

Size

40.8 MB

Last updated

over 2 years ago

docker pull deserializeme/pxeless