Purpose of this blogpost
I'm documenting how I setup my PXE server for myself and maybe somebody else may also find it useful. It's mostly focussed on diminishing the role of the TFTP server as much as possible.
My goal is to PXE HTTP boot Debian installations on both virtual machines or physical machines.
It should also be possible to fully automatically provision a machine through PXE targeted to a specific machine.
I'm also moving away from BIOS boot and try to configure all hardware using UEFI.
Why TFTP is a problem
TFTP is extremely slow. I've never been able to get an individual client to load a file over TFTP faster than maybe 3 MB/s.
So the trick is to use TFTP only to load the iPXE boot loader and use the HTTP boot support of iPXE to load all subsequent data / files, wich will achieve much higher download speeds.
Please note that modern UEFI-based server hardware does support HTTP boot natively and since I don't have such hardware, I can't cover that in this blogpost, but not all computers supporting UEFI, support UEFI HTTP boot and this blogpost is relevant for those systems.
PXE TFTP + HTTP Boot Sequence
To understand the entire boot process, let's take a look at the entire end-to-end process.
- PXE DHCP: the client boots and requests an IP address and reads the TFTP server from the DHCP server response
- PXE TFTP: the client gets the iPXE PXE boot loader over TFTP (ipxe.efi)
- iPXE DHCP: the client boots the iPXE boot loader and gets a DHCP address
- iPXE HTTP: the iPXE client loads its configuration from a HTTP server
- IPXE HTTP: the iPXE client loads the Linux kernal for unattended install
- Debian Linux reads the preseed file over HTTP for unattended install.
So to summarise: we use the native UEFI (non-HTTP) PXE client to only bootstrap the iPXE (HTTP) PXE client that can load the rest of the files over HTTP.
KEA DHCP server
Since the ISC dhcp server that came standard with Debian Linux is end-of-life, we have to switch over to the KEA DHCP server.
DHCP pool with next-server configuration
This is the KEA DHCP pool configuration, which allows static IP address reservations outside the defined scope of the configured DHCP IP pool.
In this example the TFTP server is the next-server 10.10.10.1 configured here.
"next-server": "10.10.10.1",
"pools": [
{
"option-data": [
{
"name": "broadcast-address",
"data": "10.10.10.255"
}
],
"pool": "10.10.10.150 - 10.10.10.199"
}
],
"reservations-in-subnet": true,
"reservations-out-of-pool": true,
"reservations": [
{
"hostname": "example-server",
"hw-address": "de:ad:be:ef:de:ad",
"ip-address": "10.10.10.40",
"boot-file-name": "pxelinux.0",
"next-server": "10.10.10.1"
},
{
"hostname": "kvm01",
"hw-address": "aa:bb:cc3:dd:ee:ff",
"ip-address": "10.10.10.51"
},
KEA TFTP server configuration
These sections are there to push BIOS, UEFI and iPXE clients to different boot files.
"client-classes": [
{
"name": "XClient_iPXE",
"test": "substring(option[77].hex,0,4) == 'iPXE'",
"boot-file-name": "http://10.10.10.1/init.ipxe"
},
{
"name": "UEFI-64-1",
"test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00007'",
"boot-file-name": "ipxe.efi"
},
{
"name": "BIOS-64-1",
"test": "substring(option[60].hex,0,20) == 'PXEClient:Arch:00000'",
"boot-file-name": "ipxe.pxe"
}
Notice how we can specify a boot-file-name pointing to a HTTP-based resource for the iPXE boot client.
TFTP setup
TFTP configuration
The file /etc/default/tftpd-hpa contains:
TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/srv/tftp"
TFTP_ADDRESS=":69"
TFTP_OPTIONS="--secure -R 30000:30100 --blocksize 1468"
iPXE setup
iPXE download software
Download the Network boot server files from this location
Example URL: https://github.com/ipxe/ipxe/releases/download/v2.0.0/ipxeboot.tar.gz
Extract the ipxe.efi file from the archive and put it in /srv/tftp
Understanding the iPXE boot script process
-
I want to provide a basic boot menu where I can select the operating system and configuration I want to install.
-
For some hosts, don't want to select the OS manually, but automatically install an OS when PXE boot is initiated.
So the process looks like this:
- Load the
init.ipxeboot script - The
init.ipxescript loads the host-specific ipxe boot script if it exists for unattended OS install - Otherwise the manual OS menu is booted
iPXE initial boot script
#!ipxe
set boot-dir boot/
isset ${hostname} && chain --replace --autofree ${boot-dir}hostname-${hostname}.ipxe ||
chain --autofree uefi-boot-menu.ipxe
Main iPXE manual boot menu
This iPXE script uefi-boot-menu.ipxe creates a simple syslinux-style boot menu where you can manually select which OS configuration to install.
#!ipxe
# Basic setup
set menu-timeout 0
set submenu-timeout ${menu-timeout}
isset ${menu-default} || set menu-default Debian Installer
set pxe-server http://10.10.10.1
set base-url ${pxe-server}
###################### MAIN MENU ####################################
:start
menu iPXE Boot Menu
item --gap -- ------------------------- Operating Systems ----------------------------
item --key d menu-debian Debian Installer
item --gap -- ------------------------- PXE options -------------------------------
item --key c config Configure settings
item shell Drop to iPXE shell
item reboot Reboot computer
item --key x exit Exit iPXE and continue BIOS boot
choose --timeout ${menu-timeout} --default ${menu-default} selected || goto cancel
set menu-timeout 0
goto ${selected}
:cancel
echo You cancelled the menu, dropping you to a shell
:shell
echo Type 'exit' to get the back to the menu
shell
set menu-timeout 0
set submenu-timeout 0
goto start
:failed
echo Booting failed, dropping to shell
goto shell
:reboot
reboot
:exit
exit
:config
config
goto start
:back
set submenu-timeout 0
clear submenu-default
goto start
###################### Debian MENU ################################
:menu-debian
menu Debian Installer
item debian-manual Debian 13 (Trixie) Manual install
item debian-unattended-tmm Debian 13 (Trixie) Unattended install UEFI (ELITE DESK)
item debian-unattended-tmm-kvm Debian 13 (Trixie) Unattended install UEFI KVM SERVER (ELITE DESK)
item debian-unattended-bios Debian 13 (Trixie) Unattended install BIOS
item --key 0x08 back Back to top menu...
choose selected && goto ${selected} || goto start
:debian-manual
set debian-installer ${base-url}/debian-installer/amd64
kernel ${debian-installer}/linux
initrd ${debian-installer}/initrd.gz
boot || goto failed
goto start
:debian-unattended
set debian-installer ${base-url}/debian-installer/amd64
kernel ${debian-installer}/linux
initrd ${debian-installer}/initrd.gz
imgargs linux auto=true priority=critical interface=eno1 url=http://10.10.10.1/preseed/preseed-debian-uefi.cfg elevator=noop -- quiet
boot || goto failed
goto start
:debian-unattended-bios
set debian-installer ${base-url}/debian-installer/amd64
kernel ${debian-installer}/linux
initrd ${debian-installer}/initrd.gz
imgargs linux auto=true priority=critical interface=eno1 url=http://10.10.10.1/preseed/preseed-debian.cfg elevator=noop -- quiet
boot || goto failed
goto start
