We have a somewhat unique (well, probably not unique but not frequently discussed) hardware build process. We start with a request to our NOC to build a machine with certain specs. They do so, and run a couple scripts before handing it over to us. At this point, we manually undo some of the steps they did, then manually run the tail end of Foreman’s provisioning script (install EPEL, install Puppet, edit puppet.conf, service puppet start, log in to puppetmaster and `puppet ca sign`, restart puppet again). This is pretty tedious, and it’s really time consuming to set up a new machine. This really happens because we’re using Foreman wrong. Our network setup is complex enough that Foreman doesn’t really know how to reserve IPs for individual machines. We’ve also got a custom PXE system, so Foreman isn’t even the default next-host.

Today, I got tired of dealing with all this. I spent a bunch of time fixing up our Foreman boot system (it had slowly decayed over time because nothing was actually using it). I also made some changes to our PXE system so that it generates a list of hosts set to ‘Build’ in Foreman, and lets you choose which one the new machine should be. Once you’ve chosen, it reconfigures the network adapter (yay, iPXE) and chainloads to Foreman. This means that I can now setup a machine in Foreman, and tell our NOC staff to netboot the machine, and choose option X. The rest of it will be handled for them, which means quicker setups, fewer mistakes, and in general everyone is happier.

We’ve set our PXE system up so it works across our entire network, so I no longer have to get them to install a machine in one location then drag it across the datacenter to it’s final location.

Note: All the code that follows is a reconstruction of what I did earlier. My actual scripts reference internal systems rather then Foreman, so some of this might be buggy!

The process looks like this:

1) Machine boots up and grabs a generic iPXE config. This contains an option that goes to a custom PHP script. Basically, it looks like this:

#!ipxe

menu iPXE boot menu
item localboot          Local boot
item Foreman            Choose foreman options
choose --default localboot --timeout 60000 bootoption && goto ${bootoption} ||

:localboot
exit

:foreman
chain http://${next-server}/showpendinghosts.php

2) showpendinghosts.php generates a list of all the hosts that are currently in the build state in Foreman. It looks a little bit like this:

#!ipxe

:start
menu Foreman Boot Options
item --gap --	-------- Pending Hosts --------
<?php
	$db = mysql_connect();
	$host = '';
	$res = mysql_query('select * from hosts where build=1');
	while ($row = mysql_fetch_assoc($res))
	{
		$res2 = mysql_query('select * from subnets where inet_aton("'.$row['ip'].'") between inet_aton(from) and inet_aton(to)');
		$subnet = mysql_fetch_assoc($res);
		$host .= "
			:host_{$row['id']}
			set netX/ip {$row['ip']}
			set netX/gateway {$subnet['gateway']}
			set netX/netmask {$subnet['mask']}
			chain http://your_foreman_server/unattended/gPXE
		";
		echo "item host_{$row['id']}	Install {$row['name']}\n";
	}
?>

choose bootoption && goto ${bootoption} ||
echo Invalid option selected!
shell

<?php
	echo $host;
?>

Prerequsites:

1) Your DHCP server must be setup to serve iPXE instead of pxelinux.

2) Foreman must have gPXE support enabled

3) All your subnets in Foreman must have a from/to IP address range set. You can avoid this with some clever bitmask tricks with the gateway/subnet, but I have not implemented those.