Cleaner micro Kubernetes on OSX

While my main workstation is a Linux Mint machine, I occasionally use my OSX ${WORK} laptop for traveling and composing. I’m not really fond of the OS, but at least it’s an UNIX-like, and pkgin runs well with it ;)
When I’m “on the go”, I like to try things and play along with technologies I’m currently obsessed with, among them Kubernetes.
On OSX, the natural choice is to go with minikube, it’s kind of integrated and does the job well, but if you tried it already and also happen to run docker for OSX you might have found yourself struggling with versions and consistency between the two. Added to this that I wanted to have a fully functional Linux virtual machine, preferably Debian GNU/Linux, there was way too much inconsistencies and wasted disk and CPU space to come. So I dug by myself and found a clean and fast solution by spawning my own virtual machine using OSX native hypervisor, which runs Canonical’s microk8s, a nicely done snap package to install a fully working and modular Kubernetes cluster on a Linux machine.

First thing is obviously to create the virtual machine.

If you search how to install Debian on OSX’s native hypervisor, you’ll probably find very well put blogs and tutorials speaking about xhyve, a fork of FreeBSD’s bhyve for OSX, Here are two good resources on how to achieve this:

• Install Debian on xhyve(OS X)
• lloeki’s Gist

Unfortunately, this was a working solution for Debian until version 8, but now when trying to install version 9 with it, the Debian installer kernel panics with a missing init message.
In order to fix this, you will need to use HyperKit, an xhyve fork maintained by Moby, the container infrastructure. Here’s what the modified xhyve script look like:

# Installer kernel and initrd are here:
# http://ftp.debian.org/debian/dists/stretch/main/installer-amd64/current/images/netboot/debian-installer/amd64/
KERNEL="linux"
INITRD="initrd.gz"
CMDLINE="earlyprintk=serial console=ttyS0"
ISO="debian-9.7.0-amd64-netinst.iso"

MEM="-m 1G"
#SMP="-c 2"
NET="-s 2:0,virtio-net"
IMG_CD="-s 3,ahci-cd,$ISO"
IMG_HDD="-s 4,virtio-blk,debian.img" # raw disk image
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
ACPI="-A"
UUID="-U 549293DC-EA12-441C-B285-C64BA7A48BF4"

sudo hyperkit $ACPI $MEM $SMP $PCI_DEV $LPC_DEV $NET $IMG_CD $IMG_HDD $IMG_CD $UUID -f kexec,$KERNEL,$INITRD,"$CMDLINE"

As you can see, only the xhyve command was changed to hyperkit, which is 100% backward compatible. Apart from that detail, you can follow the previous procedures to proceed with installation and startup, notably vmlinuz and initrd extraction before reboot.

Once the Debian installation is complete and your usual tools installed, you may proceed with microk8s installation, as described on the official website. Do not forget, like I did, to add /snap/bin to the PATH, otherwise the microk8s.* commands won’t be automatically found.

$ microk8s.kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   167m

If you’d like to manage the cluster directly from OSX, export its configuration using:

$ microk8s.kubectl config view --raw

Copy the exported content to OSX’s ~/.kube/config, and then:

$ kubectl get all
NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   5h

And voila! here’s a brand new micro-Kubernetes cluster installed on your laptop, without the hassle of handling multiple specific virtual machines tailored for every use case.