booting alpine linux directly with virtualization.framework on macos monterey

this guide is for the lima-linuxboot fork, not upstream Lima. the fork adds VZ linux direct-kernel boot support for macos Monterey.

to install this fork with Homebrew:

brew tap mainnika/lima-linuxboot https://github.com/mainnika/lima-linuxboot
brew install --HEAD mainnika/lima-linuxboot/lima@2-linuxboot

it requires a full Xcode 13 or newer installation to build against a macos 13 SDK.

the problem

macos monterey has Virtualization.framework, but it does not have all of the linux virtual machine APIs that later macos versions have.

the important limitation for me was EFI boot. the regular lima VZ path expects the newer EFI and persistent machine-identifier APIs, so it normally requires macos 13 or newer.

there is another way to boot linux. Virtualization.framework can receive a linux kernel, an initramfs, and a kernel command line directly. this is enough for a normal linux VM, but it means i need to prepare these files myself.

in this post i use an Alpine cloud image and turn it into a small Lima template that boots on an Intel Mac running macos Monterey.

the plan

the Alpine image starts as qcow2 with a partition table. by using direct kernel boot we can use a raw partition image, also direct kernel boot needs the kernel and initramfs as separate files.

the final directory contains:

alpine.ext4
vmlinuz-virt
initramfs-virt

then the Lima template points to all three files:

disk image     -> alpine.ext4
kernel         -> vmlinuz-virt
initramfs      -> initramfs-virt
command line   -> root device, Alpine modules, and console

the important detail is that alpine.ext4 is not the original complete qcow2 disk. it is the root partition copied into a raw file. when VZ attaches it, linux sees that filesystem directly as /dev/vda.

extracting the raw root filesystem

i started with this Alpine Lima cloud image:

nocloud_alpine-3.23.4-x86_64-uefi-cloudinit-r0.qcow2

first i attached it through the network block device driver:

sudo modprobe nbd
sudo qemu-nbd -c /dev/nbd0 ./nocloud_alpine-3.23.4-x86_64-uefi-cloudinit-r0.qcow2
sudo blkid

blkid showed the partitions. the Alpine root filesystem was on /dev/nbd0p2, so i mounted it:

sudo mount /dev/nbd0p2 /mnt/root0

then i copied the partition itself into a raw file:

sudo dd if=/dev/nbd0p2 of=alpine.ext4 bs=4M status=progress

this gives me a plain ext4 filesystem image. it does not contain the original GPT table or the first partition. that is intentional: this file becomes the whole VZ block device, so Linux mounts it as /dev/vda.

extracting the matching kernel and initramfs

the kernel must match the root filesystem. using a random kernel from another Alpine release is an easy way to get a VM that starts but cannot mount its root filesystem.

while the image is still mounted, i copied these files from /boot:

cp /mnt/root0/boot/vmlinuz-virt .
cp /mnt/root0/boot/initramfs-virt .

the virt kernel is the useful Alpine kernel here. the initramfs contains the drivers and early userspace needed to find and mount the root filesystem.

getting the kernel command line

direct boot does not run GRUB. that means GRUB does not get a chance to supply the root device, modules, or console settings. i need to take the useful parts of the existing GRUB entry and put them into the Lima template.

i checked the image configuration with:

sudo grep "/boot/vmlinuz-virt" /mnt/root0/boot/grub/grub.cfg

the entry included:

linux /boot/vmlinuz-virt root=UUID=53a898f0-60bb-4d47-83a6-8879781b3d34 ro modules=sd-mod,usb-storage,ext4 console=ttyS0,115200n8 console=ttyAMA0,115200n8

i kept the Alpine module list:

modules=sd-mod,usb-storage,ext4

but changed the root device. because i copied the root partition into the whole raw disk image, the root filesystem is /dev/vda, not /dev/vda2 and not the original UUID:

root=/dev/vda rw

finally, VZ exposes the serial console as a virtio console. the useful console argument is:

console=hvc0

the final command line is:

root=/dev/vda rw modules=sd-mod,usb-storage,ext4 console=hvc0

after copying the root filesystem, kernel, initramfs, and GRUB settings, i disconnected the image:

sudo umount /mnt/root0
sudo qemu-nbd -d /dev/nbd0

the Lima template

i saved the three prepared files under ~/.lima/_install/. a minimal template looks like this:

vmType: vz

# VZ graphics requires macos 13. Monterey direct boot is headless.
video:
  display: none

images:
  - location: "file:///Users/you/.lima/_install/alpine.ext4"
    arch: "x86_64"
    kernel:
      location: "file:///Users/you/.lima/_install/vmlinuz-virt"
      cmdline: "root=/dev/vda rw modules=sd-mod,usb-storage,ext4 console=hvc0"
    initrd:
      location: "file:///Users/you/.lima/_install/initramfs-virt"

this example is for an Intel Mac. on Apple silicon, VZ cannot run an x86_64 guest. i need an Alpine aarch64 image and the matching aarch64 kernel and initramfs instead.

also, every fallback item in images needs its own kernel section on Monterey. otherwise Lima could select an EFI-only image after a download fallback, which Monterey cannot boot with VZ.

first boot

i started the instance with:

limactl start --name alpine vz.yaml

the useful thing to watch is the serial log:

tail -f ~/.lima/alpine/serial*.log

a successful boot includes these lines:

[    4.719734] Mounting root: ok.
...
OpenRC 0.63 is starting up Linux 6.18.22-0-virt (x86_64)
...
 * Starting lima-guestagent ... [ ok ]
 * Starting sshd ... [ ok ]

once lima-guestagent and sshd are running, Lima can finish its normal connection setup.

what changes with direct kernel boot

this setup is simple, but it changes who owns the boot configuration.

with EFI boot, GRUB inside the guest chooses the kernel. with direct boot, the Lima template chooses the kernel every time the VM starts.

that means a kernel update inside Alpine does not automatically change the next boot. to update the VM kernel, i need to extract the new matching vmlinuz-virt and initramfs-virt, update the files referenced by the template, and check the command line again. Lima caches these files in the instance directory during creation, so i also recreate the instance after changing the template artifacts.

for macos Monterey this tradeoff is acceptable. it makes the VZ driver useful on an older macos release without needing the newer EFI virtual machine APIs.