Worked example: run a node on macOS¶
greasewood has no macOS build, on purpose. The daemon drives kernel WireGuard,
ip, and nft directly — none of which exist on macOS the way they do on
Linux (its WireGuard is userspace behind the GUI app; there is no nft). A port
would be a whole second backend. So a Mac joins a mesh the same way any
appliance would: a tiny Linux VM whose only job is to run one node.
The tool for that is Lima — headless Linux VMs driven
entirely from the command line, no window to click around in. The config below
is stripped to barely-enough Linux; limactl start builds it, one gw join
enrolls it.
Why a VM at all — and why NAT is fine¶
Two greasewood invariants decide the whole shape of this:
- The overlay address is
hash(id_pub)— completely independent of the underlay. The node's mesh address survives DHCP changes, prefix renumbering, even rebuilding the VM (as long as you keep its keys). The VM is disposable; the identity isn't. - Direct-or-fail — every granted pair needs a direct WireGuard tunnel. No relays, no hole-punch coordinator.
The second one usually forces a decision about VM networking (NAT vs bridged).
For a laptop it doesn't, because a laptop node's job is to dial out to a
reliable peer (your anchor, a server), never to be cold-dialed. greasewood pins
PersistentKeepalive = 25 on healthy peers, so the outbound tunnel stays open
through NAT indefinitely and WireGuard roaming lets the peer reply. NAT is
exactly right — no bridging, no socket_vmnet, no sudoers entry.
One wrinkle inside that decision: use vzNAT (Apple's vmnet NAT), not Lima's
default user-mode network. The default carries no IPv6 at all, and on an
IPv6-first mesh many peers advertise v6-only endpoints — undialable from a
v4-only guest. The symptom is maddeningly quiet: gw diagnose shows the grant
fine and the endpoint fine, gw watch just says no handshake. vzNAT NATs
both families (NAT66 for v6) and needs no privileges either.
The one thing NAT can't do is reach another NAT'd node directly — two nodes both behind NAT never handshake. As long as the peers this laptop talks to are themselves directly reachable (a GUA'd server, your anchor), that never comes up. If you genuinely need this node to be dialed inbound, you want bridged networking instead — a different, heavier setup — but most laptop clients don't.
The firewall is scoped to the VM
On Linux, greasewood's per-role nftables filter governs the whole host. Here it governs only the VM's interface, not macOS. For a laptop that normally runs no firewall at all, that's a reasonable trade — the node is sealed, the Mac is untouched.
Set it up¶
The short way — Homebrew installs the whole Mac side (Lima, the gw and
gw-mac commands, and the VM recipes), and gw-mac creates the VM on first
run:
The rest of this section is the same setup by hand — read it to know what the formula is doing for you, or to customize the VM.
Install Lima (brew install lima), then drop in
greasewood-node.yaml:
# greasewood-node.yaml — barely-enough Linux to run one greasewood node.
# limactl start greasewood-node.yaml # build + provision (once)
# limactl shell greasewood-node # then join, see message below
#
# See docs/macos.md for the full walkthrough.
vmType: "vz" # Apple Virtualization.framework; fast, no QEMU. (default on macOS 13+)
images:
# Debian 13 — has systemd (the greasewood@ unit needs it) and kernel WireGuard.
# Pin a real digest for reproducibility; `latest` shown for brevity.
- location: "https://cloud.debian.org/images/cloud/trixie/latest/debian-13-genericcloud-amd64.qcow2"
arch: "x86_64"
- location: "https://cloud.debian.org/images/cloud/trixie/latest/debian-13-genericcloud-arm64.qcow2"
arch: "aarch64"
cpus: 1
memory: "1GiB" # daemon idles at a fraction; 1G just keeps first-boot apt off the OOM line
disk: "4GiB"
mounts: [] # do NOT share the Mac's filesystem in — the daemon needs nothing from the host
containerd: # Lima installs containerd/nerdctl by default; a node wants none of it
system: false
user: false
networks: # NAT is right (laptop dials out, keepalive=25 holds it) — but vzNAT, not
- vzNAT: true # Lima's default user-mode net: the default has no IPv6, and v6-only peer
# endpoints would be undialable (symptom: "no handshake", nothing else wrong)
provision:
- mode: system
script: |
#!/bin/sh
set -eu
command -v gw >/dev/null 2>&1 && exit 0 # idempotent: skip on every reboot after the first
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y --no-install-recommends nftables wireguard-tools pipx
# /usr/local/bin so it matches the greasewood@ unit's ExecStart, old pipx or new.
PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install greasewood
probes:
- description: "gw installed"
script: |
#!/bin/sh
command -v gw >/dev/null 2>&1 || exit 1
hint: "greasewood didn't install — limactl shell greasewood-node sudo journalctl -b"
message: |
greasewood is installed; this node hasn't joined a mesh yet.
on your anchor: sudo gw invite --hostname {{.Name}}
then here: limactl shell {{.Name}} sudo gw join <token>
join enrolls the node AND enables greasewood@<mesh> to start at boot.
The choices that make it an appliance rather than a dev box:
| Setting | Why |
|---|---|
networks: [vzNAT] |
The NAT-is-fine decision — but Apple's NAT, not Lima's default user-mode net, which has no IPv6 and silently strands v6-only peer endpoints. |
containerd: {system: false, user: false} |
Lima installs containerd/nerdctl by default; a node wants none of it. This is most of the "not a dev box" difference. |
mounts: [] |
The node is sealed — your Mac's files aren't exposed to a root daemon. Also a faster boot. |
vmType: vz |
Apple's native hypervisor, no QEMU emulation. Fast on Apple Silicon and Intel. |
the command -v gw && exit 0 guard |
Provisioning is idempotent, so reboots skip apt and the VM comes back in seconds. |
| Debian, not Alpine | Both work — gw join installs a systemd unit on Debian, an OpenRC service on Alpine. Debian is the default here because systemd gives the daemon a kernel-enforced exec sandbox (CAP_NET_ADMIN bounding, ProtectSystem, syscall filters) that OpenRC can't; on Alpine the daemon runs as unconfined root. For the leaner, sandbox-free Alpine build see below. |
PIPX_BIN_DIR=/usr/local/bin |
Lands gw where the unit's ExecStart looks for it, on old pipx or new (no reliance on pipx install --global). |
Join the mesh¶
The VM is now running greasewood but hasn't joined anything. Tokens are short-lived and seed-bound, so you mint one on the anchor and paste it once — never bake it into the YAML.
# on your anchor
sudo gw invite --hostname macbook
# then, on the Mac
limactl shell greasewood-node sudo gw join <token>
(With the brew-installed gw shim, plain gw join <token> also names the
node like Linux would — it claims the Mac's hostname, not the VM's
Lima-internal one, unless you pass --hostname or the invite pinned a name.)
gw join enrolls the node and enables greasewood@<mesh> to start at boot
— nothing else to configure. Confirm it:
Day-to-day¶
- Start/stop the node:
limactl start greasewood-node/limactl stop greasewood-node. The VM disk (including/var/lib/greasewood) persists across stops, so the node keeps its identity and credential — no re-join on reboot. - A shell in it:
limactl shell greasewood-node(then anygwcommand withsudo). - Or skip the shell entirely: install
gw-shim.shas a Mac command (install -m 755 gw-shim.sh /opt/homebrew/bin/gw) andgw watch,gw diagnose <peer>, … work straight from the Mac terminal — nolimactl shell, nosudo(commands run as root inside the VM; typingsudo gw …out of habit is handled — the shim drops the Lima leg back to your user, since instances are per-user). - Rebuild from scratch:
limactl delete greasewood-nodethenlimactl start greasewood-node.yamlagain. This is a new node — new keys, new overlay address; revoke the old one on the anchor and join fresh. To keep the same identity across a rebuild, back up the VM's/var/lib/greasewoodand/etc/greasewood_<mesh>.tomlfirst and restore them before re-joining.
Don't lose /var/lib/greasewood
The node's directory lives there. Deleting the VM without backing it up is a
full re-enrollment — the same directory-loss caveat as any
node, just easier to trigger with a throwaway VM. limactl stop is safe;
limactl delete is not.
Reach a peer from a Mac app¶
limactl shell covers the command line, but a GUI app — a remote desktop
client, a database browser — can't type that. It also can't dial overlay
addresses directly: the mesh terminates inside the VM, macOS has no route to
it, and direct-or-fail means nothing will relay for it.
The missing piece is Lima itself: any port the guest listens on is
auto-forwarded to 127.0.0.1 on the Mac (ports ≥1024 — Lima can't bind
privileged ports on the host). So a small relay inside the VM puts a peer's
service on localhost, where any Mac app can reach it.
Ad-hoc — an ssh tunnel via Lima's own ssh config (sshd inside the VM
resolves the mesh name, so the hosts block works here
too). RDP to a peer named desktop:
ssh -F ~/.lima/greasewood-node/ssh.config lima-greasewood-node \
-N -L 3389:desktop.mymesh.internal:3389
Persistent — a socat unit inside the VM (socat ships in the genericcloud
image); Lima picks up the listener the moment it appears:
limactl shell greasewood-node -- sudo systemd-run --unit rdp-desktop \
socat TCP6-LISTEN:3389,fork,reuseaddr TCP6:desktop.mymesh.internal:3389
Point the app at localhost:3389 either way. The Mac leg never leaves
loopback; the peer leg is this node's ordinary WireGuard tunnel, so the relay
reaches exactly what the node's grants allow — it widens nothing. A hang here
is almost always a grant, not the relay: the peer must grant this node's role
that port. Stop the unit with sudo systemctl stop rdp-desktop; systemd-run
units are transient, so a VM restart clears them — re-run it, or promote it to
a real unit file if a relay should survive reboots.
Route the whole Mac into the overlay¶
The relay is the right default: one port, one peer, nothing widened. But if
you want overlay addresses and mesh names to work from every Mac app with no
per-service setup, the VM can be the Mac's gateway into the mesh — no
bridged networking, no socket_vmnet; the vzNAT link already carries
host↔guest traffic both ways.
The trick is NAT66, and it's load-bearing: WireGuard's cryptokey routing means
every peer accepts exactly one source address from this node — its own
overlay /128. Routing the Mac's traffic in unmasqueraded would be silently
dropped by every peer's WireGuard, and "fixing" that fleet-wide would mean a
subnet-routes concept that collides with the addr = hash(id_pub) invariant.
Masquerading to the node's own address sidesteps all of it: to the fleet,
the Mac is this node — same identity, same grants, enforced at each
receiving peer's input filter exactly as before.
Three small files inside the VM (forwarding + NAT66 + MSS clamp for the
1500→1420 MTU step — and note accept_ra=2, without which enabling forwarding
would silently kill the VM's own SLAAC underlay):
#!/usr/sbin/nft -f
# greasewood mac-gateway: NAT the host Mac's traffic into the overlay as this
# node. Masquerade = to the fleet, the Mac IS this node (grants apply as-is,
# enforced at each receiving peer). MSS clamp covers the 1500->1420 MTU step.
# "gw-*" matches whatever mesh this node joins — the file needs no editing.
destroy table ip6 gwmac
table ip6 gwmac {
chain post {
type nat hook postrouting priority srcnat; policy accept;
iifname "lima0" oifname "gw-*" masquerade
}
chain clamp {
type filter hook forward priority mangle; policy accept;
tcp flags syn tcp option maxseg size set rt mtu
}
}
# greasewood mac-gateway: forward Mac traffic into the overlay.
net.ipv6.conf.all.forwarding=1
# forwarding=1 makes the kernel ignore RAs, but lima0's address + default
# route COME from RA (Apple vmnet NAT66). accept_ra=2 keeps them.
net.ipv6.conf.lima0.accept_ra=2
[Unit]
Description=greasewood mac-gateway (NAT66 for the host Mac into the overlay)
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/nft -f /etc/gw-mac-gateway.nft
[Install]
WantedBy=multi-user.target
Install once (gw-mac does this automatically when it creates the VM — this
is the manual path for a VM you built yourself):
limactl cp gw-mac-gateway.nft gw-mac-gateway.sysctl.conf gw-mac-gateway.service greasewood-node:/tmp/
limactl shell greasewood-node -- sudo sh -c '
mv /tmp/gw-mac-gateway.nft /etc/ &&
mv /tmp/gw-mac-gateway.sysctl.conf /etc/sysctl.d/99-gw-mac-gateway.conf &&
mv /tmp/gw-mac-gateway.service /etc/systemd/system/ &&
chown root:root /etc/gw-mac-gateway.nft /etc/sysctl.d/99-gw-mac-gateway.conf /etc/systemd/system/gw-mac-gateway.service &&
sysctl --system >/dev/null && systemctl daemon-reload && systemctl enable --now gw-mac-gateway'
Then on the Mac, install gw-mac-net.sh as a
command:
gw-mac (short for gw-mac up) starts the VM if it's stopped, installs the
mesh /64 route via the VM, and syncs the VM's managed hosts block into the
Mac's /etc/hosts — idempotent, and it only asks for sudo when something
actually needs changing. The VM half of the setup is permanent, but macOS
routes are not files: the route dies with a Mac reboot or VM stop. So the
whole day-to-day is:
gw-mac # after a reboot, or anytime — safe to re-run
ssh gp2.mymesh.internal # any app, any port the node's grants allow
gw-mac down removes the route and stops the VM; gw-mac status shows both
layers at a glance.
Or stop thinking about it entirely — up is an idempotent reconciler, so it
can run on a timer (brew install only):
sudo gw-mac install-autostart # once: root helper + scoped sudoers rule
brew services start greasewood # runs 'gw-mac up' every 2 minutes at login
Root operations (the route, /etc/hosts) don't prompt after that: they go
through a small audited helper installed root-owned at
/usr/local/libexec/gw-mac-priv — outside the user-writable brew prefix, so
the NOPASSWD rule covers exactly that file and nothing a non-root process can
rewrite. The trade: your user can adjust the mesh route and hosts block
without a password. sudo gw-mac uninstall-autostart undoes both.
Know what you're trading:
- Per-port becomes whole-node. The relay exposed one port; this hands every Mac process the node's full identity and grant set. For a personal laptop that's the same trust shape as any mesh VPN client on the host — but it's a real widening; that's why the relay stays the default recipe.
- Outbound only. NAT66 has no inbound mappings — the Mac still can't be dialed from the mesh, which is the laptop posture anyway.
- Names go stale on the Mac. The VM's hosts block updates every reconcile;
the Mac's copy updates when you run
gw-mac. Rerun it after joins, departures, or renames (resolution only — reachability and revocation are still enforced live, at the peers).
Leaner alternative: Alpine (OpenRC)¶
If you're counting resources — an old Mac, a small SSD, or several node VMs —
Alpine is the featherweight option. gw join installs an OpenRC service
there just as automatically as it installs a systemd unit on Debian, so the
workflow is identical; only the base OS and the service commands change.
What it actually saves, and what it costs:
- Disk: ~0.8 GB less (Alpine + Python +
cryptographyis ~400–500 MB used, vs Debian's ~1.2–1.4 GB). Most of what remains is Python +cryptography, which is the same on both — you can't shrink below that floor. - RAM: inside the guest, an Alpine node idles around ~100 MB used vs
Debian's ~250 MB — systemd + journald + page cache Alpine simply doesn't
carry. What Activity Monitor shows on the Mac is a different (larger) number,
and it's the least meaningful one: the "Memory" column is physical
footprint, which tracks the high-water mark of guest pages ever touched —
Linux fills its RAM with page cache during boot alone, so the figure sits
pinned at the configured ceiling forever and never comes back down. It does
not mean that much of your RAM is occupied. Guest RAM under
Virtualization.framework is ordinary pageable memory, and macOS's compressor
quietly reclaims the cold pages (a running node typically has a third or more
of its writable pages compressed or swapped out — check with
vmmap --summary <pid>); footprint counts those reclaimed pages as if they were still resident. There's no memory balloon in the Lima/vz stack and none is needed — the compressor already does that job, guest cooperation not required. Real steady-state cost is on the order of 100–150 MB, less under host memory pressure. The ceiling is still the real lever, and Alpine's is a quarter: 256 MiB vs 1 GiB. What makes 256 safe is the recipe's in-guest swapfile — pip's install-time bursts (the VM's only hungry moment) spill to the virtual disk instead of needing RAM ceiling held in reserve for them. - The cost: OpenRC can't apply the systemd unit's exec sandbox
(
CAP_NET_ADMINbounding,ProtectSystem, syscall filters), so the daemon runs as unconfined root. For a laptop that normally runs no firewall this is a reasonable trade; it's still a real downgrade to weigh.
Use greasewood-node-alpine.yaml:
# greasewood-node-alpine.yaml — the leaner variant: Alpine + OpenRC.
# limactl start greasewood-node-alpine.yaml # build + provision (once)
# limactl shell greasewood-node-alpine # then join, see message below
#
# vs the Debian recipe (greasewood-node.yaml): ~0.8 GB less disk, a guest that
# idles ~100 MB instead of ~250 MB, and a 512MiB ceiling (vs 1GiB) bounding
# what the VM can ever pin on the host.
# The trade: OpenRC can't apply the systemd exec sandbox, so the daemon runs as
# unconfined root. See docs/macos.md.
vmType: "vz"
images:
# Official Alpine cloud images, as pinned by Lima's own alpine template
# (Lima ≥2.0 uses these, not the old alpine-lima ISOs). Alpine point-releases
# move over time; refresh the block from Lima's current pin with:
# limactl template copy template:_images/alpine-3.23 -
- location: "https://dl-cdn.alpinelinux.org/alpine/v3.23/releases/cloud/nocloud_alpine-3.23.4-x86_64-uefi-cloudinit-r0.qcow2"
arch: "x86_64"
digest: "sha512:849196e26640b33fb1f106d46cebdf341c715f7f49363564eacece8097464b6af7041618e94c4eb1a006394663252a86c0fb3158fee5d0ffb094899730679a71"
- location: "https://dl-cdn.alpinelinux.org/alpine/v3.23/releases/cloud/nocloud_alpine-3.23.4-aarch64-uefi-cloudinit-r0.qcow2"
arch: "aarch64"
digest: "sha512:737b0eb365944ffcf6da477c059c105bbde222942fd5a8380c07db088365b18dbc60a0f2824ea9a3904d57e194e0d05a7074302c3ebc104d0aecb6ddc6ffd91c"
cpus: 1
memory: "256MiB" # enough: the daemon idles ~85 MB, and the swapfile below
# (created before anything hungry runs) absorbs pip's
# install-time bursts without pinning host RAM for them
disk: "2GiB" # Alpine + Python + cryptography ~500 MB + the 512 MB swapfile
mounts: [] # sealed node — the Mac's files are not exposed to a root daemon
containerd:
system: false
user: false
networks: # NAT is right (laptop dials out, keepalive=25 holds it) — but vzNAT, not
- vzNAT: true # Lima's default user-mode net: the default has no IPv6, and v6-only peer
# endpoints would be undialable (symptom: "no handshake", nothing else wrong)
provision:
- mode: system
script: |
#!/bin/sh
set -eu
command -v gw >/dev/null 2>&1 && exit 0 # idempotent: skip on every reboot after the first
# Swap FIRST: pip installing cryptography is the one memory-hungry moment
# this VM ever has; 256 MiB RAM + this file covers it (and every upgrade).
if [ ! -f /swapfile ]; then
dd if=/dev/zero of=/swapfile bs=1M count=512 2>/dev/null
chmod 600 /swapfile && mkswap /swapfile
echo "/swapfile none swap sw 0 0" >> /etc/fstab
rc-update -q add swap boot 2>/dev/null || true
swapon -a
fi
apk add --no-cache wireguard-tools iproute2 nftables pipx
# /usr/local/bin so `gw` is on PATH for interactive use. The OpenRC service
# launches the daemon by its absolute interpreter path, so it needs no PATH.
PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install greasewood
# If pipx can't find a cryptography wheel (very old pip), add build deps:
# apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo
probes:
- description: "gw installed"
script: |
#!/bin/sh
command -v gw >/dev/null 2>&1 || exit 1
hint: "greasewood didn't install — limactl shell greasewood-node-alpine cat /var/log/messages"
message: |
greasewood is installed; this node hasn't joined a mesh yet.
on your anchor: sudo gw invite --hostname {{.Name}}
then here: limactl shell {{.Name}} sudo gw join <token>
join enrolls the node AND enables the OpenRC service (greasewood.<mesh>) at boot.
manage it with: sudo rc-service greasewood.<mesh> {status,restart}
limactl start greasewood-node-alpine.yaml
# on your anchor: sudo gw invite --hostname macbook
limactl shell greasewood-node-alpine sudo gw join <token>
The only day-to-day difference is the service command — rc-service
greasewood.<mesh> {status,restart} instead of systemctl/journalctl, and
logs land in /var/log/greasewood.<mesh>.log. Everything else (identity
survives rebuilds, limactl stop safe / delete not, the directory-loss
caveat) is the same.
One image-line chore
The YAML pins the official Alpine cloud images at the point-release Lima
itself currently pins (Lima ≥2.0 uses these, not the old alpine-lima ISOs).
Alpine point-releases move over time; refresh the images: block from
Lima's current pin with limactl template copy template:_images/alpine-3.23 -
and paste it in (digests included) before limactl start.
The Mac-app sections above work here too
Route the whole Mac carries over as-is: gw-mac detects the guest's
init system and installs the gateway as an OpenRC service
(gw-mac-gateway.initd) instead of a
systemd unit. Only the Reach a peer transient relay needs adapting —
systemd-run has no OpenRC analog, so write a small
/etc/init.d/ script (the same shape gw itself installs) or run
socat under nohup for a one-off.