I’m running all services in seperate FreeBSD’s own old, mature and trusted “containerization” called jails. On this host, these jails are managed via BastilleBSD and make use of ZFS features.

There are some jails that run in production or live mode, and some in which I test things. Most of the jails need access to the ports directory as I often need to compile my own versions with special settings. So, even as diskspace is cheap(er) nowadays, I’m rather oldshool and I think having one and only one read-only ports tree for all “machines” eases integrity, upgrading processes and general oversight.

To achieve this, I prepared special working directories. Normally when you compile a port, there’s a “working” directory inside the ports directory in which temporary files are stored. Also there are ports/distfiles and ports/packages directories. After preparing these directories, I mount the hosts port tree read-only into the jails. Then I mount the special working, distfiles and packages directories read-write into the jails. The host will accumulate a cache of distfiles, so we don’t need to download the distfiles for go or python n times in every single jail in which we need them. Maybe I’d should write a script that cleans up the distfile directory once a month or so.

preparation on the host

on the host, we’ll use ZFS for the ports tree and the directories: zfs create -p tank/build/{ports, ports-packages, ports-distfiles}

“mount” the ports tree to default /usr/ports and special dirs into /var/ports zfs set mountpoint=/usr/ports/ tank/build/ports zfs set mountpoint=/var/ports/packages tank/build/ports-packages zfs set mountpoint=/var/ports/distfiles tank/build/ports-distfiles

preparation inside the jails

mkdir -p /var/ports/{packages,distfiles}

we’ll have to tell the jail build tools to not use the default directories, instead use in /etc/make.conf WRKDIRPREFIX= /var/ports DISTDIR= /var/ports/distfiles PACKAGES= /var/ports/packages

(of course you can do this from outside the jail, too)

setting mountpoints for jail:

in /usr/local/bastille/jails/testjail/fstab:

/usr/local/bastille/releases/13.0-RELEASE /usr/local/bastille/jails/testjail/root/.bastille nullfs ro 0 0
/usr/ports /usr/local/bastille/jails/testjail/root/usr/ports nullfs ro 0 0
/var/ports/distfiles /usr/local/bastille/jails/testjail/root/var/ports/distfiles nullfs rw 0 0
/var/ports/packages /usr/local/bastille/jails/testjail/root/var/ports/packages nullfs rw 0 0

(do this for every fstab of every jail)

fire up the jails: bastille start ALL