The best free filesystem on Earth – ZFS – also often named OpenZFS recently – has also become very portable in recent years of its development. The OpenZFS Distributions page lists 6 (six) operating systems already.
They are:
- FreeBSD
- Illumos
- Linux
- MacOS
- NetBSD
- Windows
… but if you would like to create a ZFS pool compatible with all of them … which options and ZFS features should you choose? There is OpenZFS Feature Flags page dedicated exactly to that topic.
These are the ones that have yes value in all operating systems.
- async_destroy
- bookmarks
- empty_bpobj
- enabled_txg
- filesystem_limits
- lz4_compress
- hole_birth
- multi_vdev_crash_dump
- spacemap_histogram
I would also include these as only older NetBSD 4.0.5 version does not support them – but they are supported in newer NetBSD 5.3 version.
- embedded_data
- large_blocks
- sha512
- skein
There is also a dedicated zpool-features(7) man page for that information on the OpenZFS page and also zpool-features(7) man page on the FreeBSD page.
On the FreeBSD system the /usr/share/zfs/compatibility.d directory has files with supported ZFS Feature Flags for many major operating systems and ZFS versions.
% ls /usr/share/zfs/compatibility.d
2018
2019
2020
2021
compat-2018
compat-2019
compat-2020
compat-2021
freebsd-11.0
freebsd-11.1
freebsd-11.2
freebsd-11.3
freebsd-11.4
freebsd-12.0
freebsd-12.1
freebsd-12.2
freenas-11.0
freenas-11.1
freenas-11.2
freenas-11.3
freenas-9.10.2
grub2
openzfs-2.0-freebsd
openzfs-2.0-linux
openzfs-2.1-freebsd
openzfs-2.1-linux
openzfsonosx-1.7.0
openzfsonosx-1.8.1
openzfsonosx-1.9.3
openzfsonosx-1.9.4
truenas-12.0
ubuntu-18.04
ubuntu-20.04
zol-0.6.1
zol-0.6.4
zol-0.6.5
zol-0.7
zol-0.8
Unfortunately it misses NetBSD and Illumos systems for example … but having information from the OpenZFS Feature Flags page we can find Feature Flags set that will be supported everywhere.
Here are the stats for supported ZFS Feature Flags. The higher the number the more operating systems and ZFS version it covers.
% grep -h '^[^#]' /usr/share/zfs/compatibility.d/* \
| sort -n \
| uniq -c \
| sort -n
2 draid
5 bookmark_written
5 device_rebuild
5 livelist
5 log_spacemap
5 redacted_datasets
5 redaction_bookmarks
5 zstd_compress
9 allocation_classes
9 bookmark_v2
9 project_quota
9 resilver_defer
10 edonr
10 encryption
11 large_dnode
11 userobj_accounting
18 spacemap_v2
20 device_removal
20 obsolete_counts
20 zpool_checkpoint
31 sha512
31 skein
32 multi_vdev_crash_dump
36 filesystem_limits
36 large_blocks
37 bookmarks
37 embedded_data
37 enabled_txg
37 extensible_dataset
37 hole_birth
37 spacemap_histogram
38 async_destroy
38 empty_bpobj
38 lz4_compress
As the GNU GRUB is very outdated when it comes to ZFS support it should be pretty bulletproof to use it as a starting point of limited ZFS Feature Flags support.
% cat /usr/share/zfs/compatibility.d/grub2 # Features which are supported by GRUB2 async_destroy bookmarks embedded_data empty_bpobj enabled_txg extensible_dataset filesystem_limits hole_birth large_blocks lz4_compress spacemap_histogram
To make sure we are compatible we will now cross-link the GNU GRUB data.
First we will ‘generate’ the grep(1) command arguments that we will use in the next command.
% grep '^[^#]' /usr/share/zfs/compatibility.d/grub2 \
| while read I
do
echo "-e ' ${I}' \\"
done
-e ' async_destroy' \
-e ' bookmarks' \
-e ' embedded_data' \
-e ' empty_bpobj' \
-e ' enabled_txg' \
-e ' extensible_dataset' \
-e ' filesystem_limits' \
-e ' hole_birth' \
-e ' large_blocks' \
-e ' lz4_compress' \
-e ' spacemap_histogram' \
Lets now use these arguments to filter the ZFS features.
% grep -h '^[^#]' /usr/share/zfs/compatibility.d/grub2/* \ | sort -n \ | uniq -c \ | sort -n \ | grep -e ' async_destroy' \ -e ' bookmarks' \ -e ' embedded_data' \ -e ' empty_bpobj' \ -e ' enabled_txg' \ -e ' extensible_dataset' \ -e ' filesystem_limits' \ -e ' hole_birth' \ -e ' large_blocks' \ -e ' lz4_compress' \ -e ' spacemap_histogram' \ | wc -l 11 % grep -h '^[^#]' /usr/share/zfs/compatibility.d/grub2 | wc -l 11
So if seems that GRUB list of ZFS Feature Flags seems pretty compatible.
Lets now cross-reference that GRUB data with the data from the OpenZFS Feature Flags page.
I will create new /usr/share/zfs/compatibility.d/OZFF file that has these ZFS Feature Flags as content.
% cat /usr/share/zfs/compatibility.d/OZFF async_destroy bookmarks empty_bpobj enabled_txg filesystem_limits lz4_compress hole_birth multi_vdev_crash_dump spacemap_histogram embedded_data large_blocks sha512 skein % wc -l /usr/share/zfs/compatibility.d/OZFF 13
So there are 11 GRUB ZFS Feature Flags and 13 OpenZFS ‘Compatible’ Feature Flags.
Lets see how it they compare.
% cat /usr/share/zfs/compatibility.d/OZFF \
| grep -e async_destroy \
-e bookmarks \
-e embedded_data \
-e empty_bpobj \
-e enabled_txg \
-e extensible_dataset \
-e filesystem_limits \
-e hole_birth \
-e large_blocks \
-e lz4_compress \
-e spacemap_histogram \
| wc -l
10
I expected 11 here instead of 10 … we will not have to compare GRUB results to the OpenZFS Feature Flags section.
% grep -h '^[^#]' /usr/share/zfs/compatibility.d/{grub2,OZFF} \
| sort -n \
| uniq -c \
| sort -n
1 extensible_dataset
1 multi_vdev_crash_dump
1 sha512
1 skein
2 async_destroy
2 bookmarks
2 embedded_data
2 empty_bpobj
2 enabled_txg
2 filesystem_limits
2 hole_birth
2 large_blocks
2 lz4_compress
2 spacemap_histogram
Seems that we finally got our 10 most compatible OpenZFS Feature Flags set.
Its this set:
- async_destroy
- bookmarks
- embedded_data
- empty_bpobj
- enabled_txg
- filesystem_limits
- hole_birth
- large_blocks
- lz4_compress
- spacemap_histogram
To make it more comfortable to use we will put them it into the separate /usr/share/zfs/compatibility.d/COMPATIBLE file.
# cat /usr/share/zfs/compatibility.d/COMPATIBLE
async_destroy
bookmarks
embedded_data
empty_bpobj
enabled_txg
filesystem_limits
hole_birth
large_blocks
lz4_compress
spacemap_histogram
Lets now try to make that best effort most-compatible ZFS pool.
I will use one of my scripts – mdconfig.sh – to easy manipulate md(4) memory disks on FreeBSD.
# truncate -s 1g FILE # mdconfig.sh -c FILE IN: created vnode at /dev/md0 # zpool create -o compatibility=COMPATIBLE compatible /dev/md0 # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT compatible 960M 116K 960M - - 0% 0% 1.00x ONLINE - zroot 118G 48.6G 69.4G - - 36% 41% 1.00x ONLINE -
Now lets see what is zpool upgrade command showing us.
# zpool upgrade This system supports ZFS pool feature flags. All pools are formatted using feature flags. Some supported features are not enabled on the following pools. Once a feature is enabled the pool may become incompatible with software that does not support the feature. See zpool-features(7) for details. Note that the pool 'compatibility' feature can be used to inhibit feature upgrades. POOL FEATURE --------------- compatible multi_vdev_crash_dump large_dnode sha512 skein userobj_accounting encryption project_quota device_removal obsolete_counts zpool_checkpoint spacemap_v2 allocation_classes resilver_defer bookmark_v2 redaction_bookmarks redacted_datasets bookmark_written log_spacemap livelist device_rebuild zstd_compress draid zroot userobj_accounting encryption project_quota allocation_classes resilver_defer bookmark_v2 redaction_bookmarks redacted_datasets bookmark_written log_spacemap livelist device_rebuild zstd_compress draid
There are LOTS of ZFS Feature Flags to be activated but if we want to have our ZFS pool keep compatible – we will have to stay away from it π
You may get impression that you miss a lot … but you do not miss that much. Here are the ZFS Feature Flags you can fully utilize.
# zpool get all compatible | grep -v disabled
NAME PROPERTY VALUE SOURCE
compatible size 960M -
compatible capacity 0% -
compatible altroot - default
compatible health ONLINE -
compatible guid 5395735446052695775 -
compatible version - default
compatible bootfs - default
compatible delegation on default
compatible autoreplace off default
compatible cachefile - default
compatible failmode wait default
compatible listsnapshots off default
compatible autoexpand off default
compatible dedupratio 1.00x -
compatible free 960M -
compatible allocated 116K -
compatible readonly off -
compatible ashift 0 default
compatible comment - default
compatible expandsize - -
compatible freeing 0 -
compatible fragmentation 0% -
compatible leaked 0 -
compatible multihost off default
compatible checkpoint - -
compatible load_guid 17463015630652190527 -
compatible autotrim off default
compatible compatibility COMPATIBLE local
compatible feature@async_destroy enabled local
compatible feature@empty_bpobj enabled local
compatible feature@lz4_compress active local
compatible feature@spacemap_histogram active local
compatible feature@enabled_txg active local
compatible feature@hole_birth active local
compatible feature@extensible_dataset enabled local
compatible feature@embedded_data active local
compatible feature@bookmarks enabled local
compatible feature@filesystem_limits enabled local
compatible feature@large_blocks enabled local
You get the very decent LZ4 compression and also ZFS Bookmarks feature which are very useful feature for sync|recv mechanism.
Keep in mind that the -o compatibility= switch for zpool(8) is available on OpenZFS 2.1 or newer. The 2.1 version is already available on the FreeBSD 13.1-BETA* releases and will be part of the FreeBSD 13.1-RELEASE systems. To make use of it on older FreeBSD releases you will have to use openzfs and openzfs-kmod packages and also use the following settings in the /boot/loader.conf file.
From that one:
zfs_load=YES
Into that one:
zfs_load=NO
openzfs_load=YES
With these openzfs and openzfs-kmod packages and above settings in the /boot/loader.conf file you can use this OpenZFS 2.1 on FreeBSD 12.2 – on FreeBSD 12.3 – and on FreeBSD 13.0 … and of course on upcoming FreeBSD 13.1 release.
Not sure if I should have add anything more here. Feel free to remind me in the commends π
EOF
ππππππππ 