Tag Archives: freebsd

Unbound DNS Blacklist

Today I will show you how to configure unbound(8) to block spam/malicious/malware domains at DNS level.

unbound

I will use FreeBSD for that purpose but you can use any system that unbound(8) runs on.

logo-freebsd

Earlier I used generated /etc/hosts file but that was limited in several ways. The ZSH shell will autocomplete all these blocked domains to the ssh(1)/scp(1) commands (which takes needless time and shows useless completions). Subdomains are not handled. The malicious.com is blocked but ads.malicious.com is not. You need to duplicate all those domains in the /etc/hosts file.

TL;DR

Not all people have time for my long boring stories so this is meritum of this article.

# rm -rf /var/unbound
# mkdir -p /var/unbound/conf.d
# chown -R unbound:unbound /var/unbound
# service local_unbound setup
# service local_unbound enable
# service local_unbound start
# mkdir /root/bin
# cd 
# fetch -o /root/bin/unbound-blacklist-fetch.sh \
> https://raw.githubusercontent.com/vermaden/scripts/master/unbound-blacklist-fetch.sh
# chmod +x /root/bin/unbound-blacklist-fetch.sh
# /root/bin/unbound-blacklist-fetch.sh
# service local_unbound restart
# cat << BSD >> /var/cron/tabs/root
> # FETCH FRESH unbound(8) BLACKLIST
>   0 0 * * * /root/bin/unbound-blacklist-fetch.sh
> BSD

Whole Story

The unbound(8) caching DNS resolver has been added to FreeBSD base system in 2014 with 10.0-RELEASE version so being on FreeBSD you do not need to install anything. We will start with cleaning the any existing unbound(8) configuration which relies at /var/unbound. Keep in mind that /etc/unbound links to it.

# ls -l -d /etc/unbound /var/unbound
lrwxr-xr-x 1 root    wheel   14 2019.09.21 16:23 /etc/unbound -> ../var/unbound
drwxr-xr-x 3 unbound unbound  8 2020.11.17 16:48 /var/unbound

# rm -rf /var/unbound

# mkdir -p /var/unbound/conf.d

# chown -R unbound:unbound /var/unbound

The service local_unbound setup will create all needed configuration.

Just keep in mind that this process will setup all DNS servers that you have in the /etc/resolv.conf file.

You may want to put two of your favorite DNS servers before this process.

Configuration

# cat << BSD > /etc/resolv.conf
nameserver 9.9.9.9
nameserver 1.1.1.1
BSD

# service local_unbound setup
Performing initial setup.
destination: 
Extracting forwarders from /etc/resolv.conf.
/var/unbound/forward.conf created
/var/unbound/lan-zones.conf created
/var/unbound/control.conf created
/var/unbound/unbound.conf created
/etc/resolvconf.conf created
Original /etc/resolv.conf saved as /var/backups/resolv.conf.20201115.235254

# rm /var/backups/resolv.conf.20201115.235254

# find /var/unbound
/var/unbound
/var/unbound/lan-zones.conf
/var/unbound/control.conf
/var/unbound/unbound.conf
/var/unbound/forward.conf

% find /var/unbound -ls
 12685  17  drwxr-xr-x  3  unbound  unbound    8  Nov 17 16:48  /var/unbound
 13072   1  -rw-r--r--  1  root     unbound   98  Nov 17 05:00  /var/unbound/forward.conf
 12688   9  -rw-r--r--  1  root     unbound  354  Nov 15 23:56  /var/unbound/unbound.conf
 12686   1  drwxr-xr-x  2  unbound  unbound    3  Nov 16 00:23  /var/unbound/conf.d
 12158   9  -rw-r--r--  1  root     unbound  193  Nov 15 23:56  /var/unbound/control.conf
 11732   9  -rw-r--r--  1  root     unbound  189  Nov 15 23:56  /var/unbound/lan-zones.conf

# tail -n 999 /var/unbound/*
==> /var/unbound/conf.d <==
tail: /var/unbound/conf.d: Is a directory

==> /var/unbound/control.conf <==
# This file was generated by local-unbound-setup.
# Modifications will be overwritten.
remote-control:
	control-enable: yes
	control-interface: /var/run/local_unbound.ctl
	control-use-cert: no

==> /var/unbound/forward.conf <==
# Generated by resolvconf

forward-zone:
	name: "."
	forward-addr: 9.9.9.9
	forward-addr: 1.1.1.1

==> /var/unbound/lan-zones.conf <==
# This file was generated by local-unbound-setup.
# Modifications will be overwritten.
server:
	# Unblock reverse lookups for LAN addresses
	unblock-lan-zones: yes
	insecure-lan-zones: yes

==> /var/unbound/unbound.conf <==
# This file was generated by local-unbound-setup.
# Modifications will be overwritten.
server:
	username: unbound
	directory: /var/unbound
	chroot: /var/unbound
	pidfile: /var/run/local_unbound.pid
	auto-trust-anchor-file: /var/unbound/root.key

include: /var/unbound/lan-zones.conf
include: /var/unbound/control.conf
include: /var/unbound/conf.d/*.conf

We will now enable the local_unbound service and start it. At this point without any DNS blocking configuration.

# service local_unbound enable
local_unbound enabled in /etc/rc.conf

# service local_unbound start
Starting local_unbound.

The /etc/resolv.conf will now have hour favorite DNS servers hashed/disabled and 127.0.0.1 address will be specified. You can also use sockstat(8) to check that unbound(8) is indeed listening on port 53.

# cat /etc/resolv.conf
# nameserver 9.9.9.9
# nameserver 1.1.1.1
nameserver 127.0.0.1
options edns0

% sockstat -l -4
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS 
unbound local-unbo 7362 5 udp4 127.0.0.1:53 *:*
unbound local-unbo 7362 6 tcp4 127.0.0.1:53 *:*

Test

After unbound(8) has been enabled it should now be visible that first DNS request should be longer and the second one and following requests should be very fast.

% time host ftp.freebsd.org
ftp.freebsd.org is an alias for ftp.geo.freebsd.org.
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has address 213.138.116.78
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org has IPv6 address 2001:41c8:112:8300::15:0
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org mail is handled by 0 .
host ftp.freebsd.org  0.00s user 0.01s system 1% cpu 0.501 total

% time host ftp.freebsd.org
ftp.freebsd.org is an alias for ftp.geo.freebsd.org.
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has address 213.138.116.78
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org has IPv6 address 2001:41c8:112:8300::15:0
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org mail is handled by 0 .
host ftp.freebsd.org  0.01s user 0.00s system 88% cpu 0.007 total

Yep. Works.

Blacklist

I have written a simple and short unbound-blacklist-fetch.sh to automate the process of generating up to date DNS blocked domains config.

It uses one unbound(8) source and several hosts(5) sources, then combines them in unbound(8) compatible format while removing the duplicated entries.

unbound-blacklist-script.256

We will now fetch it, put it under /root/bin directory (or use your favorite one), make it executable and start it.

# mkdir /root/bin

# fetch -o /root/bin/unbound-blacklist-fetch.sh \
> https://raw.githubusercontent.com/vermaden/scripts/master/unbound-blacklist-fetch.sh

# chmod +x /root/bin/unbound-blacklist-fetch.sh

# /root/bin/unbound-blacklist-fetch.sh

# ls -l /var/unbound/conf.d/blacklist.conf
-rw-r--r-- 1 root unbound 3003929 2020.11.16 00:23 /var/unbound/conf.d/blacklist.conf

# tail /var/unbound/conf.d/blacklist.conf
local-zone: "zyrtec.1.p2l.info" always_nxdomain
local-zone: "zyrtec.3.p2l.info" always_nxdomain
local-zone: "zyrtec.4.p2l.info" always_nxdomain
local-zone: "zyski-z-innowacji.pl" always_nxdomain
local-zone: "zytpirwai.net" always_nxdomain
local-zone: "zz.cqcounter.com" always_nxdomain
local-zone: "zzhc.vnet.cn" always_nxdomain
local-zone: "zzz.clickbank.net" always_nxdomain
local-zone: "zzz.onion.pet" always_nxdomain
local-zone: "zzzrtrcm2.com" always_nxdomain

The unbound(8) daemon already includes all /var/unbound/conf.d/*.conf files and we use that here.

You can change where the script generates blocked domains config under the # SETTINGS section directly in the script.

% grep -A 5 SETTINGS scripts/unbound-blacklist-fetch.sh 
# SETTINGS
FILE=/var/unbound/conf.d/blacklist.conf
TEMP=/tmp/unbound
TYPE=always_nxdomain
ECHO=0

After the /var/unbound/conf.d/blacklist.conf file is generated you can now restart the unbound(8) service.

# service local_unbound restart
Stopping local_unbound.
Waiting for PIDS: 87745.
Starting local_unbound.
Waiting for nameserver to start... good

We will also add that script to crontab(5) so it will fetch fresh information every day.

# cat << BSD >> /var/cron/tabs/root
> 
> # FETCH FRESH unbound(8) BLACKLIST
>   0 0 * * * /root/bin/unbound-blacklist-fetch.sh
> 
> BSD

# crontab -l | tail -4

# FETCH FRESH unbound(8) BLACKLIST
  0 0 * * * /root/bin/unbound-blacklist-fetch.sh

Test Blocked Domains

From 60000+ blocked domains I have chosen ad.track.us.org as target for verification.

% ping ad.track.us.org
ping: cannot resolve ad.track.us.org: Unknown host

% host ad.track.us.org
Host ad.track.us.org not found: 3(NXDOMAIN)

% dog ad.track.us.org
Status: NXDomain

% dog @1.1.1.1 ad.track.us.org
CNAME ad.track.us.org. 11m30s   "track.us.org."
    A track.us.org.     6m30s   185.59.208.177


unbound-test.256

As You can see the domain is successfully blocked.

The above blocking configuration does not mean that I will now disable the uBlock Origin plugin from Firefox but its a welcome addition to blocking unwanted information tools workshop.

UPDATE 1 – Reworked Script and Alternatives

After reading comments on Hacker News / Lobsters / Reddit I got a lot of good ideas how to improve my script even more.

Some people suggested that very similar functionality already exists in dns/void-zones-tools package on FreeBSD. One can also use get_unbound_adblock.sh script or lie-to-me solution.

There are also more sophisticated tools like Pi-hole which also include DHCP server and web interface for management and statistics. Unfortunately Pi-hole does not run on FreeBSD.

After reworking and adding additional sources to my unbound-blacklist-fetch.sh script its now twice the amount of blocked unwanted domains. In the first release about 60000 domains were blocked. Now its more then 120000.

Here is the distribution of data between various types of sources.

% wc -lc /tmp/unbound/lists-*
   54587 1059592 /tmp/unbound/lists-domains
  143553 4115745 /tmp/unbound/lists-hosts
   32867 1596409 /tmp/unbound/lists-unbound
  231007 6771746 total

Now the /var/unbound/conf.d/blacklist.conf before these changes.

% wc -l blacklist.conf
   60009 blacklist.conf

% ls -l /var/unbound/conf.d/blacklist.conf
-rw-r--r-- 1 root unbound 2907535 2020-11-20 00:00 /var/unbound/conf.d/blacklist.conf

… and after adding additional sources.

% wc -l blacklist.conf
  122190 blacklist.conf

% ls -l /var/unbound/conf.d/blacklist.conf
-rw-r--r-- 1 root unbound 6086623 2020-11-20 15:07 /var/unbound/conf.d/blacklist.conf

Here is also performance summary about which part takes what amount of time.

Combining various sources and generating the final config takes about 5 seconds.

Most of the time is spent in fetching the data from various sources.

UPDATE1.unbound.script.256

The script is already uploaded to the GitHub repo.

Just fetch it and enjoy ๐Ÿ™‚

UPDATE 2 – Huge Domains List Version

Thanks to Luca Castagnini from bsd.network who pointed me to https://oisd.nl/ site with HUGE list of domains that can/could/should be blocked I made another variant (or version) of the script unbound-blacklist-fetch-huge.sh with a total of 145 (!) various sources for domains to block.

It of course takes little longer to fetch and generate then the ‘casual’ version.

UPDATE2.unbound.time

Its little less then 2 minutes to fetch and generate new config while the longest part is the fetching of those 145 sources. Generation takes about 15 seconds.

These 145 sources provide more then a million domains to block.

% wc -l /tmp/unbound/* 
 551704 lists-domains
 439505 lists-hosts
  60835 lists-unbound
1052044 total

The script after removing duplicated entries makes little more then 480000 domains of it.

% wc -l /var/unbound/conf.d/blacklist.conf 
 484829 /var/unbound/conf.d/blacklist.conf

Unfortunately it comes at a price. In this HUGE variant with domains from 145 sources the unbound(8) server now uses about 150 MB of RAM.

% top -b -o res|grep -E 'RES|unbound'
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C    TIME    WCPU COMMAND
75849 unbound       1  20    0   158M   149M select   4    0:03   0.00% local-unbound

I leave up to you which version to use and which sources to choose for blocking, but as my Firefox with about 20 tabs opened takes little more then 4226 MB of RAM these additional 150 MB from unbound(8) does not hurt that much ๐Ÿ™‚

% ./FIREFOX.RAM.sh
4226 MB

% cat FIREFOX.RAM.sh 
#! /bin/sh

SUM=0

top -b -o res \
  | sed 1,10d \
  | grep firefox \
  | awk '{print $7}' \
  | tr -cd '0-9\n' \
  | while read I
    do
      SUM=$(( ${SUM} + ${I} ))
      echo ${SUM}
    done | tail -1 | tr -d '\n'
echo " MB"
One more thing related to Firefox. After checking ‘free’ memory with Firefox running and after closing it the difference was about 2.6 GB which means that above script to calculate Firefox memory usage is not a lot accurate ๐Ÿ™‚
EOF

Realtek RTL8188CUS – USB 802.11n WiFi Review

When using FreeBSD on a new laptop you sometimes find out that the WiFi chip that it came with is not supported … or not yet supported in RELEASE version and support exists in CURRENT development version that you do not want to use.

This is where Realtek RTL8188CUS chip comes hand.

realtek

Its used in many appliances and products but we are interested in its small USB WiFi version that is really small.

The Realtek company even got Taiwan Green Classics Award 2011 for their 802.11b/g/n 2.4GHz 1T1R WLAN Single Chip Controller (RTL8188CE/RTL8188CUS) on 2011 year when it was introduced.

chip

chip-look

Its not very powerful as it comes with 1×1 antennas and 802.11n support – yes only single antenna. 150Mbps at most.

Its also very small and almost does not stick out of the laptop.

chip-space

When connected it also gives subtle little dim light.

chip-light

FreeBSD

I will now show you how it works on FreeBSD. This is for 12.2-RELEASE version but it worked the same for 11.1-RELEASE 3 years ago.

My ThinkPad W520 laptop already has Intel 6300 with 3×3 antennas and 802.11n standard WiFi card supported by iwn(4) driver.

# sysctl net.wlan.devices
net.wlan.devices: iwn0

We will now attach Realtek RTL8188CUS chip and will check whats coming in dmesg(8) command.

# dmesg
(...)
ugen2.3:  at usbus2
rtwn0 on uhub4
rtwn0:  on usbus2
rtwn0: MAC/BB RTL8188CUS, RF 6052 1T1R

… and some more information from usbconfig(8) command.

# usbconfig
(...)
ugen2.3:  at usbus2, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA)

# usbconfig -d 2.3 show_ifdrv
ugen2.3:  at usbus2, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (500mA)
ugen2.3.0: rtwn0: 

Its now listed as rtwn0 as its supported by the rtwn(4) driver on FreeBSD.

# sysctl net.wlan.devices
net.wlan.devices: rtwn0 iwn0

Lets connect to some wireless network with this Realtek chip. I will create wlan1 device as wlan0 is already taken by the other Intel 6300 card.

# ifconfig wlan1 create wlandev rtwn0

# ifconfig wlan1
wlan1: flags=8802<broadcast,simplex,multicast> metric 0 mtu 1500
        ether 00:1d:43:21:2d:1c
        groups: wlan
        ssid "" channel 1 (2412 MHz 11b)
        regdomain FCC country US authmode OPEN privacy OFF txpower 30 bmiss 7
        scanvalid 60 wme bintval 0
        parent interface: rtwn0
        media: IEEE 802.11 Wireless Ethernet autoselect (autoselect)
        status: no carrier
        nd6 options=21<performnud,auto_linklocal>

# wpa_supplicant -i wlan1 -c /etc/wpa_supplicant.conf
Successfully initialized wpa_supplicant
wlan1: Trying to associate with d8:07:b8:b8:f4:81 (SSID='wireless' freq=2442 MHz)
wlan1: Associated with d8:07:b6:b8:f4:81
wlan1: WPA: Key negotiation completed with d8:07:b6:b8:f4:81 [PTK=CCMP GTK=CCMP]
wlan1: CTRL-EVENT-CONNECTED - Connection to d8:07:b6:b8:f4:81 completed [id=40 id_str=]
^Z // HIT THE [CTRL]+[Z] KEYS HERE
zsh: suspended  wpa_supplicant -i wlan1 -c /etc/wpa_supplicant.conf
# bg
[1]  + continued  wpa_supplicant -i wlan1 -c /etc/wpa_supplicant.conf
#

We should now have network LAYER 2 connected and wpa_supplicant(8) should be running in a background and wlan1 interface should have associated status.

# ps ax | grep wpa_supplicant
48693  4  S        0:00.43 wpa_supplicant -i wlan1 -c /etc/wpa_supplicant.conf
50687  4  S+       0:00.00 grep --color wpa_supplicant

# ifconfig wlan1
wlan1: flags=8843<up,broadcast,running,simplex,multicast> metric 0 mtu 1500
        ether 00:1d:43:21:2d:1c
        groups: wlan
        ssid wireless channel 7 (2442 MHz 11g ht/20) bssid d8:07:b6:b8:f4:81
        regdomain FCC country US authmode WPA2/802.11i privacy ON
        deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 7 scanvalid 60
        protmode CTS ht20 ampdulimit 64k ampdudensity 4 shortgi -stbc -ldpc
        -uapsd wme roaming MANUAL
        parent interface: rtwn0
        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
        status: associated
        nd6 options=29<performnud,ifdisabled,auto_linklocal>

</performnud,ifdisabled,auto_linklocal></up,broadcast,running,simplex,multicast>

Lets add LAYER 3 with IP address using dhclient(8) command.

# dhclient wlan1
DHCPDISCOVER on wlan1 to 255.255.255.255 port 67 interval 3
DHCPOFFER from 10.0.0.1
DHCPREQUEST on wlan1 to 255.255.255.255 port 67
DHCPACK from 10.0.0.1
bound to 10.0.0.9 -- renewal in 3600 seconds.

We just got the 10.0.0.9 IP address.

One last step with DNS and we will test the connection with ping(8) command.

# echo nameserver 1.1.1.1 > /etc/resolv.conf

# ping -c 3 freebsd.org
PING freebsd.org (96.47.72.84): 56 data bytes
64 bytes from 96.47.72.84: icmp_seq=0 ttl=50 time=119.870 ms
64 bytes from 96.47.72.84: icmp_seq=1 ttl=50 time=119.371 ms
64 bytes from 96.47.72.84: icmp_seq=2 ttl=50 time=119.128 ms

--- freebsd.org ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 119.128/119.456/119.870/0.309 ms

Works.

FreeBSD Benchmark

I next tested the performance of this simple single antenna Realtek chip using NFS large file transfer in thunar(1) file manager.

not-great-not-terrible

The results are not that bad but not great either.

The file copy from LAN server attached directly to WiFi router to my laptop was about 2.9 MB/s fast. I was 5 meters away from the router.

server  ==LAN==>  router  ==WiFi==>  laptop  @  2.9 MB/s

The file copy from laptop using WiFi to LAN server attached directly to WiFi router was about 2.6 MB/s fast. Still about 5 meters away from the router.

laptop  ==WiFi==>  router  ==LAN==>  server  @  2.6 MB/s

That is 23.2 Mbps and 20.8 Mbps respectively. Really far from theoretical single antenna 802.11n 150 Mbps transfer … its probably fault of the FreeBSD wireless stack.

I would say that its sufficient for Internet browsing but using local LAN resources over NFS can be painful.

On the contrary my Intel 6300 WiFi card does 5.5 MB/s on the laptop-to-router-to-server copy and 10.5 MB/s on the server-to-router-to-laptop road. That is 44 Mbps and 84 Mbps respectively instead of 450 Mbps theoretical maximum. Both the Intel 6300 and my router have 3×3 antennas.

Would love to see these number closer to 30 MB/s …

Raspberry Pi

One of the other benefit of the Realtek RTL8188CUS chip is that it works very well on small Raspberry Pi boxes. Personally I have tested it on the Raspberry Pi 2B and it worked like a charm.

rpi

Price

This chip is also great when it comes to price. Products based on this chip are available everywhere. They are on EBAY. They are on ALIEXPRESS. And it costs as low as $2.50 in many cases.

Sometimes the delivery costs more then the product itself ๐Ÿ™‚

Enjoy.

UPDATE 1 – Middle Ages

Reddit user Yaazkal user from Reddit just reminded me thatย  rtwn(4) driver on FreeBSD still does not support 802.11n protocol.

It’s still in the middle ages of 802.11g transfers.

FreeBSD GNOME 3 Fast Track

This article is dedicated to Abraham Joseph who recently asked me if I could make an article on how to configure GNOME 3 on FreeBSD 12.2. At the moment 12.2-RC3 version is available so that is what I used but it will be the same on FreeBSD 12.2-RELEASE (or 12-STABLE). All commands here are executed as root user.

Here is the Table of Contents for this article.

  • Install
  • Connection to Internet
    • LAN with DHCP
    • LAN with Static IP Address
    • WIFI
    • DNS
  • Packages
  • Settings
  • GNOME 3
  • Fix the Icons
  • Rest of the Setup
  • UPDATE 1 – GDM Icons Fixed

Install

First you will have to install FreeBSD. You may use FreeBSD Handbook or one of my guides – Install FreeBSD 12 – available here.

Connection to Internet

Then after booting to new system you need to get connectivity to the Internet. If its LAN connection then its pretty fast. Its for em0 interface.

LAN with DHCP

# ifconfig em0 up
# dhclient em0

… assuming that you are on the LAN network with DHCP enabled.

To make it permanent put below line to the /etc/rc.conf file.

ifconfig_em0="DHCP"

LAN with Static IP Address

If not then execute these for static IPv4 connection on your em0 interface.

First add these two lines to the /etc/rc.conf file.

ifconfig_em0="inet 10.0.10.80/24 up"
defaultrouter="10.0.10.1"

This is how you /etc/rc.conf file should look like now assuming that you want 10.0.10.80/24 IP address and 10.0.10.1 gateway.

# grep -A 1 ifconfig /etc/rc.conf
ifconfig_em0="inet 10.0.10.80/24 up"
defaultrouter="10.0.10.1"

Then restart the netif and routing services.

# /etc/rc.d/netif restart
# /etc/rc.d/routing restart

WIFI

If you want to use WiFi to connect to the Internet then its slightly more typing. On my system I have iwn0 wireless card so that is what I will use here. The SSID is the name of your WiFi network and PSK is password for that network.

# sysctl -n net.wlan.devices
iwn0
# ifconfig wlan0 create wlandev iwn0
# wpa_passphrase SSID PSK >> /etc/wpa_supplicant.conf
# wpa_supplicant -i wlan0 -c /etc/wpa_supplicant.conf
// wait for CONNECTED state and hit [CTRL]-[Z]
# bg
# dhclient wlan0

To make it permanent across reboots add these to /etc/rc.conf file. I assume that information about your network is already in the /etc/wpa_supplicant.conf file generated by the wpa_passphrase(8) command above.

wlans_iwn0=wlan0
ifconfig_wlan0="WPA SYNCDHCP"

DNS

Last but not least you also need DNS. Put your favorite here or just paste the one below.

# echo nameserver 1.1.1.1 > /etc/resolv.conf

Packages

We will now switch to the latest branch or pkg(8) repository and install needed gnome3 and xorg packages.

# sed -i '' s/quarterly/latest/g /etc/pkg/FreeBSD.conf

# grep /latest /etc/pkg/FreeBSD.conf
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",

# pkg install -y gnome3 xorg

# pkg stats | head -3
Local package database:
        Installed packages: 523
        Disk space occupied: 3 GiB

Settings

Now you need to add ‘yourself’ to wheel and video groups.

# pw groupmod video -m yourself

# pw groupmod wheel -m yourself

GNOME 3 can not live without the /proc filesystem.

# cat << EOF >> /etc/fstab
proc  /proc  procfs  rw  0  0
EOF

Enable needed services.

# sysrc dbus_enable=YES

# sysrc hald_enable=YES

# sysrc gdm_enable=YES

# sysrc gnome_enable=YES

Enable EVDEV support.

# cat << EOF >> /etc/sysctl.conf
kern.evdev.rcpt_mask=6
EOF

Make the boot process faster and more clean.

# cat << EOF >> /boot/loader.conf
autoboot_delay=2
boot_mute=YES
EOF

Done. Now you can reboot into your new GNOME 3 system on FreeBSD.

# reboot

GNOME 3

Your GNOME 3 desktop is now ready and you can login. For the purpose of this article I used asd user.

gnome-0-gdm

gnome-1-gdm

gnome-2-desktop

gnome-3-menu

gnome-4-apps

The default font sizes on GNOME 3 are way too big for me so I tweaked them to 0.8 scale as shown on last screenshot. I also set the font in Terminal app to Monospaced.

Fix the Icons

As you probably saw on the screenshots above the buttons on the windows are broken. There is very simple fix for that. Like shown below on the screenshots first open the Tweak Tool. Then go to Appearance page. The 3rd item from top on the right panel (under the Themes sign) is Icons – please set it to Gnome.

gnome-5-tweak

gnome-6-appearance

gnome-7-icons

gnome-8-fixed

Viola! Now all icons look properly now.

Rest of the Setup

Now there are some things that will need be addressed.

Like with the drawing below, you have just drawn the circles ๐Ÿ™‚

owl

Just kidding ๐Ÿ™‚

Add your favorite applications with pkg(8) like LibreOffice or Firefox for example.

If you run GNOME 3 on a laptop, then I would suggest adding net-mgmt/networkmgr package to get Networkmgr networking manager from GhostBSD.

I would also suggest using some of the FreeBSD Desktop series articles for the completeness of your GNOME 3 setup. If you need to tweak X11 then check X11 Window System part. I would also suggest visiting Configuration – Fonts & Frameworks for fonts fine tuning. If you want to have automatic mounting of removable media (and you probably do) then check Configuration – Automount Media part. If its laptop then tuning the power management will give you extra battery time. Check the details at The Power to Serve – FreeBSD Power Management part. As you are using GNOME 3 you may want to check Dash to Dock plugin or use Plank described in the Configuration – Plank – Skippy-XD part. If you did not liked the net-mgmt/networkmgr package (Networkmgr from GhostBSD) you may want to try my network.sh solution – FreeBSD Network Management with network.sh – described here.

Not sure what else I can add here as I do not use GNOME 3 daily.

UPDATE 1 – GDM Icons Fixed

Thanks to Romain Tartiere from bsd.network the icons in GDM are now also fixed. The package graphics/ligvrsvg2-rust is now preferred instead of the graphics/librsvg2 package which was used previously.

Here is how now the GDM login page looks like.

gnome-9-gdm-fixed

EOF

Oldschool Gaming on FreeBSD

When was the last time you played a computer game? I really like one of Benjamin Franklin quotes – “We do not stop playing because we grow old, we grow old because we stop playing.” – he lived in times where computer games did not existed yet but the quote remains current. I do not play games a lot, but when I do I make sure that they are the right and best ones. They are often games from the past and some of these games just do not age … they are timeless actually. Today I will show you some oldschool gaming on FreeBSD system.

Here is the Table of Contents for the article.

  • Native Games
    • Native Console/Terminal Games
      • Interactive
      • Passive
    • Native X11 Games
  • AMIGA Games
  • DOS Games
    • Fourteen Years Later
  • Windows Games
  • Flash/SWF Games
  • Web Browser Games
  • Last Resort
  • Closing Thoughts

Here is my Openbox ‘games’ menu.

openbox-games-menu

Discussions and comments from ‘external’ sources are available here:

Native Games

First we will start with ‘native’ games on FreeBSD – as of today there are more then thousand games available in the FreeBSD Ports collection.

% ls /usr/ports/games | wc -l
    1130

You can get nice description for each of these games (from the pkg-descr file) by using the below command. I assume that your FreeBSD Ports tree is under /usr/ports directory.

% for I in /usr/ports/games/*/pkg-descr
> do
>   echo ${I}
>   echo
>   cat ${I}
>   echo
>   echo
>   echo
> done \
>   | grep \
>       --color=always \
>       -A 100 \
>       -E "^/usr/ports/games/.*/pkg-descr" \
>   | less -R

Here is the one-liner that you can actually copy and paste into your terminal.

% for I in /usr/ports/games/*/pkg-descr; do echo ${I}; echo; cat ${I}; echo; echo; echo; done | grep --color=always -A 100 -E "^/usr/ports/games/.*/pkg-descr" | less -R

Here is how it looks.

native-ports-list

This way you can browse (and search in less(1) command) for interesting titles.

Native Console/Terminal Games

Interactive

Lets start with the most simple games – the text games played in terminal. I play only two of these and they are 2048 and ctris games.

The 2048 is generally a single C file – 2048.c – from here – https://github.com/mevdschee/2048.c/blob/master/2048.c – you need to compile it with the cc(1) command – like that.

% cc -o 2048 2048.c
% ./2048

game-2048

The other one ctris is available in the FreeBSD Ports or you can add it by package with pkg(8) command.

# pkg install -y ctris

game-ctris

There are also several other terminal games like Tetris in the FreeBSD Ports – they are bsdtris or vitetris ones for example.

Passive

The are also terminal ‘non-interactive’ games (or maybe I should call them terminal screensavers alternatively).

My favorite two are cmatrix and pipes. The first one is available from FreeBSD Ports.

IMHO it looks best when launched this way.

% cmatrix -a -b -u 6 -C blue

game-cmatrix

Some time ago I ‘ported’ or should I say modified the pipes so it will work properly on FreeBSD and its available from – https://github.com/vermaden/pipes/blob/master/pipes.sh – here.

game-pipes

Native X11 Games

Time to move to some more graphically appealing games – the X11 games.

One of the better open source games it the Battle for Wesnoth which is also available in the FreeBSD Ports so adding package it easy.

# pkg install -y wesnoth

game-wesnoth

AMIGA Games

Most AMIGA games have been ported to DOS and its generally more convenient and a lot faster to play the DOS ‘ports’ using dosbox(1) instead of playing their original AMIGA versions under fs-uae(1) emulator. Some games like Sensible World of Soccer are better in original AMIGA version (little larger field view for example in the AMIGA version – but that only makes the DOS game little harder as you see less) then in DOS port but still the difference is not that huge to wait for each game start roughly 60 seconds with fs-uae(1) and manually switching virtual floppies.

swos-amiga-dos-xbla

As you can see on the far right the Sensible World of Soccer game has been even ported to the Microsoft XBOX console – SWOS – available here ๐Ÿ™‚

There is however (at least) one AMIGA game that has not been ported to DOS and its made by the legendary TEAM17 studio. Its the All Terrain Racing game. When you check its reviews back when it was released it did not get that high scores as Sensible World of Soccer for example but its one of the better looking and fun racing games made for AMIGA. But Sensible World of Soccer was named one of The 10 Most Important Video Games of All Time on 2007 so it really hard to beat that. Even Sensible Gold got a lot worse reviews.

game-atr

Originally it came in two floppies version so everytime you will launch this game in fs-uae(1) you will need to change the virtual floppy … which is real PITA I must say … not to mention 60 seconds of waiting for it to start. But there is other possibility. The All Terrain Racing game was also created for the AMIGA CD32 variant which used CD-ROM discs instead of floppies. That way by loading single ISO file you do not need to switch floppies anymore each time the game starts. Yay!

Fortunately the fs-uae(1) config for All Terrain Racing game is not long or complicated either.

fs-uae

The fs-uae(1) is also easily installable on FreeBSD by using packages.

# pkg install -y fs-uae

As the All Terrain Racing game is started/loaded from ISO file the save/load game state is not made ‘natively’ in the game but level up above – in the fs-uae(1) itself with SAVE STATE and LOAD STATE options as shown below.

game-atr-save-load

Not all AMIGA games are available as CD32 version but one may also use virtual Hard Disk option on the fs-uae(1) emulator to avoid switching floppies.

DOS Games

The DOS games can be very conveniently played by using the DOSBox which is available on FreeBSD as dosbox packages (or port).

# pkg install -y dosbox

Games in DOSBox start very quickly which is very nice. They also run very smoothly.

dosbox

Like you see I prefer to keep my games outside of the ~/.doxbox directory while keeping only configuration files there. But that is just ‘organizational’ choice. Make your own choices how and where to keep the games that suits you best.

Its also very convenient to redefine all keyboard shortcuts with DOSBox builtin keyboard remapper. For example instead of default [CTRL] for ‘FIRE’ button in Sensible World of Soccer I prefer to use [Z] key instead and that is my only mapping currently.

dosbox-keys

Keep in mind that as the DOSBox main config file is kept as ~/.dosbox/dosbox-${VERSION}.conf file (its ~/.dosbox/dosbox-0.74-3.conf as of time of writing the article) the remapped keyboard shortcuts as kept in the ~/.dosbox/mapper-${VERSION}.map file (its ~/.dosbox/mapper-0.74-3.map as of time of writing the article). Also keep in mind that if you will start dosbox in ~ (home) dir and not in ~/.dosbox~dir then dosbox will creates ~/mapper-0.74-3.map file (in your home dir) instead of proper ~/.dosbox/mapper-0.74-3.map place.

I also made script wrappers for each game so I can launch them quickly both from command line or by using dmenu.

scripts-games

You will find them all as games-* scripts in my GitHub repository – https://github.com/vermaden/scripts – available here. The DOSBox configuration files are in the dosbox dir on the same repo – https://github.com/vermaden/scripts/tree/master/dosbox – here.

My favorite DOS (originally from AMIGA) game is Sensible World of Soccer. I also like to play first Settlers game and Theme Hospital occasionally.

The DOSBox also allows you to easily record both audio (into WAV files) and video (into AVI files) with keyboard shortcuts.

For example I have recorded replay of my Sensible World of Soccer goals this way (then converted it to GIF using ffmpeg(1) for this).

SWOS Goals.

This is the ffmpeg(1) spell that I used to convert the DOSBox made AVI file to GIF file.

% ffmpeg -i ~/.dosbox/capture/sws_eng_001.avi -vf "fps=30" -loop 0 swos.goals.gif

Keep in mind that some games – and Sensible World of Soccer is one of these games – have more then one graphical mode to run them. When you start the game without any switches then it starts in low graphics mode which is easy to spot on by looking at pixelated/dotted ‘S‘ logo on the top right corner. The lines on the field are also not antialiased.

game-swos-not-full

When you add /f flag to the Sensible World of Soccer binary then it starts in full graphics mode and the ‘S‘ letter has now solid grey color on the back and lines on the field are also antialiased now.

game-swos-full
Here is how it looks in the DOSBox config file.

[autoexec]
@echo off
mount C: ~/.dosbox
C:
cd swos-SFX
sws_eng.exe /f

The Sensible World of Soccer has a special place in my private games ‘Hall of Fame’. Its the only game that I was able to play straight for 26 hours with breaks only for meals and pee … but that was in the old AMIGA times in the 90s.

Fourteen Years Later

One of the very old but also very nice logic games I played two decades ago was Swing game. I was not able to start this game in ‘normal’ mode as it started in ‘network’ mode each time. While searching for a possible solution I found … my own bug on DOSBox created 14 years agohttps://www.dosbox.com/comp_list.php?showID=2499 – here. I was not able to force the Swing game to start in ‘normal’ mode back then so I ‘marked’ it as ‘non working’ and moved on.

Now when I checked the bug report I see useful solutions to the problem. Pity I am not able to login and ‘thank’ as I do not remember my password and DOSBox page does not offer password reset service.

Seems that Swing needs to have its game directory mounted again as CD-ROM device. That way Swing starts in ‘normal’ mode and local Single and Multi Player games are now possible.

game-swing

The most important part of DOSBox config is here:

[autoexec]
@echo off
mount C ~/.dosbox
mount D ~/.dosbox/swing -t cdrom -usecd 0
C:
cd swing
swing.bat

Windows Games

Good old WINE. On FreeBSD there are two WINE versions. There is 64bit version as emulators/wine package and 32bit version names emulators/i386-wine. You want to use the latter because most games are 32bit and the 64bit version of WINE is not able to run them 32bit games. The installation on FreeBSD is typical as shown below.

# pkg install i386-wine

Old/classic Windows games usually keep your saved games directly in their installation folders under dirs named ‘SAVE’ or ‘SAVEDGAMES’ but in some time between 2005 and now the game developers started to think that its a ‘great’ idea to store them in your ‘My Documents’ directory … I do not have to tell you how I fell about that ‘decision’ but on FreeBSD it means that you will have saved games directories created directly in your ~ home directory (its /home/vermaden in my case) directory. What a mess.

winecfg

That is probably the only thing I configure in WINE on FreeBSD with winecfg – I set ‘My Documents’ location to ~/games.EXTRACT/profile directory instead.

The DOSBox is also better for gaming then WINE because it allows convenient [ALT]+[ENTER] shortcut to switch between fullscreen and windowed modes. With WINE I need to keep two game ‘startup’ scripts. Separate ones for windowed mode and for fullscreen mode.

wine-window-fullscreen

Below is an example of Colin McRae Rally 2.0 game under WINE on FreeBSD.

game-colin

My best time for Stage 1 on Italy was ‘only’ 2:09.84 so I was not fast enough to beat the all time best with 2:05:75 immortalized here – https://youtu.be/iLLMIJzpoVk – on YouTube.

Other classic – original Baldur’s Gate game below. It was possible to dual class into specialist mage – not possible now in Enhanced Edition.

game-baldurs-bg1

More up to date Baldur’s Gate: Enhanced Edition also works well.

game-baldurs-play

Less popular titles like Lionheart: Legacy of the Crusader also work well under WINE on FreeBSD. Very unusual game as it used the S.P.E.C.I.A.L system from Fallout instead of ‘typical’ choice like Advanced Dungeons and Dragons like in other Black Isle games.

game-lionheart-play

If for some reason your game does not work under WINE on FreeBSD then you should try Project Homura solution. Its also available as games/homura package (or port) on FreeBSD.

Flash/SWF Games

As I really hate Adobe Flash technology when browsing the web pages but I quite like the compact SWF files as simple flash games using WINE and Flash Player Projector from Adobe. I also use WINE to start the Windows version of that Flash Player Projector program. Its available here – https://www.adobe.com/support/flashplayer/debug_downloads.html – in the debug downloads.

You can pick one of these two but I use the first one.

An example of Governor of Poker 2 game running in the Flash Player Projector under WINE.

game-poker

All of these games can be found on various sites Flash games by looking in the View Page Source in your browser and looking for the link to the SWF file. I can not post these games here for download but if you will have problem finding them then let me know ๐Ÿ™‚

Web Browser Games

A class of games that are played directly in the web browser. Examples of such games can be Krunker

game-krunker

… or Spelunky for example.

game-spelunky

If you are VERY bored then you can also try the Chrome Dinosaur Game built into the Chromium browser. To access it try to open the page that does not exists like http://non-existing-site.com for example.

game-chromium

The Chromium browser will then display No Internet error message. Press [UP] arrow now and start to play.

game-chromium-end

If you liked the 2048 game and you DO have Internet connection you may also play 2048 directly at DuckDuckGo page.

game-duck-2048

Last Resort

Sometimes WINE does not work and the game is available only for Windows or Linux. The solution is to use the Virtualbox here. Remember to select/enable the 3D acceleration and install Virtualbox Guest Additions for good performance.

virtualbox

Closing Thoughts

All of these games were played smoothly on oldschool Intel HD Graphics 3000 card from 2011 Sandy Bridge CPU model i7-2820QM as this is with what my ThinkPad W520 came.

If I forgot to post something or its not obvious then feel free to let me know. This post as usual grow more then it should ๐Ÿ™‚ Also if you think that I missed some important dosbox(1)/wine(1)/fs-uae(1) options then let me know please.

EOF

Quare FreeBSD?

I really wanted to make this article short … but I failed miserably. At least I tried to organize it well so one may get back to it after ‘some’ reading because its not a short lecture. I wanted to title it Why FreeBSD? but when you type that into your favorite duck.com search engine there are so many similar articles. I wanted it to have distinguished and unique name so I used Latin word for ‘why‘ which is ‘quare‘.

logo-freebsd

What FreeBSD can offer you that other operating systems does not? From all of the operating systems I used I find FreeBSD to suck the least. This post is not here to convince you to use or try FreeBSD – this you will have to do by yourself. This article will show you why FreeBSD is valuable or better alternative to other operating systems and is definitely not dying.

This is the Table of Contents for this article.

  • Base System
  • ZFS Boot Environments
  • Rescue
  • Audio
  • Jails
  • FreeBSD Ports Infrastructure
  • Updating/Building from Source
  • Storage
  • Init System
  • Linux Binary Compatibility
  • Simplicity
  • Evolution Instead Rewriting
  • Documentation
  • Community
  • Closing Thoughts
  • External Discussions

Base System

When you install a Linux system its just a bunch of RPM or DEB packages. For example of you install CentOS 7.8 Minimal variant you end up with several hundred RPM packages installed. After a week or month many of these packages will get updates sometimes making this CentOS system unusable or even unbootable (recent GRUB Boothole problem for example). On the contrary FreeBSD comes with a Base System concept. This means that when you install FreeBSD you install a minimal system as a whole. No packages or subsystems to be separately updated. Just whole Base System. That means that /boot /bin /sbin /usr /etc /lib /libexec /rescue directories are untouchable by any packages. When you decide to install packages (or build them using FreeBSD Ports) they will all fall into the /usr/local prefix. That means /usr/local/etc for configuration. The /usr/local/bin and /usr/local/sbin directories for binaries. The /usr/local/lib and /usr/local/libexec for libraries and so on. The FreeBSD Base System kernel modules are kept in the same dir along with the kernel in the /boot/kernel directory. To make things tidy all kernel modules that are provided by packages go into the /boot/modules dir. Everything has its place and its separated.

That is separation between Base System binaries (at /bin /sbin /usr/bin /usr/sbin dirs) and Third Party Packages maintained by pkg(8) and are located at /usr/local/bin and /usr/local/sbin dirs. We all know differences between bin (user) and sbin (root) binaries but in FreeBSD there is also another more UFS related separation. When there was only UFS filesystem in the FreeBSD world the /bin and /sbin binaries were available at boot after the root (/) filesystem was mounted and yet before /usrย  filesystem was mounted – this is historical (and still useful in the UFS setups) distinction dating to old UNIX days. In ZFS setups it does not matter as all files are on ZFS pool anyway.

The FreeBSD Base System separation also helps with another thing – if any package gets the ‘great’ idea to install new compiler named cc and override the default system compiler … or to add libraries/includes in such a way that makes it super hard to get back into a working system. If some random FreeBSD package would add libc.so to /usr/local/lib dir then you are covered and not prevented from running programs as usual because FreeBSD system binaries are linked to stuff in /usr/lib dir. This is why there is PATH variable on UNIX systems (and FreeBSD as well) to set which directories should be searched for binaries first. On FreeBSD by default its set search Base System binaries dirs first and then Third Party Packages later.

You can update (or not) the Base System separately from the installed packages with freebsd-update(8) command when using RELEASE or by recompiling with make buildworld and make installworld commands when using STABLE/CURRENT systems. When it comes to packages you can update them using the pkg(8) tool or portmaster when building from FreeBSD Ports tree under /usr/ports dir. That means that any packages updates will not touch your FreeBSD Base System at all. For example when you mess up (and I have done that in the beginning of my FreeBSD journey) the compiled ports and packages and you want to start over the only thing you have to do is remove /usr/local and /boot/modules and /var/db/pkg directories. That’s it. You are just reverted to your Base System and can start over. This is just not possible when using Linux system. Even with Gentoo that many concepts are based on FreeBSD ideas does not have Base System feature. This Base System also have additional feature. Because its separated from packages version no one stops you from running oldshool FreeBSD 9.0 from 2012 and install there latest Firefox 80 or LibreOffice 7.0. You can not install latest Firefox on Ubuntu from 2012 …

One may be ‘afraid’ that such Base System independent from installed packages would take more space but nothing far more from the truth. The fresh installed FreeBSD 12.1 system uses less then 1 GB of disk space and takes less then 75 MB of RAM with sshd(8) running. For the comparison fresh CentOS 7.8 install with ‘Minimal’ set chosen takes 1.1 GB of disk space and uses more then 100 MB RAM with sshd(8) running. Such CentOS system is really naked and really needs more packages to be usable while FreeBSD with its Base System is far more capable and powerful and comes along with builtin latest version of LLVM/CLANG compiler suite for example.

More on the Base System topic:

ZFS Boot Environments

I have talked about this many times and probably one time too less because Linux world still ignores this bless. Having ZFS Boot Environments its such a game changer that once you realize how powerful it is you will never want to use a system that does not support it. The idea is that you can snapshot a running system at any moment of time and then reboot into that moment (or snapshot) if something happened. Its perfect solution for upgrade or changes to the system. The FreeBSD systems are already well ‘protected’ from problems arising after updating the packages but ZFS Boot Environments takes this to a whole new level.

groundhog

Like in the movie Groundhog Day (1993) with ZFS Boot Environments you will have limitless chances to get your shit toghether. Even the Base System updates and changes are protected by it. You can even transport that Boot Environment by using zfs send and zfs recv commands to other system … or propagate it on many systems. You can create Jails containers from it … or install new version of FreeBSD in the new Boot Environment and reboot into it while still having your older ‘production’ system untouched.

More on the ZFS Boot Environments topic:

Rescue

When you really mess up to the point that even Base System concept or ZFS Boot Environments feature did not stopped you from killing your FreeBSD installation then there is one more level of rescue … the Rescue subsystem.

rescue

You have about 150 statically linked binaries available at your disposal for the rescue mission of that FreeBSD installation. You probably think now that if its so many binaries then it probably takes a lot of space … nothing far more from the truth. Its actually one static binary with hardlinks … and it takes whooping 11 MB of disk space.

# ls -lh /rescue | head -5
total 1118446
-r-xr-xr-x  146 root  wheel    11M 2020.02.19 21:10 [
-r-xr-xr-x  146 root  wheel    11M 2020.02.19 21:10 bectl
-r-xr-xr-x  146 root  wheel    11M 2020.02.19 21:10 bsdlabel
-r-xr-xr-x  146 root  wheel    11M 2020.02.19 21:10 bunzip2

They Rescue subsystem even contains such binaries as bectl(8) for ZFS Boot Environments management or zfs(8) and zpool(8) commands for the ZFS filesystem. Here is complete list of these binaries.

# ls /rescue
[           dd               fsck_ffs      init       mdmfs          ping      rtsol        unlink
bectl       devfs            fsck_msdosfs  ipf        mkdir          ping6     savecore     unlzma
bsdlabel    df               fsck_ufs      iscsictl   mknod          pkill     sed          unxz
bunzip2     dhclient         fsdb          iscsid     more           poweroff  setfacl      unzstd
bzcat       dhclient-script  fsirand       kenv       mount          ps        sh           vi
bzip2       disklabel        gbde          kill       mount_cd9660   pwd       shutdown     whoami
camcontrol  dmesg            geom          kldconfig  mount_msdosfs  rcorder   sleep        xz
cat         dump             getfacl       kldload    mount_nfs      rdump     spppcontrol  xzcat
ccdconfig   dumpfs           glabel        kldstat    mount_nullfs   realpath  stty         zcat
chflags     dumpon           gpart         kldunload  mount_udf      reboot    swapon       zdb
chgrp       echo             groups        ldconfig   mount_unionfs  red       sync         zfs
chio        ed               gunzip        less       mt             rescue    sysctl       zpool
chmod       ex               gzcat         link       mv             restore   tail         zstd
chown       expr             gzip          ln         nc             rm        tar          zstdcat
chroot      fastboot         halt          ls         newfs          rmdir     tcsh         zstdmt
clri        fasthalt         head          lzcat      newfs_msdos    route     tee          
cp          fdisk            hostname      lzma       nextboot       routed    test         
csh         fsck             id            md5        nos-tun        rrestore  tunefs       
date        fsck_4.2bsd      ifconfig      mdconfig   pgrep          rtquery   umount   

More on the Rescue topic:

Audio

Not many people expect from FreeBSD to shine in that department but it shines a lot here and not from yesterday but from decades. Remember when Linux got rid of the old OSS subsystem with one channel and came up with ‘great’ idea to write ALSA? I remember because I used Linux back then. Disaster is very polite word to describe Linux audio stack back then … and then PulseAudio came and whole Linux audio system got much worse. Back then because of that one OSS channel and many ALSA channels meant that ONLY ONE application with OSS backend could do the sound (for example WINE). But if another application would want to ‘make’ sound using OSS and you already have WINE started then it will be soundless because that one and only OSS channel was already taken. And remember that ALSA was so bad back then that KDE or GNOME made their own sound daemons mixing audio in userspace that were incompatible with each other. That means if you used KDE and GNOME apps back then you could have sound from GNOME apps but not from KDE apps or vice versa. One big fucking audio hell on Linux.

audio

Lets get back to FreeBSD audio then. What FreeBSD offered? A whooping 256 OSS channels mixed live in kernel for low latency. Everything audio related just worked out of the box – and still works today. You could have WINE or KDE/GNOME sound backends attached to their OSS channels and also ALSA apps getting their sound device without a problem. Even when you plugged a 5.1 surround system into FreeBSD it worked out of the box without any configuration and applications were able to use it immediately. That FreeBSD audio supremacy remains today as PulseAudio sound mixing in userspace while generally working incorporates large latency on Liunx compared to in kernel FreeBSD mixing with low latency.

Comrade meka suggested that FreeBSD is also the only OS which has virtual_oss that allows mixing/resampling/compressing in user space and allows one to have Bluetooth headphones and USB microphone represented as single sound card.

More on the Audio topic:

Jails

The FreeBSD Jails are one of the oldest OS Level Virtualization implementations dating back to 1999. Even the Solaris Zones/Containers came five years later in 2004.

containers

After Docker was introduced in Linux the term OS Level Virtualization became less used to the Containers term and now the FreeBSD Jails along with Solaris Zones/Containers are named 1st generation containers. But that naming nomenclature change does not make FreeBSD Jails less powerful. They are also really brain dead simple to use. You just need a directory – for example /jail/nextcloud – where you will extract the FreeBSD Base System for desired release version – for example base.txz from 12.1-RELEASE and create the Jail config in the /etc/jail.conf file as shown below.

# mkdir -p /jail/nextcloud
# fetch -o - http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/12.1-RELEASE/base.txz | tar --unlink -xpJf - -C /jail/nextcloud
# cat /etc/jail.conf
nextcloud {
  host.hostname = nextcloud.local;
  ip4.addr = 10.0.0.100;
  path = /jail/nextcloud;
}

Now you can start you Jail right away.

# service jail onestart nextcloud
Starting jails: nextcloud.

Voila! Your FreeBSD Jail is already running.

# jls
   JID  IP Address      Hostname                      Path
     1  10.0.0.100      nextcloud.local               /jail/nextcloud

You can of course have a trimmed down version of FreeBSD Base System in the Jail if that is needed. The ZFS filesystem also helps here greatly because with zfs clone only your ‘base’ Jail will take space and only the changes you make to Jails created from it. Thanks to other FreeBSD subsystem – the Linux Binary Compatibility – you can also create a Linux Jail – for example running Devuan Jail.

The FreeBSD Jails are also very lightweight. You can boot and use about 1000 FreeBSD Jails on a single FreeBSD system with 4 GB RAM.

They are also very easy to debug and troubleshoot comparing even to plain Docker – not to even mention Kubernetes which requires whole team of highly skilled people to maintain.

The FreeBSD Jails may be configured/managed only by the Base System utilities such as jls(8)/jexec(8) but you can also select from many third party Jail management frameworks. From all available ones I would choose BastilleBSD because of their modern approach and many ready to use templates for all needed use cases.

More on the Jails topic:

FreeBSD Ports Infrastructure

This is one of another examples why FreeBSD rocks that much. When you install Ubuntu or CentOS in some version there is chance that you will end up with not latest versions of packages but with versions that were quite up-to-date when this distribution version was released. Its especially visible in the CentOS world (and its upstream enterprise source system from Red Hat) where packages are quite up-to-date when .0 (dot zero) release is published but are VERY outdated when .8 or .9 incarnation of that release is available. Not to even mention that Firefox for example is released every month …

packages

As I said before when describing the FreeBSD Base System the FreeBSD Ports (and packages built from it available through pkg(8)) are independent. That means that third party software from FreeBSD Ports is almost always up-to-date (or very close to it). You can even check it on the repology.org site for the details. Below you will find a ‘snapshot’ of the repology.org stats from time of writing this article. The ‘online’ table is very long so I copy/pasted just the systems relevant to the article.

repology

One of the other advantages of FreeBSD Ports is that it offers really MASSIVE amount of software counting 40354 ports when writing this article and still rising. Amount of ready to be installed packages are little smaller with more then 32000 available.

I once migrated for a while to OpenSolaris in 2009 on my Dell Latitude D630 laptop because I really liked all the Solaris features (including ZFS and ZFS Boot Environments that were not available on FreeBSD back then) and the OpenSolaris GNOME based desktop was pretty nice back then even with Time Slider feature for ZFS snapshots in the Nautilus file manager. I got working WiFi connection, sound was working, generally everything on my laptop was supported and working with OpenSolaris … but there was no software. Of course ‘large’ projects like GIMP or OpenOffice was available even in the default pkg(8) repository but not much else. There was less then 4000 packages back then on OpenSolaris while about 25000 packages on FreeBSD if I recall correctly.

You can also easily browse available FreeBSD Ports (and its options) on the web by using the https://freshports.org/ page.

ports

The count of FreeBSD Ports is one thing, the features is another. No matter which Linux distribution you are using you will find a software that was compiled and shipped without that needed flag that you desperately need. If you find such software on FreeBSD it ‘hurts’ only for a moment because you can VERY EASILY recompile that software with needed options and replace that ‘default’ package with yours. For example the FreeBSD project is afraid to provide packages of Lame because of existing MP3 patents, so multimedia/ffmpeg package is built without MP3 support (with --disable-libmp3lame flag). That is why I have my own audio/lame and multimedia/ffmpeg packages built with my configure options and that is very easy to achieve. You need to go to the /usr/ports/multimedia/ffmpeg dir type make config and select [x] LAME at the ncurses dialog. Your chosen options will be saved as plain /var/db/ports/multimedia_ffmpeg/options file. If you remove that file (or type make rmconfig) then these custom options will reset to defaults. Then you type make build deinstall install clean and your port with new options is ready and installed as package. Nothing more is needed. You can even lock that package from the pkg(8) upgrades with pkg lock -y ffmpeg command so it will not be modified later but its better to rebuild such packages everytime you do a pkg upgrade procedure because of libraries versions bump and changes. While its very easy and fast to create a script with these commands to make it more automated you can also use other parts of the FreeBSD Ports infrastructure – enter Poudriere (or Synth) – more on that in the next part.

You also do not have to configure each port that way (which could be PITA for large amount of ports) but you may specify your needed (OPTIONS_SET) or unwanted (OPTIONS_UNSET) parameters only once globally using the /etc/make.conf file. You can also specify which default versions of software you want to use, for example Apache 2.2 instead of 2.4 and PHP 7.0 instead of 7.2. You can find all default versions in the /usr/ports/Mk/bsd.default-versions.mk file. Once you setup these options you can build/rebuild or update your packages from FreeBSD Ports by portmaster(8) tool. Like on Gentoo Linux with USE flags. But this is the original. Gentoo took all/most of its ideas from FreeBSD system and its Ports infrastructure.

The Poudriere is a build framework that uses FreeBSD Ports and FreeBSD Jails to build requested packages in clean reproducible way. You can create whole new binary package repository for pkg(8) command to use with it. I mentioned Synth because while Poudriere is often used to produce whole package repository the Synth is usually used just to rebuild several packages that does not fit your needs.

There is one important things about FreeBSD Ports that is often misunderstood by newcomers. What is the difference between the Ports and packages that are fetched and installed by pkg(8) tool? Its quite simple. A package is just a build and installed port. Nothing more or less. When you use the binary packages using pkg(8) command you are using packages that someone (the FreeBSD project in that case) built for you from the FreeBSD Ports in some point in time. While FreeBSD strives to maintain as up-to-date built packages as possible its the nature of FreeBSD Ports that they are always more up-to-date then the built packages. That is why you may build and install a new version of needed packages by yourself using FreeBSD Ports. One may think of such usage when it comes to security holes. When some locally executed commands (like file(1) for example) has a security hole then its not critical for you to update it as fast as possible because that security hole can be harmless for you, but when new version of Firefox fixes very important security hole then its better to update from FreeBSD Ports version faster because waiting 2 days for the package to be built (along with other packages) can be too long.

More on the FreeBSD Ports topic:

Updating/Building from Source

While the FreeBSD Ports infrastructure is for third party software the FreeBSD Base System (or its parts) also can be easily and convenient build from source. The FreeBSD kernel config is also very small and simple. While Linux kernel config contains thousands of options – 4432 for example in the default CentOS 8.2 install the FreeBSD GENERIC config has about 20 times options less – only 260 options. But that does not saturate the topic. You can start with MINIMAL FreeBSD kernel config which has only 75 options specified.

Linux # grep -c '^CONFIG' /boot/config-$( uname -r )
4432

FreeBSD # grep -c -E '^(device|options)' /usr/src/sys/amd64/conf/GENERIC
260

FreeBSD # grep -c -E '^(device|options)' /usr/src/sys/amd64/conf/MINIMAL
75

… and its not only about smaller amount of options. Can you tell my how many steps (and which ones are required) to rebuild CentOS or Ubuntu for example without Bluetooth support?

code

On the contrary its very simple (and fast) on the FreeBSD side. While /etc/make.conf file is used to enable/disable Ports options the /etc/src.conf file is used to enable/disable FreeBSD Base System options while building it from source. To build FreeBSD without Bluetooth support just add WITHOUT_BLUETOOTH=yes to the /etc/src.conf file and type these to build it:

# beadm create safe
# cd /usr/src
# make buildworld kernel
# reboot
# cd /usr/src
# make installworld
# mergemaster -iU
# reboot

Voila! You now have FreeBSD without Bluetooth support … and if any of the steps failed or because of your lack of experience/expertise your FreeBSD system does not boot or is broken you can use tools from /rescue to try to fix it (or at least figure out what is broken) and when you do not want to cope with this jest select safe ZFS Boot Environment at the FreeBSD loader(8) to boot to the system before you started building modified version of FreeBSD. Yes, You are bulletproof here. While having 294 WITHOUT_X options and 125 WITH_X options you can really tune FreeBSD Base System to your needs.

# zgrep -c WITHOUT_ /usr/share/man/man5/src.conf.5.gz
294

# zgrep -c WITH_ /usr/share/man/man5/src.conf.5.gz
125

The big downside of updating FreeBSD by source is that you can not use the freebsd-update tools to do it … but nothing stops you from creating your own FreeBSD Update Server so you will be able to use freebsd-update by adding updates using a CURRENT or STABLE system instead of RELEASE. That process is described in the Build Your Own FreeBSD Update Server article of official FreeBSD documentation.

More on the FreeBSD Source Updates/Builds topic:

Storage

Storage is one of the parts where FreeBSD really shines. Lots of people adore FreeBSD for well integrated ZFS filesystem and its really true. ZFS in FreeBSD has always been first class citizen. Lately OpenZFS 2.0 has been also integrated from the upstream joint FreeBSD and Linux repository. More and more FreeBSD features and solutions are using ZFS features.

openzfs

Most of these people that like integrated ZFS in FreeBSD do not know about the FreeBSD GEOM modular disk transformation framework which provides various storage related features and utilities like software RAID0/RAID1/RAID10/RAID3/RAID5 configurations or transparent encryption of underlying devices with GELI/GDBE (like LUKS on Linux). It also allows transparent filesystem journaling for ANY filesystem with GJOURNAL (yes also for FAT32 or exFAT) or allows one to export block devices over network with GEOM GATE devices (like NFS for block devices).

storage

FreeBSD also has its own FUSE implementation which allows all these FUSE based filesystems to work natively on FreeBSD. While lots of Linux folks know DRBD probably very few of them knew that FreeBSD comes with its own DRBD like solution called HAST – which does exactly the same thing. While ZFS has a lot features and possibilities FreeBSD still maintains and develops fast and small memory footprint UFS filesystem which today is used either with Soft Updates (SU) or Journaled Soft Updates (SUJ) depending on the use case. For example 10 TB data on UFS filesystem with Journaled Soft Updates (SUJ) takes about 1 minute under fsck(8). These storage solutions are available from FreeBSD Base System alone. The FreeBSD Ports offers much more with distributed filesystems solutions such as CEPH, LeoFS, LizardFS or Minio for Amazon S3 compatible storage.

More on the Storage topic:

Init System

FreeBSD offers really simple yet very powerful init system. It has system wide config under /etc/rc.conf file when you can enable/disable needed services with service_enable=YES and service_enable=NO stanzas. You do not even need to launch vi(1) to add them – just type sysrc service_enable=YES and they are added to the /etc/rc.conf file. There are also default values and services that are enabled and you will find them – along with many comments – in the /etc/defaults/rc.conf file. Each FreeBSD service file has PROVIDE/REQUIRE stanzas which are then used to automatically order the services to start. Services that can be run in parallel are started in parallel to save time. For example its pointless to start sshd(8) daemon without network. To start or stop the serivice you need to type service sshd start or service sshd stop command. But when a service is not enabled in the /etc/rc.conf file then you need to used add onestart and onestop instead. The Base System separation remains here as FreeBSD Base System services are located at /etc/rc.d directory and third party applications from ports/packages are kept under /usr/local prefix which means /usr/local/etc/rc.d dir.

When using systemd(1) you never know how the services gonna start because it will be different each time. Zero determinism. On FreeBSD you know exactly which services will start when because they are always ordered in the same state according to the PROVIDE/REQUIRE stanzas. FreeBSD also offers tools that will tell you the exact order – rcorder(8) – which can be used for all services, Base System services or third party services separately. There is also service -r command that will show you what was the orfer at the boot time.

# rcorder /etc/rc.d/* | head
/etc/rc.d/growfs
/etc/rc.d/sysctl
/etc/rc.d/hostid
/etc/rc.d/zvol
/etc/rc.d/dumpon
/etc/rc.d/ddb
/etc/rc.d/geli
/etc/rc.d/gbde
/etc/rc.d/ccd
/etc/rc.d/swap

# rcorder /usr/local/etc/rc.d/* | tail
/usr/local/etc/rc.d/hald
/usr/local/etc/rc.d/git_daemon
/usr/local/etc/rc.d/fscd
/usr/local/etc/rc.d/cupsd
/usr/local/etc/rc.d/cups_browsed
/usr/local/etc/rc.d/clamav-clamd
/usr/local/etc/rc.d/clamav-milter
/usr/local/etc/rc.d/clamav-freshclam
/usr/local/etc/rc.d/avahi-dnsconfd
/usr/local/etc/rc.d/aria2

# rcorder /etc/rc.d/* /usr/local/etc/rc.d/* 2> | grep -C 3 sshd
/etc/rc.d/ubthidhci
/etc/rc.d/syscons
/etc/rc.d/swaplate
/etc/rc.d/sshd
/etc/rc.d/cron
/etc/rc.d/jail
/etc/rc.d/localpkg

Adding new service to FreeBSD is also very easy as template for new service is very small and simple.

#!/bin/sh

. /etc/rc.subr

name=dummy
rcvar=dummy_enable

start_cmd="${name}_start"
stop_cmd=":"

load_rc_config $name
: ${dummy_enable:=no}
: ${dummy_msg="Nothing started."}

dummy_start()
{
	echo "$dummy_msg"
}

run_rc_command "$1"

If its not simple enought for you there is dedicated FreeBSD article about writing them – Practical rc.d Scripting in BSD – available here.

More on the Init System topic:

Linux Binary Compatibility

While Linux can not be FreeBSD – the FreeBSD can be Linux – and its not some slow emulation – its implementation of Linux system calls. There was time when enterprises used to work with Linux only applications (not available on FreeBSD by then) using the Linux Binary Compatibility on FreeBSD because it was faster then running them natively on Linux – FreeBSD Used to Generate Spectacular Special Effects – an official FreeBSD Press Release about FreeBSD being used to generate spacial effects to the one of the best movies of all time – The Matrix (1999).

matrix

Today the LINUX_COMPAT is also natively fast and allows one to run Linux applications – even Linux games in X11 with hardware acceleration for graphics. Think of it as WINE but for Linux applications. It lives under /compat/linux directory. It even implements Linux /proc virtual filesystem which can be mounted at the /compat/linux/proc dir but its not mandatory. For any software that does not come with source code and works on Linux the Linux Binary Compatibility saves the day. For example the f.lux project. Before I got to know Redshift I used f.lux Linux binary using LINUX_COMPAT to suppress blue spectrum light from my FreeBSD screen. The Linux Binary Compatibility subsystem can also be used to run Linux bases FreeBSD Jails – with Devuan for example.

More on the Linux Binary Compatibility topic:

Simplicity

FreeBSD is simple but not coarse/ornery. For example as Linux the FreeBSD system also supports the /proc virtual filesystem but on FreeBSD its optional and not used by default while Linux could not live without it. But while Linux has mandatory /proc it also has another virtual filesystem residing under /sys … but why Linux people need two different virtual filesystems with similar purposes? Why they could not create everything under /proc as it already existed. That is big enigma for my sanity.

But /sys is not the end of that madness. Its just a beginning.

What about these?

  • securityfs
  • devpts
  • cgroup
  • pstore
  • bpf
  • configfs
  • selinuxfs
  • systemd-1
  • mqueue
  • debugfs
  • hugetlbfs

Take a look at the FreeBSD mount(8) output after the default install on ZFS.

FreeBSD # mount
zroot/ROOT/12.1 on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
zroot/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot/var/mail on /var/mail (zfs, local, nfsv4acls)
zroot/usr/home on /usr/home (zfs, local, noatime, nfsv4acls)
zroot/var/crash on /var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/audit on /var/audit (zfs, local, noatime, noexec, nosuid, nfsv4acls)
zroot/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot/usr/src on /usr/src (zfs, local, noatime, nfsv4acls)
zroot/usr/ports on /usr/ports (zfs, local, noatime, nosuid, nfsv4acls)

Several ZFS datasets and one virtual devfs filesystem for /dev directory. With install on UFS it would be similar with several UFS partitions mounted instead of ZFS datasets.

Take a look at the CentOS 8.2 installation with just one physical root (/) XFS filesystem.

[root@centos8 ~]# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=919388k,nr_inodes=229847,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpuset)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,memory)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,blkio)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpu,cpuacct)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,freezer)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,perf_event)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,rdma)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,pids)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,devices)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda1 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17309)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=187088k,mode=700)

Fuck me. Its even really hard to just find any REAL filesystem there … fortunately we can ask for only XFS filesystems to display.

[root@centos8 ~]# mount -t xfs
/dev/sda1 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

Lets get on the networking now. Lets assume that you want to make standard enterprise networking setup on a physical server with two interfaces aggregated together into highly available interface bond0 (lagg0 on FreeBSD) and then you want to put VLAN tag and IP address on that VLAN. The CentOS 7.x/8.x installer (Anaconda) will welcome you with this mess.

[root@centos7 ~]# ls -1 /etc/sysconfig/network-scripts/ifcfg-*
ifcfg-Bond_connection_1
ifcfg-eno49
ifcfg-eno49-1
ifcfg-eno50
ifcfg-eno50-1
ifcfg-VLAN_connection_1

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-Bond_connection_1
DEVICE=bond0
BONDING_OPTS="miimon=1 updelay=0 downdelay=0 mode=active-backup"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_PRIVACY=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME="Bond connection 1"
UUID=ca85417f-8852-43bf-96ee-5bd3f0f83648
ONBOOT=yes

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno49
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eno49
UUID=2f60f50b-38ad-492a-b90a-ba736acf6792
DEVICE=eno49
ONBOOT=no

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno49-1
HWADDR=xx:xx:xx:xx:xx:xx
TYPE=Ethernet
NAME=eno49
UUID=342b8494-126d-4f3a-b749-694c8c922aa1
DEVICE=eno49
ONBOOT=yes
MASTER=bond0
SLAVE=yes

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno50
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eno50
UUID=4fd36e24-1c6d-4a65-a316-7a14e9a92965
DEVICE=eno50
ONBOOT=no

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno50-1
HWADDR=xx:xx:xx:xx:xx:xx
TYPE=Ethernet
NAME=eno50
UUID=a429b697-73c2-404d-9379-472cb3c35e06
DEVICE=eno50
ONBOOT=yes
MASTER=bond0
SLAVE=yes

[root@centos7 ~]# cat/etc/sysconfig/network-scripts/ifcfg-VLAN_connection_1
VLAN=yes
TYPE=Vlan
PHYSDEV=ca85417f-8852-43bf-96ee-5bd3f0f83648
VLAN_ID=601
REORDER_HDR=yes
GVRP=no
MVRP=no
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=10.20.30.40
PREFIX=24
GATEWAY=10.20.30.1
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_PRIVACY=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME="VLAN connection 1"
UUID=90f7a9bb-1443-4adf-a3eb-86a03b23ecfb
ONBOOT=yes

For the record – I have choosen ‘STATIC’ IPv4 address but installer made these interfaces to use DHCP and that STATIC address. That could be a bug but lets get to the point.

After manual fixing with vi(1) (and hour later) this is how it supposed to look.

[root@centos7 ~]# cat /etc/sysconfig/network
GATEWAY=10.20.30.1
NOZEROCONF=yes

[root@centos7 ~]# ls -1 /etc/sysconfig/network-scripts/ifcfg-*
ifcfg-bond0
ifcfg-bond0.601
ifcfg-eno49
ifcfg-eno50

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BONDING_OPTS="miimon=1 updelay=0 downdelay=0 mode=active-backup"
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
IPV4_FAILURE_FATAL=no
IPV6INIT=no
ONBOOT=yes

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0.601
VLAN=yes
TYPE=Vlan
VLAN_ID=601
DEVICE=bond0.601
REORDER_HDR=yes
GVRP=no
MVRP=no
BOOTPROTO=none
IPADDR=10.20.30.40
PREFIX=24
IPV4_FAILURE_FATAL=no
IPV6INIT=no
ONBOOT=yes

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno49
BOOTPROTO=none
IPV4_FAILURE_FATAL=no
IPV6INIT=no
TYPE=Ethernet
NAME=eno49
DEVICE=eno49
ONBOOT=yes
MASTER=bond0
SLAVE=yes

[root@centos7 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno50
BOOTPROTO=none
IPV4_FAILURE_FATAL=no
IPV6INIT=no
TYPE=Ethernet
NAME=eno50
DEVICE=eno50
ONBOOT=yes
MASTER=bond0
SLAVE=yes

Better … but still takes A LOT OF SPACE and several files to cover that quite simple setup. Not to mention its level of complication and making that very error prone way. The same configuration on FreeBSD would take just 7 lines within single /etc/rc.conf file as shown below.

ifconfig_fxp0="up"
ifconfig_fxp1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport fxp0 laggport fxp1"
vlans_lagg0="601"
ifconfig_lagg0_601="inet 10.20.30.40/24"
defaultrouter="10.20.30.1"

What about the boot process? FreeBSD boots from root on ZFS partition with just small 512 KB not mountable partition. No separate /boot device is needed. On the other side Linux always needs that separate /boot partition filled with GRUB modules. No matter if its ZFS or LVM. That is why implementation of ZFS Boot Environments is quite complicated on Linux bacause even if you have root on ZFS on a Linux system there is still unprotected /boot filesystem that can not be snapshoted with ZFS and has to be protected in old classic way which kill the idea of ZFS Boot Environments or Linux.

FreeBSD is really simple and well thought operating system. But also a very underestimated one.

Evolution Instead Rewriting

How many Linux tools or subsystems are abandoned or superseeded by new ones? Why the ifconfig(8) command was not updated with new options and instead a new ip(8) command was introduced? Same with netstat(8) being replaced by ss(8). Same with arp(8)/iwconfig/route(8) and many more. What about whole init system? The Linux world has been taken over by systemd(1) whenever you like it or not. Even distributions that have grown their mature init systems like Ubuntu with its Upstart has moved to systemd(1) altogether. The distributions that do not use it are very few and considered a niche today.

evolution

In the FreeBSD land on the countary such things happen only if there is no other way to implement new things. Its the last thing wanted in the FreeBSD. FreeBSD evolves and is developed with stability and backward compatibility in mind. Userland tools are grown and updated with new options instead of rewriting them over and over again. Not to mention how many new bugs are introduced by changing one tool to another.

More on the Evolution Instead Rewriting topic:

Documentation

Having system that can do almost anything but not knowing how to do that makes that system pretty useless (or at least pretty PITA to use). FreeBSD offers second to none documentation that is actively maintained and updated. Along with its legendary FreeBSD Handbook and FreeBSD FAQ the FreeBSD project also offers official FreeBSD Articles about various FreeBSD topics. The Man Pages are also very detailed and contain many examples. There is also FreeBSD Wiki page for work in progress documentation and ideas related to FreeBSD development and if you have any problems or questions related to FreeBSD there are official FreeBSD Forums and oldschool Mailing Lists available.

documentation

These were only the official project knowledge sources but there are also lots of FreeBSD books. Here are the best and up-to-date ones.

  • Absolute FreeBSD – Complete Guide to FreeBSD – 3nd Edition (2019)
  • Beginning Modern Unix (2018)
  • Book of PF – 3rd Edition (2015)
  • Design and Implementation of FreeBSD 11 Operating System – 2nd Edition (2015)
  • FreeBSD Device Drivers (2012)
  • FreeBSD Mastery – ZFS (2015)
  • FreeBSD Mastery – Advanced ZFS (2016)
  • FreeBSD Mastery – Storage Essentials (2014)
  • FreeBSD Mastery – Specialty Filesystems (2015)
  • FreeBSD Mastery – Jails (2019)

There are also two magazines that are dedicated to BSD and FreeBSD systems. Both are free and cover lots of interesting topics regarding FreeBSD.

With all this knowledge and support its really hard not to achieve what you need/want with FreeBSD system.

Community

Last but not least and I would say its even more important then good documentation (which FreeBSD has awesome). People that use FreeBSD do that conciously and are often experienced not only in FreeBSD land but also in topics related to other UNIX systems. Often they took long road of first using the Linux systems before finally setting on the FreeBSD land or they still do Linux adminitration for a living while resting using far more reasonable and sensible FreeBSD solution. I always find FreeBSD Community helpful and friendly. Always willingly helpful – especially towards newcommers. Even when you try to ‘force’ FreeBSD people to ‘fight’ in unjust/doubtful discussion they will reply with dignity and technical arguments instead of yelling at you.

The FreeBSD project even made several articles and Handbook chapters especially for Linux newcommers (or sometimes called systemd(1) refugees).

Closing Thoughts

I tried really hard to not make it a Linux rant but some may feel it that way – if so please remember that this was not my intention. FreeBSD like Linux and like any other operating system has its ups and downs. Hope that I showed you most interesting FreeBSD parts. I may add new sections here without a warning in the future ๐Ÿ™‚

External Discussions

Discussions and comments from ‘external’ sources are available here:

EOF

FreeBSD Cluster with Pacemaker and Corosync

I always missed ‘proper’ cluster software for FreeBSD systems. Recently I got to run several Pacemaker/Corosync based clusters on Linux systems. I thought how to make similar high availability solutions on FreeBSD and I was really shocked when I figured out that both Pacemaker and Corosync tools are available in the FreeBSD Ports and packages as net/pacemaker2 and net/corosync2 respectively.

In this article I will check how well Pacemaker and Corosync cluster works on FreeBSD.

pacemaker

There are many definitions of a cluster. One that I like the most is that a cluster is a system that is still redundant after losing one of its nodes (is still a cluster). This means that 3 nodes is a minimum for a cluster by that definition. The two node clusters are quite problematic because of their biggest exposure to the split brain problem. That is why often in the two node clusters additional devices or systems are added to make sure that this split brain does not happen. For example one can add third node without any resources or services just as a ‘witness’ role. Other way is to add a shared disk resource that will serve the same purpose and often its a raw volume with SCSI-3 Persistent Reservation mechanism used.

Lab Setup

As usual it will be entirely VirtualBox based and it will consist of 3 hosts. To not create 3 same FreeBSD installations I used 12.1-RELEASE virtual machine image available from the FreeBSD Project directly:

There are several formats available – qcow2/raw/vhd/vmdk – but as I will be using VirtualBox I used the VMDK one.

Here is the list of the machines for the GlusterFS cluster:

  • 10.0.10.111 node1
  • 10.0.10.112 node2
  • 10.0.10.113 node3

Each VirtualBox virtual machine for FreeBSD is the default one (as suggested in the VirtualBox wizard) with 512 MB RAM and NAT Network as shown on the image below.

machine

Here is the configuration of the NAT Network on VirtualBox.

nat-network-01

nat-network-02

Before we will try connect to our FreeBSD machines we need to make the minimal network configuration inside each VM. Each FreeBSD machine will have such minimal /etc/rc.conf file as shown example for node1 host.

root@node1:~ # cat /etc/rc.conf
hostname=node1
ifconfig_em0="inet 10.0.10.111/24 up"
defaultrouter=10.0.10.1
sshd_enable=YES

For the setup purposes we will need to allow root login on these FreeBSD machines with PermitRootLogin yes option in the /etc/ssh/sshd_config file. You will also need to restart the sshd(8) service after the changes.

root@node1:~ # grep PermitRootLogin /etc/ssh/sshd_config
PermitRootLogin yes

root@node1:~ # service sshd restart

By using NAT Network with Port Forwarding the FreeBSD machines will be accessible on the localhost ports. For example the node1 machine will be available on port 2211, the node2 machine will be available on port 2212 and so on. This is shown in the sockstat utility output below.

nat-network-03-sockstat

nat-network-04-ssh

To connect to such machine from the VirtualBox host system you will need this command:

vboxhost % ssh -l root localhost -p 2211

Packages

As we now have ssh(1) connectivity we need to add needed packages. To make our VMs resolve DNS queries we need to add one last thing. We will also switch to ‘quarterly’ branch of the pkg(8) packages.

root@node1:~ # echo 'nameserver 1.1.1.1' > /etc/resolv.conf
root@node1:~ # sed -i '' s/quarterly/latest/g /etc/pkg/FreeBSD.conf

Remember to repeat these two upper commands on node2 and node3 systems.

Now we will add Pacemaker and Corosync packages.

root@node1:~ # pkg install pacemaker2 corosync2 crmsh

root@node2:~ # pkg install pacemaker2 corosync2 crmsh

root@node3:~ # pkg install pacemaker2 corosync2 crmsh

These are messages both from pacemaker2 and corosync2 that we need to address.

Message from pacemaker2-2.0.4:

--
For correct operation, maximum socket buffer size must be tuned
by performing the following command as root :

# sysctl kern.ipc.maxsockbuf=18874368

To preserve this setting across reboots, append the following
to /etc/sysctl.conf :

kern.ipc.maxsockbuf=18874368

======================================================================

Message from corosync2-2.4.5_1:

--
For correct operation, maximum socket buffer size must be tuned
by performing the following command as root :

# sysctl kern.ipc.maxsockbuf=18874368

To preserve this setting across reboots, append the following
to /etc/sysctl.conf :

kern.ipc.maxsockbuf=18874368

We need to change the kern.ipc.maxsockbuf parameter. Lets do it then.

root@node1:~ # echo 'kern.ipc.maxsockbuf=18874368' >> /etc/sysctl.conf
root@node1:~ # service sysctl restart

root@node2:~ # echo 'kern.ipc.maxsockbuf=18874368' >> /etc/sysctl.conf
root@node2:~ # service sysctl restart

root@node3:~ # echo 'kern.ipc.maxsockbuf=18874368' >> /etc/sysctl.conf
root@node3:~ # service sysctl restart

Lets check what binaries come with these packages.

root@node1:~ # pkg info -l pacemaker2 | grep bin
        /usr/local/sbin/attrd_updater
        /usr/local/sbin/cibadmin
        /usr/local/sbin/crm_attribute
        /usr/local/sbin/crm_diff
        /usr/local/sbin/crm_error
        /usr/local/sbin/crm_failcount
        /usr/local/sbin/crm_master
        /usr/local/sbin/crm_mon
        /usr/local/sbin/crm_node
        /usr/local/sbin/crm_report
        /usr/local/sbin/crm_resource
        /usr/local/sbin/crm_rule
        /usr/local/sbin/crm_shadow
        /usr/local/sbin/crm_simulate
        /usr/local/sbin/crm_standby
        /usr/local/sbin/crm_ticket
        /usr/local/sbin/crm_verify
        /usr/local/sbin/crmadmin
        /usr/local/sbin/fence_legacy
        /usr/local/sbin/iso8601
        /usr/local/sbin/pacemaker-remoted
        /usr/local/sbin/pacemaker_remoted
        /usr/local/sbin/pacemakerd
        /usr/local/sbin/stonith_admin

root@node1:~ # pkg info -l corosync2 | grep bin
        /usr/local/bin/corosync-blackbox
        /usr/local/sbin/corosync
        /usr/local/sbin/corosync-cfgtool
        /usr/local/sbin/corosync-cmapctl
        /usr/local/sbin/corosync-cpgtool
        /usr/local/sbin/corosync-keygen
        /usr/local/sbin/corosync-notifyd
        /usr/local/sbin/corosync-quorumtool

root@node1:~ # pkg info -l crmsh | grep bin
        /usr/local/bin/crm

Cluster Initialization

Now we will initialize our FreeBSD cluster.

First we need to make sure that names of the nodes are DNS resolvable.

root@node1:~ # tail -3 /etc/hosts

10.0.10.111 node1
10.0.10.112 node2
10.0.10.113 node3

root@node2:~ # tail -3 /etc/hosts

10.0.10.111 node1
10.0.10.112 node2
10.0.10.113 node3

root@node3:~ # tail -3 /etc/hosts

10.0.10.111 node1
10.0.10.112 node2
10.0.10.113 node3


Now we will generate the Corosync key.

root@node1:~ # corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /usr/local/etc/corosync/authkey.

root@node1:~ # echo $?
0

root@node1:~ # ls -l /usr/local/etc/corosync/authkey
-r--------  1 root  wheel  128 Sep  2 20:37 /usr/local/etc/corosync/authkey

Now the Corosync configuration file. For sure some examples were provided by the package maintainer.

root@node1:~ # pkg info -l corosync2 | grep example
        /usr/local/etc/corosync/corosync.conf.example
        /usr/local/etc/corosync/corosync.conf.example.udpu

We will take the second one as a base for our config.

root@node1:~ # cp /usr/local/etc/corosync/corosync.conf.example.udpu /usr/local/etc/corosync/corosync.conf

root@node1:~ # vi /usr/local/etc/corosync/corosync.conf
               /* LOTS OF EDITS HERE */

root@node1:~ # cat /usr/local/etc/corosync/corosync.conf

totem {
  version: 2
  crypto_cipher: aes256
  crypto_hash: sha256
  transport: udpu

  interface {
    ringnumber: 0
    bindnetaddr: 10.0.10.0
    mcastport: 5405
    ttl: 1
  }
}

logging {
  fileline: off
  to_logfile: yes
  to_syslog: no
  logfile: /var/log/cluster/corosync.log
  debug: off
  timestamp: on

  logger_subsys {
    subsys: QUORUM
    debug: off
  }
}

nodelist {

  node {
    ring0_addr: 10.0.10.111
    nodeid: 1
  }

  node {
    ring0_addr: 10.0.10.112
    nodeid: 2
  }

  node {
    ring0_addr: 10.0.10.113
    nodeid: 3
  }

}

quorum {
  provider: corosync_votequorum
  expected_votes: 2
}

Now we need to propagate both Corosync key and config across the nodes in the cluster.

We can use some simple tools created exactly for that like net/csync2 cluster synchronization tool for example but plain old net/rsync will serve as well.

root@node1:~ # pkg install -y rsync

root@node1:~ # rsync -av /usr/local/etc/corosync/ node2:/usr/local/etc/corosync/
The authenticity of host 'node2 (10.0.10.112)' can't be established.
ECDSA key fingerprint is SHA256:/ZDmln7GKi6n0kbad73TIrajPjGfQqJJX+ReSf3NMvc.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node2' (ECDSA) to the list of known hosts.
Password for root@node2:
sending incremental file list
./
authkey
corosync.conf
service.d/
uidgid.d/

sent 1,100 bytes  received 69 bytes  259.78 bytes/sec
total size is 4,398  speedup is 3.76

root@node1:~ # rsync -av /usr/local/etc/corosync/ node3:/usr/local/etc/corosync/
The authenticity of host 'node2 (10.0.10.112)' can't be established.
ECDSA key fingerprint is SHA256:/ZDmln7GKi6n0kbad73TIrajPjGfQqJJX+ReSf3NMvc.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node3' (ECDSA) to the list of known hosts.
Password for root@node3:
sending incremental file list
./
authkey
corosync.conf
service.d/
uidgid.d/

sent 1,100 bytes  received 69 bytes  259.78 bytes/sec
total size is 4,398  speedup is 3.76

Now lets check that they are the same.

root@node1:~ # cksum /usr/local/etc/corosync/{authkey,corosync.conf}
2277171666 128 /usr/local/etc/corosync/authkey
1728717329 622 /usr/local/etc/corosync/corosync.conf

root@node2:~ # cksum /usr/local/etc/corosync/{authkey,corosync.conf}
2277171666 128 /usr/local/etc/corosync/authkey
1728717329 622 /usr/local/etc/corosync/corosync.conf

root@node3:~ # cksum /usr/local/etc/corosync/{authkey,corosync.conf}
2277171666 128 /usr/local/etc/corosync/authkey
1728717329 622 /usr/local/etc/corosync/corosync.conf

Same.

We can now add corosync_enable=YES and pacemaker_enable=YES to the /etc/rc.conf file.

root@node1:~ # sysrc corosync_enable=YES
corosync_enable:  -> YES

root@node1:~ # sysrc pacemaker_enable=YES
pacemaker_enable:  -> YES

root@node2:~ # sysrc corosync_enable=YES
corosync_enable:  -> YES

root@node2:~ # sysrc pacemaker_enable=YES
pacemaker_enable:  -> YES

root@node3:~ # sysrc corosync_enable=YES
corosync_enable:  -> YES

root@node3:~ # sysrc pacemaker_enable=YES
pacemaker_enable:  -> YES

Lets start these services then.

root@node1:~ # service corosync start
Starting corosync.
Sep 02 20:55:35 notice  [MAIN  ] Corosync Cluster Engine ('2.4.5'): started and ready to provide service.
Sep 02 20:55:35 info    [MAIN  ] Corosync built-in features:
Sep 02 20:55:35 warning [MAIN  ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
Sep 02 20:55:35 warning [MAIN  ] Please migrate config file to nodelist.

root@node1:~ # ps aux | grep corosync
root  1695   0.0  7.9 38340 38516  -  S    20:55    0:00.40 /usr/local/sbin/corosync
root  1699   0.0  0.1   524   336  0  R+   20:57    0:00.00 grep corosync

Do the same on the node2 and node3 systems.

The Pacemaker is not yet running so that will fail.

root@node1:~ # crm status
Could not connect to the CIB: Socket is not connected
crm_mon: Error: cluster is not available on this node
ERROR: status: crm_mon (rc=102): 

We will start it now.

root@node1:~ # service pacemaker start
Starting pacemaker.

root@node2:~ # service pacemaker start
Starting pacemaker.

root@node3:~ # service pacemaker start
Starting pacemaker.

You need to give it little time to start because if you will execute crm status command right away you will get 0 nodes configured message as shown below.

root@node1:~ # crm status
Cluster Summary:
  * Stack: unknown
  * Current DC: NONE
  * Last updated: Wed Sep  2 20:58:51 2020
  * Last change:  
  * 0 nodes configured
  * 0 resource instances configured


Full List of Resources:
  * No resources

… but after a while everything is detected and works as desired.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 21:02:49 2020
  * Last change:  Wed Sep  2 20:59:00 2020 by hacluster via crmd on node2
  * 3 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * No resources

The Pacemaker runs properly.

root@node1:~ # ps aux | grep pacemaker
root      1716   0.0  0.5 10844   2396  -  Is   20:58     0:00.00 daemon: /usr/local/sbin/pacemakerd[1717] (daemon)
root      1717   0.0  5.2 49264  25284  -  S    20:58     0:00.27 /usr/local/sbin/pacemakerd
hacluster 1718   0.0  6.1 48736  29708  -  Ss   20:58     0:00.75 /usr/local/libexec/pacemaker/pacemaker-based
root      1719   0.0  4.5 40628  21984  -  Ss   20:58     0:00.28 /usr/local/libexec/pacemaker/pacemaker-fenced
root      1720   0.0  2.8 25204  13688  -  Ss   20:58     0:00.20 /usr/local/libexec/pacemaker/pacemaker-execd
hacluster 1721   0.0  3.9 38148  19100  -  Ss   20:58     0:00.25 /usr/local/libexec/pacemaker/pacemaker-attrd
hacluster 1722   0.0  2.9 25460  13864  -  Ss   20:58     0:00.17 /usr/local/libexec/pacemaker/pacemaker-schedulerd
hacluster 1723   0.0  5.4 49304  26300  -  Ss   20:58     0:00.41 /usr/local/libexec/pacemaker/pacemaker-controld
root      1889   0.0  0.6 11348   2728  0  S+   21:56     0:00.00 grep pacemaker

We can check how Corosync sees its members.

root@node1:~ # corosync-cmapctl | grep members
runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(10.0.10.111) 
runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.1.status (str) = joined
runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(10.0.10.112) 
runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.2.status (str) = joined
runtime.totem.pg.mrp.srp.members.3.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.3.ip (str) = r(0) ip(10.0.10.113) 
runtime.totem.pg.mrp.srp.members.3.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.3.status (str) = joined

… or the quorum information.

root@node1:~ # corosync-quorumtool
Quorum information
------------------
Date:             Wed Sep  2 21:00:38 2020
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          1
Ring ID:          1/12
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
         1          1 10.0.10.111 (local)
         2          1 10.0.10.112
         3          1 10.0.10.113

The Corosync log file is filled with the following information.

root@node1:~ # cat /var/log/cluster/corosync.log
Sep 02 20:55:35 [1694] node1 corosync notice  [MAIN  ] Corosync Cluster Engine ('2.4.5'): started and ready to provide service.
Sep 02 20:55:35 [1694] node1 corosync info    [MAIN  ] Corosync built-in features:
Sep 02 20:55:35 [1694] node1 corosync warning [MAIN  ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
Sep 02 20:55:35 [1694] node1 corosync warning [MAIN  ] Please migrate config file to nodelist.
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] Initializing transport (UDP/IP Unicast).
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha256
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] The network interface [10.0.10.111] is now up.
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync configuration map access [0]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: cmap
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync configuration service [1]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: cfg
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: cpg
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync profile loading service [4]
Sep 02 20:55:35 [1694] node1 corosync notice  [QUORUM] Using quorum provider corosync_votequorum
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync vote quorum service v1.0 [5]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: votequorum
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync cluster quorum service v0.1 [3]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: quorum
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] adding new UDPU member {10.0.10.111}
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] adding new UDPU member {10.0.10.112}
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] adding new UDPU member {10.0.10.113}
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] A new membership (10.0.10.111:4) was formed. Members joined: 1
Sep 02 20:55:35 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:55:35 [1694] node1 corosync notice  [QUORUM] Members[1]: 1
Sep 02 20:55:35 [1694] node1 corosync notice  [MAIN  ] Completed service synchronization, ready to provide service.
Sep 02 20:58:14 [1694] node1 corosync notice  [TOTEM ] A new membership (10.0.10.111:8) was formed. Members joined: 2
Sep 02 20:58:14 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:14 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:14 [1694] node1 corosync notice  [QUORUM] This node is within the primary component and will provide service.
Sep 02 20:58:14 [1694] node1 corosync notice  [QUORUM] Members[2]: 1 2
Sep 02 20:58:14 [1694] node1 corosync notice  [MAIN  ] Completed service synchronization, ready to provide service.
Sep 02 20:58:19 [1694] node1 corosync notice  [TOTEM ] A new membership (10.0.10.111:12) was formed. Members joined: 3
Sep 02 20:58:19 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:19 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:19 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:19 [1694] node1 corosync notice  [QUORUM] Members[3]: 1 2 3
Sep 02 20:58:19 [1694] node1 corosync notice  [MAIN  ] Completed service synchronization, ready to provide service.

Here is the configuration.

root@node1:~ # crm configure show
node 1: node1
node 2: node2
node 3: node3
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=2.0.4-2deceaa3ae \
        cluster-infrastructure=corosync

As we will not be configuring the STONITH mechanism we will disable it.

root@node1:~ # crm configure property stonith-enabled=false

New configuraion with STONITH disabled.

root@node1:~ # crm configure show
node 1: node1
node 2: node2
node 3: node3
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=2.0.4-2deceaa3ae \
        cluster-infrastructure=corosync \
        stonith-enabled=false

The STONITH configuration is out of scope of this article but properly configured STONITH looks like that.

stonith

First Service

We will now configure our first highly available service – a classic – a floating IP address ๐Ÿ™‚

root@node1:~ # crm configure primitive IP ocf:heartbeat:IPaddr2 params ip=10.0.10.200 cidr_netmask="24" op monitor interval="30s"

Lets check how it behaves.

root@node1:~ # crm configure show
node 1: node1
node 2: node2
node 3: node3
primitive IP IPaddr2 \
        params ip=10.0.10.200 cidr_netmask=24 \
        op monitor interval=30s
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=2.0.4-2deceaa3ae \
        cluster-infrastructure=corosync \
        stonith-enabled=false

Looks good – lets check the cluster status.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:03:35 2020
  * Last change:  Wed Sep  2 22:02:53 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::heartbeat:IPaddr2):        Stopped

Failed Resource Actions:
  * IP_monitor_0 on node3 'not installed' (5): call=5, status='complete', exitreason='Setup problem: couldn't find command: ip', last-rc-change='2020-09-02 22:02:53Z', queued=0ms, exec=132ms
  * IP_monitor_0 on node2 'not installed' (5): call=5, status='complete', exitreason='Setup problem: couldn't find command: ip', last-rc-change='2020-09-02 22:02:54Z', queued=0ms, exec=120ms
  * IP_monitor_0 on node1 'not installed' (5): call=5, status='complete', exitreason='Setup problem: couldn't find command: ip', last-rc-change='2020-09-02 22:02:53Z', queued=0ms, exec=110ms

Crap. Linuxism. The ip(8) command is expected to be present in the system. This is FreeBSD and as any UNIX system it comes with ifconfig(8) command instead.

We will have to figure something else. For now we will delete our useless IP service.

root@node1:~ # crm configure delete IP

Status after deletion.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:04:34 2020
  * Last change:  Wed Sep  2 22:04:31 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * No resources

Custom Resource

Lets check what resources are available by stock Pacemaker installation.

root@node1:~ # ls -l /usr/local/lib/ocf/resource.d/pacemaker
total 144
-r-xr-xr-x  1 root  wheel   7484 Aug 29 01:22 ClusterMon
-r-xr-xr-x  1 root  wheel   9432 Aug 29 01:22 Dummy
-r-xr-xr-x  1 root  wheel   5256 Aug 29 01:22 HealthCPU
-r-xr-xr-x  1 root  wheel   5342 Aug 29 01:22 HealthIOWait
-r-xr-xr-x  1 root  wheel   9450 Aug 29 01:22 HealthSMART
-r-xr-xr-x  1 root  wheel   6186 Aug 29 01:22 Stateful
-r-xr-xr-x  1 root  wheel  11370 Aug 29 01:22 SysInfo
-r-xr-xr-x  1 root  wheel   5856 Aug 29 01:22 SystemHealth
-r-xr-xr-x  1 root  wheel   7382 Aug 29 01:22 attribute
-r-xr-xr-x  1 root  wheel   7854 Aug 29 01:22 controld
-r-xr-xr-x  1 root  wheel  16134 Aug 29 01:22 ifspeed
-r-xr-xr-x  1 root  wheel  11040 Aug 29 01:22 o2cb
-r-xr-xr-x  1 root  wheel  11696 Aug 29 01:22 ping
-r-xr-xr-x  1 root  wheel   6356 Aug 29 01:22 pingd
-r-xr-xr-x  1 root  wheel   3702 Aug 29 01:22 remote

Not many … we will try to modify the Dummy service into an IP changer on FreeBSD.

root@node1:~ # cp /usr/local/lib/ocf/resource.d/pacemaker/Dummy /usr/local/lib/ocf/resource.d/pacemaker/ifconfig

root@node1:~ # vi /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
               /* LOTS OF TYPING */

Because of the WordPress blogging system limitations I am forced to post this ifconfig resource as an image … but fear not – the text version is also available here – ifconfig.odt – for download.

Also the first version did not went that well …

root@node1:~ # setenv OCF_ROOT /usr/local/lib/ocf
root@node1:~ # ocf-tester -n resourcename /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
Beginning tests for /usr/local/lib/ocf/resource.d/pacemaker/ifconfig...
* rc=3: Your agent has too restrictive permissions: should be 755
-:1: parser error : Start tag expected, '<' not found
usage: /usr/local/lib/ocf/resource.d/pacemaker/ifconfig {start|stop|monitor}
^
* rc=1: Your agent produces meta-data which does not conform to ra-api-1.dtd
* rc=3: Your agent does not support the meta-data action
* rc=3: Your agent does not support the validate-all action
* rc=0: Monitoring a stopped resource should return 7
* rc=0: The initial probe for a stopped resource should return 7 or 5 even if all binaries are missing
* Your agent does not support the notify action (optional)
* Your agent does not support the demote action (optional)
* Your agent does not support the promote action (optional)
* Your agent does not support master/slave (optional)
* rc=0: Monitoring a stopped resource should return 7
* rc=0: Monitoring a stopped resource should return 7
* rc=0: Monitoring a stopped resource should return 7
* Your agent does not support the reload action (optional)
Tests failed: /usr/local/lib/ocf/resource.d/pacemaker/ifconfig failed 9 tests

But after adding 755 mode to it and making several (hundred) changes it become usable.

root@node1:~ # vi /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
             /* LOTS OF NERVOUS TYPING */
root@node1:~ # chmod 755 /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
root@node1:~ # setenv OCF_ROOT /usr/local/lib/ocf
root@node1:~ # ocf-tester -n resourcename /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
Beginning tests for /usr/local/lib/ocf/resource.d/pacemaker/ifconfig...
* Your agent does not support the notify action (optional)
* Your agent does not support the demote action (optional)
* Your agent does not support the promote action (optional)
* Your agent does not support master/slave (optional)
* Your agent does not support the reload action (optional)
/usr/local/lib/ocf/resource.d/pacemaker/ifconfig passed all tests

Looks usable.

The ifconfig resource. Its pretty limited and with hardcoded IP address as for now.

ifconfig

Lets try to add new IP resource to our FreeBSD cluster.

Tests

root@node1:~ # crm configure primitive IP ocf:pacemaker:ifconfig op monitor interval="30"

Added.

Lets see what status command now shows.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:44:52 2020
  * Last change:  Wed Sep  2 22:44:44 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node1

Failed Resource Actions:
  * IP_monitor_0 on node3 'not installed' (5): call=24, status='Not installed', exitreason='', last-rc-change='2020-09-02 22:42:52Z', queued=0ms, exec=5ms
  * IP_monitor_0 on node2 'not installed' (5): call=24, status='Not installed', exitreason='', last-rc-change='2020-09-02 22:42:53Z', queued=0ms, exec=2ms

Crap. I forgot to copy this new ifconfig resource to the other nodes. Lets fix that now.

root@node1:~ # rsync -av /usr/local/lib/ocf/resource.d/pacemaker/ node2:/usr/local/lib/ocf/resource.d/pacemaker/
Password for root@node2:
sending incremental file list
./
ifconfig

sent 3,798 bytes  received 38 bytes  1,534.40 bytes/sec
total size is 128,003  speedup is 33.37

root@node1:~ # rsync -av /usr/local/lib/ocf/resource.d/pacemaker/ node3:/usr/local/lib/ocf/resource.d/pacemaker/
Password for root@node3:
sending incremental file list
./
ifconfig

sent 3,798 bytes  received 38 bytes  1,534.40 bytes/sec
total size is 128,003  speedup is 33.37

Lets stop, delete and re-add our precious resource now.

root@node1:~ # crm resource stop IP
root@node1:~ # crm configure delete IP
root@node1:~ # crm configure primitive IP ocf:pacemaker:ifconfig op monitor interval="30"

Fingers crossed.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:45:46 2020
  * Last change:  Wed Sep  2 22:45:43 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node1

Looks like running properly.

Lets verify that its really up where it should be.

root@node1:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:2a:78:60
        inet 10.0.10.111 netmask 0xffffff00 broadcast 10.0.10.255
        inet 10.0.10.200 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

root@node2:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:80:50:05
        inet 10.0.10.112 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

root@node3:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:74:5e:b9
        inet 10.0.10.113 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

Seems to be working.

Now lets try to move it to the other node in the cluster.

root@node1:~ # crm resource move IP node3
INFO: Move constraint created for IP to node3

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:47:31 2020
  * Last change:  Wed Sep  2 22:47:28 2020 by root via crm_resource on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node3

Switched properly to node3 system.

root@node3:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:74:5e:b9
        inet 10.0.10.113 netmask 0xffffff00 broadcast 10.0.10.255
        inet 10.0.10.200 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

root@node1:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:2a:78:60
        inet 10.0.10.111 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

Now we will poweroff the node3 system to check it that IP is really highly available.

root@node2:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:49:57 2020
  * Last change:  Wed Sep  2 22:47:29 2020 by root via crm_resource on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node3

root@node3:~ # poweroff

root@node2:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:50:16 2020
  * Last change:  Wed Sep  2 22:47:29 2020 by root via crm_resource on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 ]
  * OFFLINE: [ node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node1

Seems that failover went well.

The crm command also colors various sections of its output.

failover

Good to know that Pacemaker and Corosync cluster runs well on FreeBSD.

Some work is needed to write the needed resource files but one with some time and determination can surely put FreeBSD into a very capable highly available cluster.

EOF

FreeBSD Desktop โ€“ Part 21 โ€“ Configuration โ€“ Compton

In this article of the FreeBSD Desktop series I will talk Compton setup – the one that does not breaks, displays everything properly and does not consume 100% of your CPU time, as unfortunately Compton is a real bitch when it comes to proper setup.

The Compton is X11 compositor.

It allows the following features on X11 desktop:

  • transparent windows/menus/titlebars/borders
  • shadows and colored shadows
  • fading effects
  • background bluring

You may want to check other articles in the FreeBSD Desktop series on the FreeBSD Desktop – Global Page where you will find links to all episodes of the series along with table of contents for each episode’s contents.

Here is how example Compton looks in action.

compton

To install Compton on FreeBSD just use the default packages as shown below.

# pkg install compton

X11 Configuration

This is the graphics card configuration I have for X11:

% cat /usr/local/etc/X11/xorg.conf.d/card.conf
Section "Device"
  Identifier "Card0"
  Driver "modesetting"
  Option "DPMS"
  Option "AccelMethod" "glamor"
EndSection

… and the meritum of this article – the Compton config file:

% cat ~/.config/compton.conf
backend = "glx";
shadow = true;
no-dock-shadow = true;
clear-shadow = true;
shadow-radius = 12;
shadow-offset-x = -15;
shadow-offset-y = -15;
shadow-opacity = 0.7;
shadow-exclude = [
    "! name~=''",
    "name = 'Notification'",
    "name = 'Plank'",
    "name = 'Docky'",
    "name = 'Kupfer'",
    "name = 'xfce4-notifyd'",
    "name *= 'VLC'",
    "name *= 'compton'",
    "name *= 'Chromium'",
    "name *= 'Chrome'",
    "name *= 'Firefox'",
    "class_g = 'Conky'",
    "class_g = 'dzen'",
    "class_g = 'dzen2'",
    "class_g = 'Kupfer'",
    "class_g = 'Synapse'",
    "class_g ?= 'Notify-osd'",
    "class_g ?= 'Cairo-dock'",
    "class_g ?= 'Xfce4-notifyd'",
    "class_g ?= 'Xfce4-power-manager'"
];
shadow-ignore-shaped = false;
menu-opacity = 1;
inactive-opacity = 0.9;
active-opacity = 1;
frame-opacity = 0.9;
inactive-opacity-override = false;
alpha-step = 0.06;
blur-background-fixed = false;
blur-background-exclude = [
    "window_type = 'dock'",
    "window_type = 'desktop'"
];
fading = true;
fade-delta = 4;
fade-in-step = 0.03;
fade-out-step = 0.03;
fade-exclude = [ ];
mark-wmwin-focused = true;
mark-ovredir-focused = true;
use-ewmh-active-win = true;
detect-rounded-corners = true;
detect-client-opacity = true;
refresh-rate = 0;
vsync = "opengl-swc";
dbe = false;
paint-on-overlay = true;
sw-opti = false;
unredir-if-possible = true;
focus-exclude = [ ];
detect-transient = true;
detect-client-leader = true;
wintypes:
{
    tooltip =
    {
        fade = true;
        shadow = false;
        opacity = 0.85;
        focus = true;
    };
};

While the above config works very well I will also add same Compton configuration file but with comments.

% cat ~/.config/compton.conf
#################################
#
# Backend
#
#################################

# Backend to use: "xrender" or "glx".
# GLX backend is typically much faster but depends on a sane driver.
backend = "glx";

#################################
#
# GLX Backend
#
#################################

# GLX backend: Copy unmodified regions from front buffer instead of redrawing them all.
# Tests with nvidia-drivers show 10% decrease in performance when whole screen
# is modified but 20% increase when only 1/4 is modified.
# Tests on nouveau show terrible slowdown.
# Useful with --glx-swap-method as well.
# glx-copy-from-front = false;

# GLX backend: Use MESA_copy_sub_buffer to do partial screen update.
# Tests on nouveau shows 200% performance boost when only 1/4 of screen is updated.
# May break VSync and is not available on some drivers.
# Overrides --glx-copy-from-front.
# glx-use-copysubbuffermesa = true;

# GLX backend: Avoid rebinding pixmap on window damage.
# Probably could improve performance on rapid window content changes
# but is known to break things on some drivers (LLVMpipe).
# Recommended if it works.
# glx-no-rebind-pixmap = true;

# GLX backend: GLX buffer swap method we assume.
# Could be:
# - undefined (0)
# - copy (1)
# - exchange (2)
# - buffer-age (-1)
# The undefined is slowest and safest (default value).
# Copy is fastest but may fail on some drivers.
# buffer-age means auto-detect using GLX_EXT_buffer_age supported by some drivers.
# Useless with --glx-use-copysubbuffermesa.
# Partially breaks --resize-damage.
# Defaults to undefined.
# glx-swap-method = "undefined";

#################################
#
# Shadows
#
#################################

# Enabled client-side shadows on windows.
shadow = true;

# Do not draw shadows on DND windows.
# no-dnd-shadow = true;

# Avoid drawing shadows on dock/panel windows.
no-dock-shadow = true;

# Zero part of shadow's mask behind window. Fix some weirdness with ARGB windows.
clear-shadow = true;

# The blur radius for shadows. (default 12)
shadow-radius = 12;

# The left offset for shadows. (default -15)
shadow-offset-x = -15;

# The top offset for shadows. (default -15)
shadow-offset-y = -15;

# The translucency for shadows. (default .75)
shadow-opacity = 0.7;

# Set if you want different colour shadows
# shadow-red = 0.0;
# shadow-green = 0.0;
# shadow-blue = 0.0;

# The shadow exclude options are helpful if you have shadows enabled.
# Due to way compton draws its shadows certain applications will have
# visual glitches (most applications are fine - only apps that do weird
# things with xshapes or argb are affected).
# The "! name~=''" part excludes shadows on any "Unknown" windows.
# This prevents visual glitch with XFWM alt-tab switcher.
shadow-exclude = [
    "! name~=''",
    "name = 'Notification'",
    "name = 'Plank'",
    "name = 'Docky'",
    "name = 'Kupfer'",
    "name = 'xfce4-notifyd'",
    "name *= 'VLC'",
    "name *= 'compton'",
    "name *= 'Chromium'",
    "name *= 'Chrome'",
    "name *= 'Firefox'",
    "class_g = 'Conky'",
    "class_g = 'dzen'",
    "class_g = 'dzen2'",
    "class_g = 'Kupfer'",
    "class_g = 'Synapse'",
    "class_g ?= 'Notify-osd'",
    "class_g ?= 'Cairo-dock'",
    "class_g ?= 'Xfce4-notifyd'",
    "class_g ?= 'Xfce4-power-manager'"
];

# Avoid drawing shadow on all shaped windows (see also: --detect-rounded-corners)
shadow-ignore-shaped = false;

#################################
#
# Opacity
#
#################################

# Opacity for menu items.
menu-opacity = 1;

# Opacity for inactive windows.
inactive-opacity = 0.9;

# Opacity for active windows.
active-opacity = 1;

# Opacity for active frame of windows.
frame-opacity = 0.9;

# Opacity for inactive frame of windows.
inactive-opacity-override = false;

# Alpha step.
alpha-step = 0.06;

# Dim inactive windows. (0.0 - 1.0)
# inactive-dim = 0.2;

# Do not let dimness adjust based on window opacity.
# inactive-dim-fixed = true;

# Blur background of transparent windows. Bad performance with X Render backend.
# GLX backend is preferred.
# blur-background = true;

# Blur background of opaque windows with transparent frames as well.
# blur-background-frame = true;

# Do not let blur radius adjust based on window opacity.
blur-background-fixed = false;

# Blue exclude list.
blur-background-exclude = [
    "window_type = 'dock'",
    "window_type = 'desktop'"
];

#################################
#
# Fading
#
#################################

# Fade windows during opacity changes.
fading = true;

# The time between steps in fade in milliseconds (default 10).
fade-delta = 4;

# Opacity change between steps while fading in (default 0.028).
fade-in-step = 0.03;

# Opacity change between steps while fading out (default 0.03).
fade-out-step = 0.03;

# Fade windows in/out when opening/closing
# no-fading-openclose = true;

# Specify a list of conditions of windows that should not be faded.
fade-exclude = [ ];

#################################
#
# Other
#
#################################

# Try to detect WM windows and mark them as active.
mark-wmwin-focused = true;

# Mark all non-WM but override-redirect windows active (e.g. menus).
mark-ovredir-focused = true;

# Use EWMH _NET_WM_ACTIVE_WINDOW to determine which window is focused instead of
# using FocusIn/Out events. Usually more reliable but depends on EWMH-compliant WM.
use-ewmh-active-win = true;

# Detect rounded corners and treat them as rectangular when --shadow-ignore-shaped is on.
detect-rounded-corners = true;

# Detect _NET_WM_OPACITY on client windows useful for window managers not passing
# _NET_WM_OPACITY of client windows to frame windows. This prevents opacity ignore
# for some apps. Without this enabled xfce4-notifyd is 100% opacity no matter what.
detect-client-opacity = true;

# Specify refresh rate. With 0 compton will detect this with X RandR extension.
refresh-rate = 0;

# Set VSync method. VSync methods currently available:
# - none: No VSync
# - drm: VSync with DRM_IOCTL_WAIT_VBLANK. May only work on some drivers.
# - opengl: VSync with SGI_video_sync OpenGL extension. Only on some drivers.
# - opengl-oml: VSync with OML_sync_control OpenGL extension. Only on some drivers.
# - opengl-swc: VSync with SGI_swap_control OpenGL extension. Only on some drivers.
#               Works with GLX backend. Known to be most effective on many drivers.
#               Does not control paint timing - only buffer swap is affected.
#               Does not have effect of --sw-opti unlike other methods. Experimental.
# - opengl-mswc: Try to VSync with MESA_swap_control OpenGL extension.
#                Basically same as opengl-swc above except extension we use.
vsync = "opengl-swc";

# Enable DBE painting mode - use with VSync to (hopefully) eliminate tearing.
dbe = false;

# Painting on X Composite overlay window. Recommended.
paint-on-overlay = true;

# Limit repaint at most once every 1 / refresh_rate second to boost performance.
# This should not be used with --vsync drm/opengl/opengl-oml as they essentially does
# --sw-opti* job unless you wish to have lower refresh rate than actual value.
sw-opti = false;

# Unredirect all windows if full-screen window is detected to maximize performance
# for full-screen windows - like games. Known to cause flickering when
# redirecting/unredirecting windows. Paint-on-overlay may flicker less.
unredir-if-possible = true;

# Specify list of conditions of windows that should always be considered focused.
focus-exclude = [ ];

# Use WM_TRANSIENT_FOR to group windows in same group focused at same time.
detect-transient = true;

# Use WM_CLIENT_LEADER to group windows in same group focused at same time.
# WM_TRANSIENT_FOR has higher priority if --detect-transient is enabled too.
detect-client-leader = true;

#################################
#
# Window Type Settings
#
#################################

wintypes:
{
    tooltip =
    {
        # fade: Fade particular type of windows.
        fade = true;
        # shadow: Give those windows shadow
        shadow = false;
        # opacity: Default opacity for type of windows.
        opacity = 0.85;
        # focus: Whether to always consider windows of this type focused.
        focus = true;
    };
};


Not sure what else could I add here so this means the end of this article ๐Ÿ™‚

EOF

FreeBSD Enterprise Storage at PBUG

Yesterday I was honored to give a talk about FreeBSD Enterprise Storage at the Polish BSD User Group meeting.

You are invited to download the PDF Slides โ€“ https://is.gd/bsdstg โ€“ available here.

bsdstg

The PBUG (Polish BSD User Group) meetings are very special. In “The Matrix” movie (which has been rendered on FreeBSD system by the way) – FreeBSD Used to Generate Spectacular Special Effects – details available here – its not possible to describe what the Matrix really is, one has to feel it. Enter it. The same I can tell you about the PBUG meetings. Its kinda like with the “Hangover” movie. What happens in Vegas PBUG meeting stays in Vegas PBUG meeting ๐Ÿ™‚

If you will have the possibility and time then join the next Polish BSD User Group meeting. You will not regret it :>

UPDATE 1 – Shorter Unified Version

The original – https://is.gd/bsdstg – presentation is 187 pages long and is suited for live presentation while not the best for later ‘offline’ view.

I have created a unified version – https://is.gd/bsdstguni – with only 42 pages.

EOF

Run broot on FreeBSD

The broot file manager is quite fresh and nice approach to files and directories filtering/searching/view/manipulation/… and whatever else you call messing with files ๐Ÿ™‚

The broot tools is not yet available on the FreeBSD systems (as package or port).

This guide will show you how to built and install it on your FreeBSD system.

Here is how it looks in action.

Filter for jails.

broot-filter-jails.jpg

Filter for zfs.

broot-filter-zfs.jpg

It has ‘size mode’ when started with -s option similar to ncdu(1) tool.

broot-filter-size.jpg

You can also check the Feature Showcase section on their GitHub page – https://github.com/Canop/broot – available here.

Build

There are three steps to make it happen.

1. You need to install the rust package.

# pkg install rust

Then you need to type (as regular user) the cargo install broot command.

% cargo install broot

It will fail here:

broot-fail.jpg

You will need to apply this patch below:

% diff -u \
  /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs.ORG \
  /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs
--- /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs.ORG  2020-01-10 23:41:29.825912000 +0100
+++ /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs      2020-01-10 23:41:07.703471000 +0100
@@ -33,7 +33,7 @@
         ws_ypixel: 0,
     };
 
-    if let Ok(true) = wrap_with_result(unsafe { ioctl(STDOUT_FILENO, TIOCGWINSZ, &mut size) }) {
+    if let Ok(true) = wrap_with_result(unsafe { ioctl(STDOUT_FILENO, TIOCGWINSZ.into(), &mut size) }) {
         Ok((size.ws_col, size.ws_row))
     } else {
         tput_size().ok_or_else(|| std::io::Error::last_os_error().into())

Then type cargo install broot command again. It will now properly compile.

% cargo install broot
    Updating crates.io index
  Downloaded broot v0.11.6
  Downloaded 1 crate (1.6 MB) in 2.89s
  Installing broot v0.11.6
   Compiling libc v0.2.66
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling autocfg v0.1.7
   Compiling semver-parser v0.7.0
   Compiling autocfg v1.0.0
   Compiling proc-macro2 v1.0.7
   Compiling log v0.4.8
   Compiling scopeguard v1.0.0
   Compiling unicode-xid v0.2.0
   Compiling bitflags v1.2.1
   Compiling syn v1.0.13
   Compiling memchr v2.2.1
   Compiling arc-swap v0.4.4
   Compiling slab v0.4.2
   Compiling smallvec v1.1.0
   Compiling serde v1.0.104
   Compiling unicode-width v0.1.7
   Compiling regex-syntax v0.6.13
   Compiling ansi_term v0.11.0
   Compiling strsim v0.8.0
   Compiling vec_map v0.8.1
   Compiling id-arena v2.2.1
   Compiling custom_error v1.7.1
   Compiling glob v0.3.0
   Compiling open v1.3.2
   Compiling umask v0.1.8
   Compiling thread_local v1.0.0
   Compiling minimad v0.6.3
   Compiling lazy-regex v0.1.2
   Compiling semver v0.9.0
   Compiling lock_api v0.3.3
   Compiling crossbeam-utils v0.7.0
   Compiling crossbeam-epoch v0.8.0
   Compiling num-traits v0.2.11
   Compiling num-integer v0.1.42
   Compiling textwrap v0.11.0
   Compiling rustc_version v0.2.3
   Compiling memoffset v0.5.3
   Compiling iovec v0.1.4
   Compiling net2 v0.2.33
   Compiling dirs-sys v0.3.4
   Compiling parking_lot_core v0.7.0
   Compiling signal-hook-registry v1.2.0
   Compiling time v0.1.42
   Compiling atty v0.2.14
   Compiling users v0.9.1
   Compiling quote v1.0.2
   Compiling aho-corasick v0.7.6
   Compiling mio v0.6.21
   Compiling dirs v2.0.2
   Compiling directories v2.0.2
   Compiling parking_lot v0.10.0
   Compiling clap v2.33.0
   Compiling crossbeam-queue v0.2.1
   Compiling crossbeam-channel v0.4.0
   Compiling toml v0.5.5
   Compiling term v0.6.1
   Compiling regex v1.3.3
   Compiling signal-hook v0.1.12
   Compiling chrono v0.4.10
   Compiling crossterm v0.14.1
   Compiling simplelog v0.7.4
   Compiling crossbeam-deque v0.7.2
   Compiling thiserror-impl v1.0.9
   Compiling crossbeam v0.7.3
   Compiling thiserror v1.0.9
   Compiling termimad v0.8.9
   Compiling broot v0.11.6
    Finished release [optimized] target(s) in 4m 56s
  Installing /home/vermaden/.cargo/bin/broot
   Installed package `broot v0.11.6` (executable `broot`)
warning: be sure to add `/home/vermaden/.cargo/bin` to your PATH to be able to run the installed binaries

% echo $?
0

Install

Now go to the ~/.cargo/bin directory and copy the broot binary to some place that is set in your ${PATH} variable.

Then start new terminal (updated ${PATH} variable) and type broot command.

% cp ~/.cargo/bin/broot ~/scripts
% rehash
% broot

You will be asked if automatic setup of the br function should tool place. I agreed with y answer.

broot-first-run.jpg

Here are things generated by this process.

% find ~/.config/broot
/home/vermaden/.config/broot
/home/vermaden/.config/broot/conf.toml
/home/vermaden/.config/broot/launcher
/home/vermaden/.config/broot/launcher/installed-v1
/home/vermaden/.config/broot/launcher/bash
/home/vermaden/.config/broot/launcher/bash/br

% find ~/.local/share/broot
/home/vermaden/.local/share/broot
/home/vermaden/.local/share/broot/launcher
/home/vermaden/.local/share/broot/launcher/fish
/home/vermaden/.local/share/broot/launcher/fish/1.fish
/home/vermaden/.local/share/broot/launcher/bash
/home/vermaden/.local/share/broot/launcher/bash/1

As I use ZSH shell it also updates my ~/.zshrc file.

% tail -3 ~/.zshrc

source /home/vermaden/.config/broot/launcher/bash/br

Finished. You now have broot installed and ready to use.

broot-filter-bhyve.jpg

UPDATE 1 – Now No Patches Are Needed

Thanks to the broot author any patches are now not needed.

It builds and works out of the box.

broot-update-fixed

UPDATE 2 – Its in Ports/Packages Now

The broot file manager is now available via usual FreeBSD Ports and packages which makes this guide pointless ๐Ÿ™‚

Its available as misc/broot port.

EOF

ย 

FreeBSD Desktop – Part 20 – Configuration – Unlock Your Laptop with Phone

I really do not like the smart card ecosystem – probably because it will be a big PITA to setup such subsystem on FreeBSD to make it lock/unlock my laptop with a smart card – not to mention of it will be even possible because of probable lack of drivers for a laptop builtin smart card reader. I mention it because you can lock and unlock your laptop with such smart card in very fast way.

Some people use finger prints readers (for fast workstation/laptop unlock purpose) – but its the same case scenario as with smart card – the time needed to setup it properly. Not to mention that is not that fast anyway as I often see my colleagues swinging the finger over the fingerprint reader over and over again so it will finally work the 7th time …

… but you wan also lock and unlock your UNIX laptop with your phone – by just attaching it to your device – this is where the FreeBSD’s devd(8) subsystem come handy.

Today I will show you how to lock/unlock your laptop with your phone.

You may want to check other articles in the FreeBSD Desktop series on the FreeBSD Desktop – Global Page where you will find links to all episodes of the series along with table of contents for each episodeโ€™s contents.

Keep in mind that in order to make it work you need to attach the phone to laptop using cable that supports data transfer – it will not work with cables that only provide power for charging your phone.

Device Detection

First we need to detect what device will be your locker/unlocker.

Stop the devd(8) daemon.

# service devd stop
Stopping devd.
Waiting for PIDS: 71455.

Now start it in ‘foreground’ for debug purposes and then attach your phone. The command below with grep(1) will help you to find needed information.

# devd -d 2>&1 | grep --line-buffered 'Processing event' | grep --line-buffered DEVICE
Processing event '!system=USB subsystem=DEVICE type=ATTACH ugen=ugen2.3 cdev=ugen2.3 vendor=0x04e8 product=0x6860 devclass=0x00 devsubclass=0x00 sernum="31000e243eb5a12e" release=0x0400 mode=host port=2 parent=ugen2.2'

I have highlited the needed information.

Do not stop this process yet.

Now you know which device will be your locker/unlocker and what even the devd(8) daemon gets when you attach your phone.

Things to note hare are:

vendor=0x04e8
product=0x6860
sernum=31000e243eb5a12e

This data above is more then enough to unlock your workstation.

Now detach your phone from the computer. You will see the DETACH even similar to the one below.

Processing event '!system=USB subsystem=DEVICE type=DETACH ugen=ugen2.3 cdev=ugen2.3 vendor=0x04e8 product=0x6860 devclass=0x00 devsubclass=0x00 sernum="31000e243eb5a12e" release=0x0400 mode=host port=2 parent=ugen2.2'

Now you know the event that will be spawned when you detach your phone.

Stop the foreground devd(8) daemon and start the service traditionally.

# devd -d 2>&1 | grep --line-buffered 'Processing event' | grep --line-buffered DEVICE
Processing event '!system=USB subsystem=DEVICE type=ATTACH ugen=ugen2.3 cdev=ugen2.3 vendor=0x04e8 product=0x6860 devclass=0x00 devsubclass=0x00 sernum="31000e243eb5a12e" release=0x0400 mode=host port=2 parent=ugen2.2'
Processing event '!system=USB subsystem=DEVICE type=DETACH ugen=ugen2.3 cdev=ugen2.3 vendor=0x04e8 product=0x6860 devclass=0x00 devsubclass=0x00 sernum="31000e243eb5a12e" release=0x0400 mode=host port=2 parent=ugen2.2'
^C
# service devd start
Starting devd.

Commands for Events

Now, what action or command should be executed when you attach or detach your phone? That depends on which screen locker you are using on your X11 setup.

I for example use the mate-screensaver for this purpose.

The ATTACH event in my case would be to kill the current process mate-screensaver which will unlock the screen and then start it again for the next lock purposes – below is the command that I will run for the ATTACH event.

pkill -9 mate-screensaver && su -l vermaden -c 'env DISPLAY=:0 mate-screensaver' &

The DETACH event will be notifying the mate-screensaver to lock the screen – here is the command that will be used for that purpose.

su -l vermaden -c 'env DISPLAY=:0 mate-screensaver-command --lock' &

Implementation

Here is how the devd(8) config file for my phone would look like.

# cat /usr/local/etc/devd/phonelock.conf

# PHONE ATTACH - UNLOCK
notify 100 {
    match "system" "USB";
    match "subsystem" "DEVICE";
    match "type" "ATTACH";
    match "vendor" "0x04e8";
    match "product" "0x6860";
    match "sernum" "31000e243eb5a12e";
    action "pkill -9 mate-screensaver && su -l vermaden -c 'env DISPLAY=:0 mate-screensaver' &";
};

# PHONE DETACH - LOCK
notify 100 {
    match "system" "USB";
    match "subsystem" "DEVICE";
    match "type" "DETACH";
    match "vendor" "0x04e8";
    match "product" "0x6860";
    match "sernum" "31000e243eb5a12e";
    action "su -l vermaden -c 'env DISPLAY=:0 mate-screensaver-command --lock' &";
};

Now restart the devd(8) daemon so it will read new configuration files.

# service devd restart
Stopping devd.
Waiting for PIDS: 1458.
Starting devd.

Viola! Now you can lock and unlock your screen just by attaching or detaching your phone. I do not have any fancy video on how it behaves but you must trust me that is less then a second to lock and unlock the laptop now – be sure to keep and additional eye on your phone now, as it can unlock the access to all your files now ๐Ÿ™‚

You can of course use any USB device or even network actions – any event that is supported by the devd(8) daemon.

You can of course create such lock/unlock config when you attach/detach your phone and additionally configure power down action when you detach other USB device.

I forgot to mention it, that method does not disables the ‘classic’ password authentication – it just adds automatic screen lock/unlock when you attach your phone – you can still login (unlock) using just password on the mate-screensaver lock screen.

UPDATE 1 – Better devd Sniffing – Better Unlock Method

As oh5nxo from Reddit suggested its not needed to stop devd and start it in ‘debug’ mode – its easier just to attach to its ‘pipe’ with nc(1) tool.

# nc -U /var/run/devd.pipe

There is also no need to kill(1) the mate-screensaver command, its more elegant to just send the mate-screensaver-command --unlock command.

Below is the updated /usr/local/etc/devd/phonelock.conf config file for the devd(8) daemon.

# cat /usr/local/etc/devd/phonelock.conf

# PHONE ATTACH - UNLOCK
notify 100 {
    match "system" "USB";
    match "subsystem" "DEVICE";
    match "type" "ATTACH";
    match "vendor" "0x04e8";
    match "product" "0x6860";
    match "sernum" "33000e343fb4a42d";
    action "su -l vermaden -c 'env DISPLAY=:0 mate-screensaver-command --unlock' &";
};

# PHONE DETACH - LOCK
notify 100 {
    match "system" "USB";
    match "subsystem" "DEVICE";
    match "type" "DETACH";
    match "vendor" "0x04e8";
    match "product" "0x6860";
    match "sernum" "33000e343fb4a42d";
    action "su -l vermaden -c 'env DISPLAY=:0 mate-screensaver-command --lock' &";
};

EOF