Tag Archives: server

Connect FreeBSD to FreeIPA/Red Hat Identity Management

Corporate needs are simple – one ring to rule them all place to get users from. On the open source path there are several ways to achieve that. Alongside ‘plain’ OpenLDAP there is also FreeIPA – which is open source and free version of the Red Hat Identity Management (IDM).

FreeIPA-logo

This guide will show you how to make basic FreeIPA install and connect a FreeBSD 13.1-RELEASE system to it.

logo-freebsd

The Table of Contents for this article looks as follows.

  • Connect FreeBSD to FreeIPA/Red Hat Identity Management
  • FreeIPA
    • FreeIPA Requirements
    • FreeIPA Setup
  • FreeBSD Client
    • FreeBSD Packages
    • FreeBSD Setup
    • FreeIPA/IDM Setup Part
    • Finish Setup with Web Browser in FreeIPA/IDM Page
    • FreeBSD FreeIPA Login Test
  • FreeBSD Jail as FreeIPA/IDM Client
    • Basic FreeBSD Jail Preparations
    • Configure FreeBSD Jail to Connect to FreeIPA/IDM Server
  • Linux FreeIPA/IDM Client
    • FreeIPA/IDM Setup Part
    • Linux FreeIPA/IDM Client Setup

FreeIPA

As typical CentOS is dead (yes there is CentOS Stream available but its not a RHEL clone) I have two options here. Rocky Linux and Alma Linux. I will use the latter as it seems more popular and more up to date. In this guide the FreeIPA/IDM server will hosted on the Alma Linux 8.6 system.

To make things easier I installed that Alma Linux 8.6 with single 20GB / root on the /dev/sda1 partition. No separate /boot. No LVM. Just good old plain fucking simple single raw partition for everything. Seems no one does it these days πŸ™‚

[root@idm ~]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0   20G  0 disk
└─sda1   8:1    0   20G  0 part /

FreeIPA Requirements

The minimum hardware requirements for installing FreeIPA server are as follows.

RAM: 4GB
CPU: 2
HDD: 10GB
DNS: fully qualified domain name for FreeIPA
     must be resolvable from DNS server configured in system

For this installation I have used the 10.0.0.0/24 network.

The FreeIPA/IDM server will get the 10.0.0.40 IP.

The FreeIPA/IDM Linux client rhlike.vercorp.org will get the 10.0.0.41 IP.

The FreeIPA/IDM FreeBSD client fbsd.vercorp.org will get the 10.0.0.42 IP.

The FreeIPA/IDM FreeBSD client fbsdjail.vercorp.org will get the 10.0.0.43 IP.

For lack of better ideas I have used vercorp.org/VERCORP.ORG as domain/realm names and idm.vercorp.org as the FreeIPA/IDM hostname.

Details of FreeIPA/IDM idm.vercorp.org system below.

      IP: 10.0.0.40/24
      GW: 10.0.0.1
     DNS: 1.1.1.1
hostname: idm.vercorp.org
  domain: vercorp.org
   realm: VERCORP.ORG

FreeBSD fbsd.vercorp.org system.

      IP: 10.0.0.42/24
      GW: 10.0.0.1
hostname: fbsd.vercorp.org

FreeBSD Jail fbsdjail.vercorp.org system.

      IP: 10.0.0.43/24
      GW: 10.0.0.1
hostname: fbsdjail.vercorp.org

If you are curious what is hidden in the 10.0.0.42 IP – then its a typical Alma Linux that I first used to try if the FreeIPA/IDM works at all πŸ™‚

FreeIPA Setup

Because Anaconda is far from being a usable installer – this is how the only enp0s3 interface config looks like after manual intervention.

[root@idm ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s3
NAME=enp0s3
DEVICE=enp0s3
TYPE=Ethernet
BOOTPROTO=none
UUID=b49cd1ab-d3eb-421d-b408-c052acc077da
ONBOOT=yes
IPADDR=10.0.0.40
NETMASK=255.255.255.0
GATEWAY=10.0.0.1
IPV6INIT=no
DNS1=1.1.1.1
PROXY_METHOD=none
BROWSER_ONLY=no
DEFROUTE=yes
IPV4_FAILURE_FATAL=no

Before we will go into setup procedures – we will update that Alma Linux system.

[root@idm ~]# yum update -y
[root@idm ~]# reboot

After the reboot we will disable the IPv6 stack for main interface (enp0s3) as FreeIPA installer has some problem with it. We do not need IPv6 here anyway so …

[root@idm ~]# cat << EOF >> /etc/sysctl.conf
# DISABLE IPv6 FOR MAIN enp0s3 INTERFACE
net.ipv6.conf.enp0s3.disable_ipv6=1
EOF

Lets set our FreeIPA/IDM hostname if we missed that at the Anaconda installer part. We will also setup the system time zone.

[root@idm ~]# hostnamectl set-hostname idm.vercorp.org

[root@idm ~]# hostnamectl
   Static hostname: idm.vercorp.org
         Icon name: computer-vm
           Chassis: vm
        Machine ID: b8bfb8bcc23147eb9cc7b62c72a09c32
           Boot ID: 06158ef430d9467d959076ab4396314e
    Virtualization: oracle
  Operating System: AlmaLinux 8.6 (Sky Tiger)
       CPE OS Name: cpe:/o:almalinux:almalinux:8::baseos
            Kernel: Linux 4.18.0-372.26.1.el8_6.x86_64
      Architecture: x86-64

[root@idm ~]# timedatectl set-timezone Europe/Warsaw

[root@idm ~]# timedatectl set-local-rtc 0

[root@idm ~]#Β timedatectl
               Local time: Mon 2022-10-17 15:08:28 CEST
           Universal time: Mon 2022-10-17 13:08:28 UTC
                 RTC time: Sat 2022-10-15 00:28:10
                Time zone: Europe/Warsaw (CEST, +0200)
System clock synchronized: yes
              NTP service: inactive
          RTC in local TZ: no

Lets add the system IP and name to the /etc/hosts file now.

[root@idm ~]# echo "$( hostname -i | awk '{print $NF}' )   $( hostname ) $( hostname -s )" >> /etc/hosts

[root@idm ~]# grep idm /etc/hosts
10.0.0.40   idm.vercorp.org idm

[root@idm ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.40   idm.vercorp.org idm

We need to enable the FreeIPA/IDM module in yum(8).

[root@idm ~]# yum module list idm
Last metadata expiration check: 0:00:31 ago on Tue 18 Oct 2022 01:02:51 PM CEST.
AlmaLinux 8 - AppStream
Name  Stream      Profiles                                  Summary
idm   DL1         adtrust, client, common [d], dns, server  The Red Hat Enterprise Linux Identity Management system module
idm   client [d]  common [d]                                RHEL IdM long term support client module

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

[root@idm ~]# yum module enable idm:DL1 -y
Last metadata expiration check: 0:01:06 ago on Tue 18 Oct 2022 01:02:51 PM CEST.
Dependencies resolved.
=====================================================================================================
 Package                 Architecture           Version                  Repository              Size
=====================================================================================================
Enabling module streams:
 389-ds                                         1.4
 httpd                                          2.4
 idm                                            DL1
 pki-core                                       10.6
 pki-deps                                       10.6

Transaction Summary
=====================================================================================================

Complete!

Some additional step needed.

[root@idm ~]# yum distro-sync
Last metadata expiration check: 0:10:07 ago on Mon 17 Oct 2022 03:14:32 PM CEST.
Dependencies resolved.
Nothing to do.
Complete!

We will now install FreeIPA/IDM with DNS as this setup is the most simplistic one. We focus on FreeBSD part here in that article.

[root@idm ~]# yum install -y bind-utils chrony nc
[root@idm ~]# yum module install idm:DL1/dns -y

One can use the ‘interactive’ installer and answer ‘by hand’ for all the asked questions – but to be honest I prefer to type my command once and make it ‘happen’ altogether without my time wasted.

As you probably guessed – we will use the unattended mode for the FreeIPA/IDM installer.

[root@idm ~]# ipa-server-install \
    --domain vercorp.org \
    --realm VERCORP.ORG \
    --reverse-zone=0.0.10.in-addr.arpa. \
    --no-forwarders \
    --no-ntp \
    --setup-dns \
    --ds-password    [password] \
    --admin-password [password] \
    --unattended

Checking DNS domain 0.0.10.in-addr.arpa., please wait ...

The log file for this installation can be found in /var/log/ipaserver-install.log
==============================================================================
This program will set up the IPA Server.
Version 4.9.8

This includes:
  * Configure a stand-alone CA (dogtag) for certificate management
  * Create and configure an instance of Directory Server
  * Create and configure a Kerberos Key Distribution Center (KDC)
  * Configure Apache (httpd)
  * Configure DNS (bind)
  * Configure SID generation
  * Configure the KDC to enable PKINIT

Excluded by options:
  * Configure the NTP client (chronyd)

Warning: skipping DNS resolution of host idm.vercorp.org
Checking DNS domain vercorp.org., please wait ...
Checking DNS domain 0.0.10.in-addr.arpa., please wait ...
Checking DNS domain 0.0.10.in-addr.arpa., please wait ...
Using reverse zone(s) 0.0.10.in-addr.arpa.
Trust is configured but no NetBIOS domain name found, setting it now.

The IPA Master Server will be configured with:
Hostname:       idm.vercorp.org
IP address(es): 10.0.0.40
Domain name:    vercorp.org
Realm name:     VERCORP.ORG

The CA will be configured with:
Subject DN:   CN=Certificate Authority,O=VERCORP.ORG
Subject base: O=VERCORP.ORG
Chaining:     self-signed

BIND DNS server will be configured to serve IPA domain with:
Forwarders:       No forwarders
Forward policy:   only
Reverse zone(s):  0.0.10.in-addr.arpa.

Disabled p11-kit-proxy
Configuring directory server (dirsrv). Estimated time: 30 seconds
  [1/41]: creating directory server instance
Validate installation settings ...
Create file system structures ...
Perform SELinux labeling ...
Create database backend: dc=vercorp,dc=org ...
Perform post-installation tasks ...
  [2/41]: tune ldbm plugin
  [3/41]: adding default schema
  [4/41]: enabling memberof plugin
  [5/41]: enabling winsync plugin
  [6/41]: configure password logging
  [7/41]: configuring replication version plugin
  [8/41]: enabling IPA enrollment plugin
  [9/41]: configuring uniqueness plugin
  [10/41]: configuring uuid plugin
  [11/41]: configuring modrdn plugin
  [12/41]: configuring DNS plugin
  [13/41]: enabling entryUSN plugin
  [14/41]: configuring lockout plugin
  [15/41]: configuring topology plugin
  [16/41]: creating indices
  [17/41]: enabling referential integrity plugin
  [18/41]: configuring certmap.conf
  [19/41]: configure new location for managed entries
  [20/41]: configure dirsrv ccache and keytab
  [21/41]: enabling SASL mapping fallback
  [22/41]: restarting directory server
  [23/41]: adding sasl mappings to the directory
  [24/41]: adding default layout
  [25/41]: adding delegation layout
  [26/41]: creating container for managed entries
  [27/41]: configuring user private groups
  [28/41]: configuring netgroups from hostgroups
  [29/41]: creating default Sudo bind user
  [30/41]: creating default Auto Member layout
  [31/41]: adding range check plugin
  [32/41]: creating default HBAC rule allow_all
  [33/41]: adding entries for topology management
  [34/41]: initializing group membership
  [35/41]: adding master entry
  [36/41]: initializing domain level
  [37/41]: configuring Posix uid/gid generation
  [38/41]: adding replication acis
  [39/41]: activating sidgen plugin
  [40/41]: activating extdom plugin
  [41/41]: configuring directory to start on boot
Done configuring directory server (dirsrv).
Configuring Kerberos KDC (krb5kdc)
  [1/10]: adding kerberos container to the directory
  [2/10]: configuring KDC
  [3/10]: initialize kerberos container
  [4/10]: adding default ACIs
  [5/10]: creating a keytab for the directory
  [6/10]: creating a keytab for the machine
  [7/10]: adding the password extension to the directory
  [8/10]: creating anonymous principal
  [9/10]: starting the KDC
  [10/10]: configuring KDC to start on boot
Done configuring Kerberos KDC (krb5kdc).
Configuring kadmin
  [1/2]: starting kadmin
  [2/2]: configuring kadmin to start on boot
Done configuring kadmin.
Configuring ipa-custodia
  [1/5]: Making sure custodia container exists
  [2/5]: Generating ipa-custodia config file
  [3/5]: Generating ipa-custodia keys
  [4/5]: starting ipa-custodia
  [5/5]: configuring ipa-custodia to start on boot
Done configuring ipa-custodia.
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/29]: configuring certificate server instance
  [2/29]: stopping certificate server instance to update CS.cfg
  [3/29]: backing up CS.cfg
  [4/29]: Add ipa-pki-wait-running
  [5/29]: secure AJP connector
  [6/29]: reindex attributes
  [7/29]: exporting Dogtag certificate store pin
  [8/29]: disabling nonces
  [9/29]: set up CRL publishing
  [10/29]: enable PKIX certificate path discovery and validation
  [11/29]: authorizing RA to modify profiles
  [12/29]: authorizing RA to manage lightweight CAs
  [13/29]: Ensure lightweight CAs container exists
  [14/29]: Ensuring backward compatibility
  [15/29]: starting certificate server instance
  [16/29]: configure certmonger for renewals
  [17/29]: requesting RA certificate from CA
  [18/29]: publishing the CA certificate
  [19/29]: adding RA agent as a trusted user
  [20/29]: configure certificate renewals
  [21/29]: Configure HTTP to proxy connections
  [22/29]: updating IPA configuration
  [23/29]: enabling CA instance
  [24/29]: importing IPA certificate profiles
  [25/29]: migrating certificate profiles to LDAP
  [26/29]: adding default CA ACL
  [27/29]: adding 'ipa' CA entry
  [28/29]: configuring certmonger renewal for lightweight CAs
  [29/29]: deploying ACME service
Done configuring certificate server (pki-tomcatd).
Configuring directory server (dirsrv)
  [1/3]: configuring TLS for DS instance
  [2/3]: adding CA certificate entry
  [3/3]: restarting directory server
Done configuring directory server (dirsrv).
Configuring ipa-otpd
  [1/2]: starting ipa-otpd
  [2/2]: configuring ipa-otpd to start on boot
Done configuring ipa-otpd.
Configuring the web interface (httpd)
  [1/22]: stopping httpd
  [2/22]: backing up ssl.conf
  [3/22]: disabling nss.conf
  [4/22]: configuring mod_ssl certificate paths
  [5/22]: setting mod_ssl protocol list
  [6/22]: configuring mod_ssl log directory
  [7/22]: disabling mod_ssl OCSP
  [8/22]: adding URL rewriting rules
  [9/22]: configuring httpd
Nothing to do for configure_httpd_wsgi_conf
  [10/22]: setting up httpd keytab
  [11/22]: configuring Gssproxy
  [12/22]: setting up ssl
  [13/22]: configure certmonger for renewals
  [14/22]: publish CA cert
  [15/22]: clean up any existing httpd ccaches
  [16/22]: enable ccache sweep
  [17/22]: configuring SELinux for httpd
  [18/22]: create KDC proxy config
  [19/22]: enable KDC proxy
  [20/22]: starting httpd
  [21/22]: configuring httpd to start on boot
  [22/22]: enabling oddjobd
Done configuring the web interface (httpd).
Configuring Kerberos KDC (krb5kdc)
  [1/1]: installing X509 Certificate for PKINIT
Done configuring Kerberos KDC (krb5kdc).
Applying LDAP updates
Upgrading IPA:. Estimated time: 1 minute 30 seconds
  [1/10]: stopping directory server
  [2/10]: saving configuration
  [3/10]: disabling listeners
  [4/10]: enabling DS global lock
  [5/10]: disabling Schema Compat
  [6/10]: starting directory server
  [7/10]: upgrading server
  [8/10]: stopping directory server
  [9/10]: restoring configuration
  [10/10]: starting directory server
Done.
Restarting the KDC
dnssec-validation yes
Configuring DNS (named)
  [1/12]: generating rndc key file
  [2/12]: adding DNS container
  [3/12]: setting up our zone
  [4/12]: setting up reverse zone
  [5/12]: setting up our own record
  [6/12]: setting up records for other masters
  [7/12]: adding NS record to the zones
  [8/12]: setting up kerberos principal
  [9/12]: setting up named.conf
created new /etc/named.conf
created named user config '/etc/named/ipa-ext.conf'
created named user config '/etc/named/ipa-options-ext.conf'
created named user config '/etc/named/ipa-logging-ext.conf'
  [10/12]: setting up server configuration
  [11/12]: configuring named to start on boot
  [12/12]: changing resolv.conf to point to ourselves
Done configuring DNS (named).
Restarting the web server to pick up resolv.conf changes
Configuring DNS key synchronization service (ipa-dnskeysyncd)
  [1/7]: checking status
  [2/7]: setting up bind-dyndb-ldap working directory
  [3/7]: setting up kerberos principal
  [4/7]: setting up SoftHSM
  [5/7]: adding DNSSEC containers
  [6/7]: creating replica keys
  [7/7]: configuring ipa-dnskeysyncd to start on boot
Done configuring DNS key synchronization service (ipa-dnskeysyncd).
Restarting ipa-dnskeysyncd
Restarting named
Updating DNS system records
Configuring SID generation
  [1/8]: creating samba domain object
  [2/8]: adding admin(group) SIDs
  [3/8]: adding RID bases
  [4/8]: updating Kerberos config
'dns_lookup_kdc' already set to 'true', nothing to do.
  [5/8]: activating sidgen task
  [6/8]: restarting Directory Server to take MS PAC and LDAP plugins changes into account
  [7/8]: adding fallback group
  [8/8]: adding SIDs to existing users and groups
This step may take considerable amount of time, please wait..
Done.
Configuring client side components
This program will set up IPA client.
Version 4.9.8

Using existing certificate '/etc/ipa/ca.crt'.
Client hostname: idm.vercorp.org
Realm: VERCORP.ORG
DNS Domain: vercorp.org
IPA Server: idm.vercorp.org
BaseDN: dc=vercorp,dc=org

Configured /etc/sssd/sssd.conf
Systemwide CA database updated.
Adding SSH public key from /etc/ssh/ssh_host_ed25519_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ecdsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub
SSSD enabled
Configured /etc/openldap/ldap.conf
Configured /etc/ssh/ssh_config
Configured /etc/ssh/sshd_config
Configuring vercorp.org as NIS domain.
Client configuration complete.
The ipa-client-install command was successful

==============================================================================
Setup complete

Next steps:
  1. You must make sure these network ports are open:
    TCP Ports:
      * 80, 443: HTTP/HTTPS
      * 389, 636: LDAP/LDAPS
      * 88, 464: kerberos
      * 53: bind
    UDP Ports:
      * 88, 464: kerberos
      * 53: bind

  2. You can now obtain a kerberos ticket using the command: 'kinit admin'
     This ticket will allow you to use the IPA tools (e.g., ipa user-add)
     and the web user interface.
  3. Kerberos requires time synchronization between clients
     and servers for correct operation. You should consider enabling chronyd.

Be sure to back up the CA certificates stored in /root/cacert.p12
These files are required to create replicas. The password for these
files is the Directory Manager password
The ipa-server-install command was successful
[root@idm ~]#

Seems that all went well and now we have our FreeIPA/IDM installed.

Lets check several things.

[root@idm ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful

[root@idm ~]# systemctl list-unit-files | grep ipa | grep service
ipa-ccache-sweep.service                   disabled
ipa-custodia.service                       disabled
ipa-dnskeysyncd.service                    disabled
ipa-healthcheck.service                    disabled
ipa-ods-exporter.service                   disabled
ipa-otpd@.service                          static
ipa.service                                enabled

Seems to be installed and working.

What about the /etc/sssd/sssd.conf config.

[root@idm ~]# cat /etc/sssd/sssd.conf
[domain/vercorp.org]

id_provider = ipa
ipa_server_mode = True
ipa_server = idm.vercorp.org
ipa_domain = vercorp.org
ipa_hostname = idm.vercorp.org
auth_provider = ipa
chpass_provider = ipa
access_provider = ipa
cache_credentials = True
ldap_tls_cacert = /etc/ipa/ca.crt
krb5_store_password_if_offline = True
[sssd]
services = nss, pam, ifp, ssh, sudo

domains = vercorp.org
[nss]
homedir_substring = /home
memcache_timeout = 600

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]
allowed_uids = ipaapi, root

[session_recording]

There is also /etc/ssh/ssh_config.d/04-ipa.conf file …

[root@idm ~]# cat /etc/ssh/ssh_config.d/04-ipa.conf
# IPA-related configuration changes to ssh_config
#
PubkeyAuthentication yes
GlobalKnownHostsFile /var/lib/sss/pubconf/known_hosts
#VerifyHostKeyDNS yes

# assumes that if a user does not have shell (/sbin/nologin),
# this will return nonzero exit code and proxy command will be ignored
Match exec true
  ProxyCommand /usr/bin/sss_ssh_knownhostsproxy -p %p %h

For some reason we need to manually enable and start the Apache HTTP server.

[root@idm ~]# systemctl enable --now httpd

You can now access the FreeIPA/IDM at your browser on the http://idm.vercorp.org address. If you use the http://10.0.0.40 IP then you will be redirected to the http://idm.vercorp.org address so make sure you have added that host to your local /etc/hosts file.

FreeIPA-login-page

FreeIPA-login

On the second screen you will see that I have already created the vermaden user.

FreeBSD Client

We will get straight into the point. After having FreeBSD 13.1-RELEASE installed in the most ‘default’ way in the bsdinstall(8) installer – the Auto ZFS road – we will now fetch the up to date FreeBSD Ports tree with portsnap(8) tool.

This is the main /etc/rc.conf config file.

# cat /etc/rc.conf
hostname="fbsd"
ifconfig_vtnet0="inet 10.0.0.42 netmask 255.255.255.0"
defaultrouter="10.0.0.1"
sshd_enable="YES"
moused_enable="YES"
dumpdev="AUTO"
zfs_enable="YES"
syslogd_flags="-ss"
sssd_enable="YES"

FreeBSD Packages

We need to have FreeBSD Ports tree.

# portsnap auto

Lets check what is the current default SAMBA version on FreeBSD.

# grep -i samba /usr/ports/Mk/*default*
        PYTHON2 PYTHON3 RUBY RUST SAMBA SSL TCLTK VARNISH
SAMBA_DEFAULT?=         4.12

To omit a lot of pointless compilation I will first install all precompiled packages – that would also install needed dependencies. Then we will rebuild just the several needed packages.

# pkg install krb5 sudo cyrus-sasl cyrus-sasl-gssapi pam_mkhomedir openldap26-client samba412

Settings in the /etc/make.conf file.

# cat /etc/make.conf
# USE /usr/ports/obj PLACE
WRKDIRPREFIX=${PORTSDIR}/obj

OPTIONS_UNSET=       DOCS EXAMPLES DEBUG X11
OPTIONS_UNSET+=      DBUS GSSAPI_BASE GSSAPI_HEIMDAL
OPTIONS_SET+=        GSSAPI_MIT
WITH_GSSAPI=         yes
WANT_OPENLDAP_SASL=  yes

Lets configure the needed ports.

# \
for I in /usr/ports/security/sudo \
         /usr/ports/security/sssd \
         /usr/ports/security/krb5 \
         /usr/ports/security/cyrus-sasl2-gssapi \
         /usr/ports/security/pam_mkhomedir \
         /usr/ports/net/openldap26-client
do
  make -C ${I} rmconfig
done

# \
for I in /usr/ports/security/sudo \
         /usr/ports/security/sssd \
         /usr/ports/security/krb5 \
         /usr/ports/security/cyrus-sasl2-gssapi \
         /usr/ports/security/pam_mkhomedir \
         /usr/ports/net/openldap26-client
do
  make -C ${I} config-recursive
done

The curses(3X) selection will look more or less like these below.

=> security/sudo | OPTIONS___________________________________________________________________________
[x] AUDIT              Enable BSM audit support
[ ] DISABLE_AUTH       Do not require authentication by default
[ ] DISABLE_ROOT_SUDO  Do not allow root to run sudo
[ ] DOCS               Build and/or install documentation
[ ] EXAMPLES           Build and/or install examples
[ ] INSULTS            Enable insults on failures
[x] LDAP               LDAP protocol support
[ ] NLS                Native Language Support
[ ] NOARGS_SHELL       Run a shell if no arguments are given
[ ] OPIE               Enable one-time passwords (no PAM support)
[ ] PAM                Pluggable authentication module support
[ ] PYTHON             Enable python plugin support
[x] SSSD               Enable SSSD backend support.
=> security/sudo | Kerberos 5 Authentication (no PAM support)
( ) GSSAPI_BASE        GSSAPI support via base system (needs Kerberos)
( ) GSSAPI_HEIMDAL     GSSAPI support via security/heimdal
(*) GSSAPI_MIT         GSSAPI support via security/krb5


=> security/sssd | OPTIONS___________________________________________________________________________
[ ] DOCS  Build and/or install documentation
[x] SMB   Install IPA and AD providers (requires Samba4)


=> security/cyrus-sasl2-gssapi | OPTIONS_____________________________________________________________
( ) GSSAPI_BASE     GSSAPI support via base system (needs Kerberos)
( ) GSSAPI_HEIMDAL  GSSAPI support via security/heimdal
(*) GSSAPI_MIT      GSSAPI support via security/krb5


=> net/openldap26-client | OPTIONS___________________________________________________________________
[ ] DEBUG   Build with debugging support
[ ] DOCS    Build and/or install documentation
[ ] FETCH   Enable fetch(3) support
[x] GSSAPI  With GSSAPI support


=> security/krb5 | OPTIONS___________________________________________________________________________
[ ] DNS_FOR_REALM  Enable DNS lookups for Kerberos realm names
[ ] EXAMPLES       Build and/or install examples
[x] KRB5_HTML      Install krb5 HTML documentation
[x] KRB5_PDF       Install krb5 PDF documentation
[x] LDAP           LDAP protocol support
[ ] LMDB           OpenLDAP Lightning Memory-Mapped Database support
[ ] NLS            Native Language Support
=> security/krb5 | Command Line Editing for kadmin and ktutil
(*) READLINE       Command line editing via libreadline
( ) LIBEDIT        Command line editing via libedit

We can now check what options have been saved.

# cat /var/db/ports/security_sudo-sssd/options
_OPTIONS_READ=sudo-sssd-1.9.11p3
_FILE_COMPLETE_OPTIONS_LIST=AUDIT DISABLE_AUTH DISABLE_ROOT_SUDO DOCS EXAMPLES INSULTS LDAP NLS NOARGS_SHELL OPIE PAM PYTHON SSSD GSSAPI_BASE GSSAPI_HEIMDAL GSSAPI_MIT
OPTIONS_FILE_SET+=AUDIT
OPTIONS_FILE_SET+=GSSAPI_MIT
OPTIONS_FILE_SET+=LDAP
OPTIONS_FILE_SET+=NLS
OPTIONS_FILE_SET+=SSSD
OPTIONS_FILE_UNSET+=DISABLE_AUTH
OPTIONS_FILE_UNSET+=DISABLE_ROOT_SUDO
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_UNSET+=EXAMPLES
OPTIONS_FILE_UNSET+=GSSAPI_BASE
OPTIONS_FILE_UNSET+=GSSAPI_HEIMDAL
OPTIONS_FILE_UNSET+=INSULTS
OPTIONS_FILE_UNSET+=NOARGS_SHELL
OPTIONS_FILE_UNSET+=OPIE
OPTIONS_FILE_UNSET+=PAM
OPTIONS_FILE_UNSET+=PYTHON

# cat /var/db/ports/security_sssd/options
_OPTIONS_READ=sssd-1.16.5_6
_FILE_COMPLETE_OPTIONS_LIST=DOCS SMB
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_SET+=SMB

# cat /var/db/ports/security_cyrus-sasl2/options
_OPTIONS_READ=cyrus-sasl-2.1.28
_FILE_COMPLETE_OPTIONS_LIST=ALWAYSTRUE AUTHDAEMOND DOCS KEEP_DB_OPEN  OBSOLETE_CRAM_ATTR OBSOLETE_DIGEST_ATTR  SASLDB_IN_VAR BDB1 BDB GDBM LMDB ANONYMOUS CRAM DIGEST LOGIN NTLM OTP PLAIN SCRAM
OPTIONS_FILE_SET+=ANONYMOUS
OPTIONS_FILE_SET+=AUTHDAEMOND
OPTIONS_FILE_SET+=BDB1
OPTIONS_FILE_SET+=CRAM
OPTIONS_FILE_SET+=DIGEST
OPTIONS_FILE_SET+=LOGIN
OPTIONS_FILE_SET+=NTLM
OPTIONS_FILE_SET+=OBSOLETE_CRAM_ATTR
OPTIONS_FILE_SET+=OBSOLETE_DIGEST_ATTR
OPTIONS_FILE_SET+=OTP
OPTIONS_FILE_SET+=PLAIN
OPTIONS_FILE_SET+=SCRAM
OPTIONS_FILE_UNSET+=ALWAYSTRUE
OPTIONS_FILE_UNSET+=BDB
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_UNSET+=GDBM
OPTIONS_FILE_UNSET+=KEEP_DB_OPEN
OPTIONS_FILE_UNSET+=LMDB
OPTIONS_FILE_UNSET+=SASLDB_IN_VAR

# cat /var/db/ports/net_openldap26-client/options
_OPTIONS_READ=openldap26-client-2.6.3
_FILE_COMPLETE_OPTIONS_LIST=DEBUG DOCS FETCH GSSAPI
OPTIONS_FILE_SET+=GSSAPI
OPTIONS_FILE_UNSET+=DEBUG
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_UNSET+=FETCH

# cat /var/db/ports/security_krb5/options
# This file is auto-generated by 'make config'.
# Options for krb5-1.20
_OPTIONS_READ=krb5-1.20
_FILE_COMPLETE_OPTIONS_LIST=DNS_FOR_REALM EXAMPLES KRB5_HTML KRB5_PDF LDAP LMDB NLS READLINE LIBEDIT
OPTIONS_FILE_UNSET+=DNS_FOR_REALM
OPTIONS_FILE_UNSET+=EXAMPLES
OPTIONS_FILE_SET+=KRB5_HTML
OPTIONS_FILE_SET+=KRB5_PDF
OPTIONS_FILE_SET+=LDAP
OPTIONS_FILE_UNSET+=LMDB
OPTIONS_FILE_SET+=NLS
OPTIONS_FILE_SET+=READLINE
OPTIONS_FILE_UNSET+=LIBEDIT

… and build them.

# \
for I in /usr/ports/security/sudo \
         /usr/ports/security/sssd \
         /usr/ports/security/krb5
         /usr/ports/security/cyrus-sasl2-gssapi \
         /usr/ports/security/pam_mkhomedir \
         /usr/ports/net/openldap26-client \
do
  make -C ${I} build
done

… unsinstall them and create their packages.

# \
for I in /usr/ports/security/sudo \
         /usr/ports/security/sssd \
         /usr/ports/security/krb5
         /usr/ports/security/cyrus-sasl2-gssapi \
         /usr/ports/security/pam_mkhomedir \
         /usr/ports/net/openldap26-client \
do
  make -C ${I} deinstall package
done

After some time we have our built packages.

# find /usr/ports/obj/ -name \*.pkg
/usr/ports/obj/usr/ports/lang/perl5.32/work/perl-5.32.1/symbian/ext/Moped/Msg/Msg.pkg
/usr/ports/obj/usr/ports/security/pam_mkhomedir/work/pkg/pam_mkhomedir-0.2.pkg
/usr/ports/obj/usr/ports/security/cyrus-sasl2-gssapi/work/pkg/cyrus-sasl-gssapi-2.1.28.pkg
/usr/ports/obj/usr/ports/security/krb5/work/pkg/krb5-1.20.pkg
/usr/ports/obj/usr/ports/security/sudo/work/pkg/sudo-1.9.11p3.pkg
/usr/ports/obj/usr/ports/security/sssd/work-default/pkg/sssd-1.16.5_6.pkg
/usr/ports/obj/usr/ports/net/openldap26-client/work/pkg/openldap26-client-2.6.3.pkg

In the future I need to add some short guide to build them regularly with Synth or Poudriere πŸ™‚

Now we need to install these packages in quite nontypical way. One by one – in specified order. It will not be needed it we would have a separate additional pkg(8) repository with Poudriere build packages.

Our packages:

# ls -1 *.pkg
cyrus-sasl-gssapi-2.1.28.pkg
krb5-1.20.pkg
openldap26-client-2.6.3.pkg
pam_mkhomedir-0.2.pkg
sssd-smb4-1.16.5_6.pkg
sudo-sssd-1.9.11p3.pkg

… and their install process.

# pkg install -y c-ares

# pkg add sssd-smb4-1.16.5_6.pkg

# pkg install -y cyrus-sasl-gssapi-2.1.28.pkg
(...)
New packages to be INSTALLED:
        cyrus-sasl: 2.1.28
        cyrus-sasl-gssapi: 2.1.28
        krb5: 1.20
(...)

# pkg delete -f -y krb5
(...)
Installed packages to be REMOVED:
        krb5: 1.20
(...)

# pkg add krb5-1.20.pkg

# pkg install -y sssd

# pkg add sudo-sssd-1.9.11p3.pkg

# pkg delete -f -y sssd
(...)
Installed packages to be REMOVED:
        sssd: 1.16.5_6
(...)

# pkg install ding-libs ldb21 nspr nss pcre samba412

# pkg add sssd-smb4-1.16.5_6.pkg

FreeBSD Setup

Create needed dirs.

# mkdir -p \
    /usr/local/etc/ipa \
    /var/log/sssd \
    /var/run/sss/private \
    /var/db/sss

Set system hostname.

# hostname fbsd.vercorp.org

# hostname
fbsd.vercorp.org

Fetch the FreeIPA/IDM certificate.

# fetch -o /usr/local/etc/ipa/ca.crt http://10.0.0.40/ipa/config/ca.crt

FreeIPA/IDM Setup Part

We need to execute several instructions on the FreeIPA/IDM to connect FreeBSD client to it.

Its adding the A and PTR records in DNS for the 10.0.0.42 address and adding the fbsd.vercorp.org host.

We also need to generate the key for our fbsd.vercorp.org system.

[root@idm ~]# kinit admin
Password for admin@VERCORP.ORG:

[root@idm ~]# ipa dnsrecord-add vercorp.org fbsd --a-rec=10.0.0.42 --a-create-reverse
  Record name: fbsd
  A record: 10.0.0.42

[root@idm ~]# ipa host-add fbsd.vercorp.org
-----------------------------
Added host "fbsd.vercorp.org"
-----------------------------
  Host name: fbsd.vercorp.org
  Principal name: host/fbsd.vercorp.org@VERCORP.ORG
  Principal alias: host/fbsd.vercorp.org@VERCORP.ORG
  Password: False
  Keytab: False
  Managed by: fbsd.vercorp.org

[root@idm ~]# ipa-getkeytab -s idm.vercorp.org -p host/fbsd.vercorp.org@VERCORP.ORG -k /root/fbsd.vercorp.org.keytab
Keytab successfully retrieved and stored in: /root/fbsd.vercorp.org.keytab

Now we need to get ‘our’ key from FreeIPA/IDM server … along with proper /etc/hosts file.

# scp root@10.0.0.40:/root/fbsd.vercorp.org.keytab /usr/local/etc/ipa/krb5.keytab

# cat << EOF > /etc/hosts
::1              localhost localhost.my.domain
127.0.0.1        localhost localhost.my.domain
10.0.0.40        idm.vercorp.org  idm
10.0.0.42        fbsd.vercorp.org fbsd
EOF

Now the /usr/local/etc/openldap/ldap.conf file.

# cat << EOF > /usr/local/etc/openldap/ldap.conf
BASE        dc=org,dc=vercorp
URI         ldap://idm.vercorp.org/
#SIZELIMIT  12
#TIMELIMIT  15
#DEREF      never
SASL_MECH   GSSAPI
SASL_REALM  VERCORP.ORG
ssl         start_tls
TLS_CACERT  /usr/local/etc/ipa/ca.crt
EOF

… and /etc/krb5.conf file.

# cat << EOF > /etc/krb5.conf
[libdefaults]
  default_realm = VERCORP.ORG
  default_keytab_name = FILE:/usr/local/etc/ipa/krb5.keytab
  default_tkt_enctypes = aes256-cts des-cbc-crc aes128-cts arcfour-hmac
  default_tgs_enctypes = aes256-cts des-cbc-crc aes128-cts arcfour-hmac
  dns_lookup_realm = false
  dns_lookup_kdc = false
  rdns = false
  ticket_lifetime = 24h
  forwardable = yes

[realms]
  VERCORP.ORG = {
    kdc = idm.vercorp.org:88
    master_kdc = idm.vercorp.org:88
    admin_server = idm.vercorp.org:749
    default_domain = vercorp.org
    pkinit_anchors = FILE:/usr/local/etc/ipa/ca.crt
}

[domain_realm]
  .vercorp.org = VERCORP.ORG
  vercorp.org = VERCORP.ORG

[logging]
  kdc = FILE:/var/log/krb5/krb5kdc.log
  admin_server = FILE:/var/log/krb5/kadmin.log
  kadmin_local = FILE:/var/log/krb5/kadmin_local.log
  default = FILE:/var/log/krb5/krb5lib.log
EOF

… and /usr/local/etc/sssd/sssd.conf file.

# cat << EOF > /usr/local/etc/sssd/sssd.conf

[domain/vercorp.org]
# debug_level = 9
cache_credentials = True
krb5_store_password_if_offline = True
krb5_realm = VERCORP.ORG
ipa_domain = vercorp.org
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = fbsd.vercorp.org
chpass_provider = ipa
ipa_server = _srv_, idm.vercorp.org
ldap_tls_cacert = /usr/local/etc/ipa/ca.crt
krb5_keytab = /usr/local/etc/ipa/krb5.keytab

[sssd]
services = nss, pam, ssh, sudo
config_file_version = 2
domains = vercorp.org

[nss]
filter_users = root,toor
homedir_substring = /usr/home/%u

[pam]

[sudo]
# debug_level = 0x3ff0

[ssh]
EOF

# chmod 600 /usr/local/etc/sssd/sssd.conf

FreeBSD have user account under /usr/home to make sure /home also points there.

# ln -s /usr/home /home

The automatic startup of sssd(8) daemon (not to confuse with sshd(8)) needs to be also configured.

# sysrc sssd_enable=YES
# service sssd start

We also need to configure /etc/nsswitch.conf file.

# cp /etc/nsswitch.conf /etc/nsswitch.conf.BCK

# diff -u /etc/nsswitch.conf.BCK /etc/nsswitch.conf
--- /etc/nsswitch.conf.BCK      2022-10-24 20:10:09.163251000 +0200
+++ /etc/nsswitch.conf  2022-10-24 20:10:57.207406000 +0200
@@ -2,15 +2,17 @@
 # nsswitch.conf(5) - name service switch configuration file
 # $FreeBSD$
 #
-group: compat
+group: files sss
 group_compat: nis
 hosts: files dns
-netgroup: compat
+# netgroup: compat
 networks: files
-passwd: compat
+passwd: files sss
 passwd_compat: nis
 shells: files
 services: compat
 services_compat: nis
 protocols: files
 rpc: files
+sudoers: sss files
+netgroup: files

The final /etc/nsswitch.conf file looks as follows.

# cat /etc/nsswitch.conf
#
# nsswitch.conf(5) - name service switch configuration file
# $FreeBSD$
#
group: files sss
group_compat: nis
hosts: files dns
# netgroup: compat
networks: files
passwd: files sss
passwd_compat: nis
shells: files
services: compat
services_compat: nis
protocols: files
rpc: files
sudoers: sss files
netgroup: files

Now the /etc/pam.d/system file.

# cp /etc/pam.d/system /root/etc---pam.d---system.BCK

# diff -u  /root/etc---pam.d---system.BCK /etc/pam.d/system
--- /root/etc---pam.d---system.BCK      2022-10-24 20:13:05.546657000 +0200
+++ /etc/pam.d/system   2022-10-24 20:16:36.722666000 +0200
@@ -7,19 +7,23 @@
 # auth
 auth           sufficient      pam_opie.so             no_warn no_fake_prompts
 auth           requisite       pam_opieaccess.so       no_warn allow_local
-#auth          sufficient      pam_krb5.so             no_warn try_first_pass
+auth           sufficient      pam_krb5.so             no_warn try_first_pass
 #auth          sufficient      pam_ssh.so              no_warn try_first_pass
+auth           sufficient      /usr/local/lib/pam_sss.so       use_first_pass
 auth           required        pam_unix.so             no_warn try_first_pass nullok

 # account
 #account       required        pam_krb5.so
 account                required        pam_login_access.so
 account                required        pam_unix.so
+account                required        /usr/local/lib/pam_sss.so       ignore_unknown_user ignore_authinfo_unavail

 # session
 #session       optional        pam_ssh.so              want_agent
 session                required        pam_lastlog.so          no_fail
+session                required        /usr/local/lib/pam_mkhomedir.so mode=0700

 # password
 #password      sufficient      pam_krb5.so             no_warn try_first_pass
+password       sufficient      /usr/local/lib/pam_sss.so       use_authtok
 password       required        pam_unix.so             no_warn try_first_pass

The final /etc/pam.d/system file looks as follows.

# cat /etc/pam.d/system
#
# $FreeBSD$
#
# System-wide defaults
#

# auth
auth            sufficient      pam_opie.so             no_warn no_fake_prompts
auth            requisite       pam_opieaccess.so       no_warn allow_local
auth            sufficient      pam_krb5.so             no_warn try_first_pass
#auth           sufficient      pam_ssh.so              no_warn try_first_pass
auth            sufficient      /usr/local/lib/pam_sss.so       use_first_pass
auth            required        pam_unix.so             no_warn try_first_pass nullok

# account
#account        required        pam_krb5.so
account         required        pam_login_access.so
account         required        pam_unix.so
account         required        /usr/local/lib/pam_sss.so       ignore_unknown_user ignore_authinfo_unavail

# session
#session        optional        pam_ssh.so              want_agent
session         required        pam_lastlog.so          no_fail
session         required        /usr/local/lib/pam_mkhomedir.so mode=0700

# password
#password       sufficient      pam_krb5.so             no_warn try_first_pass
password        sufficient      /usr/local/lib/pam_sss.so       use_authtok
password        required        pam_unix.so             no_warn try_first_pass

Now its time for /etc/pam.d/sshd file.

# cp /etc/pam.d/sshd /root/etc---pam.d---sshd.BCK

# diff -u /root/etc---pam.d---sshd.BCK /etc/pam.d/sshd
--- /root/etc---pam.d---sshd.BCK        2022-10-24 20:17:34.063630000 +0200
+++ /etc/pam.d/sshd     2022-10-24 20:19:16.165810000 +0200
@@ -7,8 +7,9 @@
 # auth
 auth           sufficient      pam_opie.so             no_warn no_fake_prompts
 auth           requisite       pam_opieaccess.so       no_warn allow_local
-#auth          sufficient      pam_krb5.so             no_warn try_first_pass
+auth           sufficient      pam_krb5.so             no_warn try_first_pass
 #auth          sufficient      pam_ssh.so              no_warn try_first_pass
+auth           sufficient      /usr/local/lib/pam_sss.so       use_first_pass
 auth           required        pam_unix.so             no_warn try_first_pass

 # account
@@ -16,11 +17,14 @@
 #account       required        pam_krb5.so
 account                required        pam_login_access.so
 account                required        pam_unix.so
+account                required        /usr/local/lib/pam_sss.so       ignore_unknown_user ignore_authinfo_unavail

 # session
 #session       optional        pam_ssh.so              want_agent
 session                required        pam_permit.so
+session                required        /usr/local/lib/pam_mkhomedir.so mode=0700

 # password
 #password      sufficient      pam_krb5.so             no_warn try_first_pass
+password       sufficient      /usr/local/lib/pam_sss.so       use_authtok
 password       required        pam_unix.so             no_warn try_first_pass

Final /etc/pam.d/sshd file below.

# cat /etc/pam.d/sshd
#
# $FreeBSD$
#
# PAM configuration for the "sshd" service
#

# auth
auth            sufficient      pam_opie.so             no_warn no_fake_prompts
auth            requisite       pam_opieaccess.so       no_warn allow_local
auth            sufficient      pam_krb5.so             no_warn try_first_pass
#auth           sufficient      pam_ssh.so              no_warn try_first_pass
auth            sufficient      /usr/local/lib/pam_sss.so       use_first_pass
auth            required        pam_unix.so             no_warn try_first_pass

# account
account         required        pam_nologin.so
#account        required        pam_krb5.so
account         required        pam_login_access.so
account         required        pam_unix.so
account         required        /usr/local/lib/pam_sss.so       ignore_unknown_user ignore_authinfo_unavail

# session
#session        optional        pam_ssh.so              want_agent
session         required        pam_permit.so
session         required        /usr/local/lib/pam_mkhomedir.so mode=0700

# password
#password       sufficient      pam_krb5.so             no_warn try_first_pass
password        sufficient      /usr/local/lib/pam_sss.so       use_authtok
password        required        pam_unix.so             no_warn try_first_pass

Small modification in the /etc/ssh/ssh_config and /etc/ssh/sshd_config files.

# cat << EOF >> /etc/ssh/ssh_config
GSSAPIAuthentication yes
EOF

# cat << EOF >> /etc/ssh/sshd_config
GSSAPIAuthentication yes
UsePAM yes
EOF

Finish Setup with Web Browser in FreeIPA/IDM Page

Visit the https://idm.vercorp.org/ipa/ui/#/e/hbacrule/details/freebsd page.

Next create the HBAC Rule named freebsd as showed below.

idm-1-hbac-rules-menu

idm-2-hbac-rules-freebsd

idm-3-hbac-rules-freebsd-details

… and the Sudo Rule named freebsd name.

idm-4-sudo-rules-menu

idm-5-sudo-rules-freebsd

idm-6-sudo-rules-freebsd-details

FreeBSD FreeIPA Login Test

After all these time consuming and pointless instructions we can now finally try to login to our FreeBSD client.

% ssh -l vermaden 10.0.0.42
(vermaden@10.0.0.42) Password:
Last login: Mon Oct 24 21:06:36 2022 from 10.0.0.3
FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

To change this login announcement, see motd(5).
You can upload the dmesg of your system to help developers get an overview of commonly
used hardware and peripherals for FreeBSD. Use the curl package to upload it like this:
curl -v -d "nickname=$USER" -d "description=FreeBSD/$(uname -m) on \
$(kenv smbios.system.maker) $(kenv smbios.system.product)" -d "do=addd" \
--data-urlencode 'dmesg@/var/run/dmesg.boot' http://dmesgd.nycbug.org/index.cgi

vermaden@fbsd:~ $ :> ~/.hushlogin

vermaden@fbsd:~ $ id
uid=1408200003(vermaden) gid=1408200000(admins) groups=1408200000(admins)

vermaden@fbsd:~ $ pwd
/home/vermaden

vermaden@fbsd:~ $ grep vermaden /etc/passwd /etc/group
vermaden@fbsd:~ $

vermaden@fbsd:~ $ getent passwd vermaden
vermaden:*:1408200003:1408200000:vermaden vermaden:/home/vermaden:/bin/sh

vermaden@fbsd:~ $ sudo su -
Password for vermaden@VERCORP.ORG:

root@fbsd:~ # logout

vermaden@fbsd:~ $ sudo -i

root@fbsd:~ #

Strange … seems to work properly πŸ™‚

FreeBSD Jail as FreeIPA/IDM Client

As ‘full’ FreeBSD system is able to connect to the FreeIPA/IDM server we will not configure FreeBSD Jail to do the same.

The FreeIPA/IDM FreeBSD client fbsdjail.vercorp.orgwill get the 10.0.0.43 IP.

Basic FreeBSD Jail Preparations

Lets setup the Jail for a start.

# mkdir -p /jail/fbsdjail /jail/BASE

# cd /jail/fbsdjail

# fetch -o /jail/BASE/13.1-RELEASE-base.txz https://download.freebsd.org/ftp/releases/amd64/13.1-RELEASE/base.txz

# tar --unlink -xvf ../BASE/13.1-RELEASE-base.txz

# cat << EOF > /etc/jail.conf
# GLOBAL
  exec.start = "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.clean;
  exec.consolelog = "/var/log/jail_${name}_console.log";
  mount.devfs;
  host.hostname = ${name};
  path = /jail/${name};

# JAILS
  fbsdjail {
    ip4.addr = 10.0.0.43;
    host.hostname = fbsdjail.vercorp.org;
    interface = wlan0;
    allow.raw_sockets;
    allow.sysvipc;
  }
EOF

# cat /etc/resolv.conf | tee /jail/fbsdjail/etc/resolv.conf
nameserver 10.0.0.1

# echo 10.0.0.43 fbsdjail.vercorp.org fbsdjail | tee -a /etc/hosts | tee -a /jail/fbsdjail/etc/hosts

# cat /etc/hosts /jail/fbsdjail/etc/hosts
10.0.0.43 fbsdjail.vercorp.org fbsdjail
10.0.0.43 fbsdjail.vercorp.org fbsdjail

# cat << EOF > /jail/fbsdjail/etc/rc.conf
# DAEMONS | yes
  syslogd_flags="-ss"
  sshd_enable=YES

# OTHER
  clear_tmp_enable=YES
  clear_tmp_X=YES
  dumpdev=NO
  update_motd=NO
EOF

# sed -i '' s/quarterly/latest/g /jail/fbsdjail/etc/pkg/FreeBSD.conf

# grep latest /jail/fbsdjail/etc/pkg/FreeBSD.conf
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",

Now we can start our FreeBSD Jail.

# service jail onestart fbsdjail
Starting jails: fbsdjail.

# jls
   JID  IP Address      Hostname                      Path
     1  10.0.0.43       fbsdjail.vercorp.org          /jail/fbsdjail

# jls -v
   JID  Hostname                      Path
        Name                          State
        CPUSetID
        IP Address(es)
     1  fbsdjail.vercorp.org          /jail/fbsdjail
        fbsdjail                      ACTIVE
        3
        10.0.0.43

# jexec fbsdjail

root@fbsdjail:/ #

Our FreeBSD Jail works. Lets move to next steps.

Configure FreeBSD Jail to Connect to FreeIPA/IDM Server

I could not repaste all of the instructions above – the ones that we used for a ‘full’ FreeBSD system – but the same applies to a FreeBSD Jail. πŸ™‚

This means that earlier Basic FreeBSD Jail Preparations section covers all that is needed in case of ‘full’ FreeBSD versus FreeBSD Jail when it comes to the FreeIPA/IDM connection.

Linux FreeIPA/IDM Client

This article is not about Linux client – which is pretty straight-forward to connect to the FreeIPA/IDM server – but for the completness of the topic – here are the instructions I used to attach Alma Linux to the FreeIPA/IDM server.

Linux rhlike.vercorp.org system.

      IP: 10.0.0.41/24
      GW: 10.0.0.1
hostname: rhlike.vercorp.org

First – install the @idm:client and sssd packages.

client # yum -y install @idm:client sssd

FreeIPA/IDM Setup Part

Now – as earlier with FreeBSD – the FreeIPA/IDM part comes to play.

[root@idm ~]# kinit admin
Password for admin@VERCORP.ORG:

[root@idm ~]# klist
Ticket cache: KCM:0
Default principal: admin@VERCORP.ORG

Valid starting       Expires              Service principal
10/19/2022 13:33:52  10/20/2022 13:11:28  krbtgt/VERCORP.ORG@VERCORP.ORG

[root@idm ~]# ipa user-find
---------------
2 users matched
---------------
  User login: admin
  Last name: Administrator
  Home directory: /home/admin
  Login shell: /bin/bash
  Principal alias: admin@VERCORP.ORG, root@VERCORP.ORG
  UID: 1896600000
  GID: 1896600000
  Account disabled: False

  User login: vermaden
  First name: vermaden
  Last name: vermaden
  Home directory: /home/vermaden
  Login shell: /bin/sh
  Principal name: vermaden@VERCORP.ORG
  Principal alias: vermaden@VERCORP.ORG
  Email address: vermaden@vercorp.org
  UID: 1896600003
  GID: 1000
  Account disabled: False
----------------------------
Number of entries returned 2
----------------------------

[root@idm ~]# id vermaden
uid=1408200003(vermaden) gid=1408200000(admins) groups=1408200000(admins)

[root@idm ~]# ipa dnsrecord-add vercorp.org rhlike --a-rec 10.0.0.41
  Record name: rhlike
  A record: 10.0.0.41

We are done on the FreeIPA/IDM side.

Linux FreeIPA/IDM Client Setup

We will now continue our work on the Linux client.

client # echo "10.0.0.40   idm.vercorp.org"    >> /etc/hosts

client # echo "10.0.0.41   rhlike.vercorp.org" >> /etc/hosts

client # hostnamectl set-hostname rhlike.vercorp.org

client # cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.40   idm.vercorp.org
10.0.0.41   rhlike.vercorp.org

client # ipa-client-install --uninstall

client # ipa-client-install \
           --hostname=rhlike.vercorp.org \
           --mkhomedir \
           --server=idm.vercorp.org \
           --domain vercorp.org \
           --realm VERCORP.ORG

This program will set up IPA client.
Version 4.9.8

Autodiscovery of servers for failover cannot work with this configuration.
If you proceed with the installation, services will be configured to always access the discovered server for all operations and will not fail over to other servers in case of failure.
Proceed with fixed values and no DNS discovery? [no]: yes
Do you want to configure chrony with NTP server or pool address? [no]: no
Client hostname: rhlike.vercorp.org
Realm: VERCORP.ORG
DNS Domain: vercorp.org
IPA Server: idm.vercorp.org
BaseDN: dc=vercorp,dc=org

Continue to configure the system with these values? [no]: yes
Synchronizing time
No SRV records of NTP servers found and no NTP server or pool address was provided.
Using default chrony configuration.
Attempting to sync time with chronyc.
Time synchronization was successful.
User authorized to enroll computers: admin
Password for admin@VERCORP.ORG:
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=VERCORP.ORG
    Issuer:      CN=Certificate Authority,O=VERCORP.ORG
    Valid From:  2022-10-18 14:52:50
    Valid Until: 2042-10-18 14:52:50

Enrolled in IPA realm VERCORP.ORG
Created /etc/ipa/default.conf
Configured /etc/sssd/sssd.conf
Configured /etc/krb5.conf for IPA realm VERCORP.ORG
Systemwide CA database updated.
Hostname (rhlike.vercorp.org) does not have A/AAAA record.
Failed to update DNS records.
Missing A/AAAA record(s) for host rhlike.vercorp.org: 10.0.0.41.
Missing reverse record(s) for address(es): 10.0.0.41.
Adding SSH public key from /etc/ssh/ssh_host_ed25519_key.pub
Adding SSH public key from /etc/ssh/ssh_host_ecdsa_key.pub
Adding SSH public key from /etc/ssh/ssh_host_rsa_key.pub
Could not update DNS SSHFP records.
SSSD enabled
Configured /etc/openldap/ldap.conf
Configured /etc/ssh/ssh_config
Configured /etc/ssh/sshd_config
Configuring vercorp.org as NIS domain.
Client configuration complete.
The ipa-client-install command was successful

client # reboot

Now we will test how it goes with login against the FreeIPA/IDM server.

laptop % ssh -l vermaden 10.0.0.41
(vermaden@10.0.0.41) Password:
(vermaden@10.0.0.41) Password expired. Change your password now.
Current Password:
(vermaden@10.0.0.41) New password:
(vermaden@10.0.0.41) Retype new password:
Last failed login: Wed Oct 19 00:47:57 CEST 2022 from 10.0.0.3 on ssh:notty
There was 1 failed login attempt since the last successful login.
/usr/bin/id: cannot find name for group ID 1000

[vermaden@rhlike ~]$ w
 00:48:16 up 29 min,  2 users,  load average: 0.22, 0.13, 0.16
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.0.0.3         00:40   58.00s  0.03s  0.03s -bash
vermaden pts/1    10.0.0.3         00:48    0.00s  0.02s  0.01s w

[vermaden@rhlike ~]$ sudo su -

[root@rhlike ~]# getent passwd admin
admin:*:1896600000:1896600000:Administrator:/home/admin:/bin/bash

[root@rhlike ~]# getent passwd vermaden
vermaden:*:1896600003:1000:vermaden vermaden:/home/vermaden:/bin/sh

Seems to work at least OK πŸ™‚

I do not have anything more to add to this guide.

If you have – then please let me know in comments πŸ™‚

Regards.

EOF

Advertisement

NFS Server Inside FreeBSD VNET Jail

FreeBSD Jails is a great piece of container technology pioneered several years before Solaris Zones. Not to mention 15 years before Docker was born. Today they still work great and offer some new features like entire network stack for each Jail called VNET. Unfortunately they also have downsides. For example anything related to NFS is broken inside FreeBSD Jails (especially when they are VNET based Jails) and the relevant 251347 Bug Report remains unsolved.

There is however a way to run NFS server inside VNET based FreeBSD Jail – we will use userspace NFS server implementation instead of using the FreeBSD base system kernel space NFS server. Its available as net/unfs3 package and this is exactly what we will gonna use for this guide.

unfs3


Same in plain text below.

/ % cd /usr/ports/net/unfs3

/usr/ports/net/unfs3 % cat pkg-descr
UNFS3 is a user-space implementation of the NFSv3 server specification. It
provides a daemon for the MOUNT and NFS protocols, which are used by NFS
clients for accessing files on the server.
Since it runs in user-space, you can use it in a jail.

WWW: https://unfs3.github.io/

/usr/ports/net/unfs3 % pkg info -l unfs3           
unfs3-0.9.22_2:
        /usr/local/man/man7/tags.7.gz
        /usr/local/man/man8/unfsd.8.gz
        /usr/local/sbin/unfsd
        /usr/local/share/licenses/unfs3-0.9.22_2/BSD3CLAUSE
        /usr/local/share/licenses/unfs3-0.9.22_2/LICENSE
        /usr/local/share/licenses/unfs3-0.9.22_2/catalog.mk

Its also pity that VNET feature for FreeBSD Jails is not well documented. Search the FreeBSD Handbook or FreeBSD FAQ for the VNET or VIMAGE keywords. Not a single match. There are only man pages and some stuff left in the /usr/share/examples/jails dir. There is also FreeBSD Mastery: Jails book by Michael W. Lucas but its 3 years old already.

Setup

Below you will find the list of systems we will use in this guide.

10.0.10.250  host
10.0.10.251  nfs_server

The host is a common FreeBSD server installed on a physical or virtual machine. We will also use it as out NFS client and mount the NFS share there. The nfs_server is a FreeBSD Jail with VNET separate network stack enabled. We will run NFS server from this host nfs_server system. Both of them run latest FreeBSD 13.1-RELEASE but I suspect that it should also work the same on older versions.

FreeBSD Host and NFS Client (host)

First we will setup the host machine. Its typical default ZFS FreeBSD install – nothing special about that. To use the VNET enabled Jails we will use jib tool from the /usr/share/examples/jails directory as we will need it to automate epair(4) interfaces management.

root@host:/ # install -o root -g wheel -m 0555 /usr/share/examples/jails/jib /usr/sbin/jib

Our next step would be to fetch and setup the nfs_server FreeBSD Jail. We will not waste time in compilation – we will fetch the base.txz directly from FreeBSD page.

root@host:/ # mkdir -p /jail/BASE
root@host:/jail/BASE # cd /jail/BASE
root@host:/jail/BASE # fetch http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE/base.txz
root@host:/jail/BASE # mv base.txz 13.1-base.txz

Now the nfs_server FreeBSD Jail.

root@host:/ # mkdir -p /jail/nfs_server
root@host:/jail/nfs_server # cd /jail/nfs_server
root@host:/jail/nfs_server # tar -xzf /jail/BASE/13.1-base.txz --unlink

The main FreeBSD /etc/rc.conf configuration file does not hold any special setting – pretty usual stuff.

root@host:/ # cat /etc/rc.conf
# NETWORK
  hostname="host"
  ifconfig_em0="inet 10.0.10.250/24 up"
  defaultrouter="10.0.10.1"
  gateway_enable="YES"

# DAEMONS
  dumpdev="AUTO"
  sshd_enable="YES"
  zfs_enable="YES"
  sendmail_enable="NO"
  sendmail_submit_enable="NO"
  sendmail_outbound_enable="NO"
  sendmail_msp_queue_enable="NO"
  update_motd="NO"

# JAILS
  jail_enable="YES"
  jail_parallel_start="YES"
  jail_list="nfs_server"

The nfs_server FreeBSD Jail as configured in the /etc/jail.conf config file.

root@host:/ # cat /etc/jail.conf
nfs_server {
  path = "/jail/${name}";
  host.hostname = "${name}";
  allow.raw_sockets = 1;
  allow.set_hostname = 1;
  allow.sysvipc = 1;
  mount.devfs;
  exec.clean;
  vnet;
  vnet.interface = "e0b_${name}";
  exec.prestart += "/usr/sbin/jib addm -b _bridge0 ${name} em0";
  exec.poststop += "/usr/sbin/jib destroy ${name}";
  exec.start += "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.consolelog = "/var/log/jail_${name}_console.log";
}

… and last but not least lets make sure the following DNS haiku will not bother us πŸ™‚

dns

Setup the /etc/hosts on the host system.

root@host:/ # tail -3 /etc/hosts
10.0.10.250 host
10.0.10.251 nfs_server

FreeBSD NFS Server VNET Jail (nfs_server)

As our FreeBSD Jail is installed we will now start it and configure it.

root@host:/ # service jail onestart nfs_server

root@host:/ # jls
   JID  IP Address      Hostname                      Path
     1                  nfs_server                    /jail/nfs_server

root@host:/ # jexec 1

root@nfs_server:/ # 

First we will install latest net/unfs3 package – this userspace NFS server is also very minimal and does not have any dependencies.

root@nfs_server:/ # echo nameserver 1.1.1.1 > /etc/resolv.conf

root@nfs_server:/ # sed -i '' s/quarterly/latest/g /etc/pkg/FreeBSD.conf

root@nfs_server:/ # pkg install unfs3

root@nfs_server:/ # pkg info -qoa
ports-mgmt/pkg
net/unfs3

Now we will configure our NFS share under /share dir and start the unfsd(8) userspace NFS server.

root@nfs_server:/ # mkdir /share

root@nfs_server:/ # cat /etc/exports
/share  10.0.10.250(rw,no_root_squash,no_all_squash)

… last but not least – DNS πŸ™‚

root@nfs_server:/ # tail -3 /etc/hosts
10.0.10.250 host
10.0.10.251 nfs_server

As we are using VNET network stack in a FreeBSD Jail we will have to address the network interface in the Jails /etc/rc.conf file. The unfsd(8) daemon does not start without rpcbind service so we will also enable it.

root@nfs_server:/ # cat /etc/rc.conf
# NETWORK
  hostname="nfs_server"
  ifconfig_e0b_nfs_server="10.0.10.251/24 up"
  defaultrouter="10.0.10.1"

# DAEMONS
  sshd_enable="YES"
  rpcbind_enable="YES"
  sendmail_enable="NO"
  sendmail_submit_enable="NO"
  sendmail_outbound_enable="NO"
  sendmail_msp_queue_enable="NO"

We will make unfsd(8) start automatically at Jails start with plain old /etc/rc.local file.

root@nfs_server:/ # cat /etc/rc.local 
/usr/local/sbin/unfsd &

We will not restart our FreeBSD Jail to make these changes take effect.

root@host:/ # service jail onerestart nfs_server

root@host:/ # jls
   JID  IP Address      Hostname                      Path
     2                  nfs_server                    /jail/nfs_server

root@host:/ # jexec 2

root@nfs_server:/ # 

After startup we can see that unfsd(8) is listening on a NFS 2049 port.

root@nfs_server:/ # sockstat -l4
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     sshd       1261  4  tcp4   *:22                  *:*
root     sendmail   1241  5  tcp4   127.0.0.1:25          *:*
root     unfsd      1223  3  udp4   *:2049                *:*
root     unfsd      1223  4  tcp4   *:2049                *:*
root     rpcbind    1196  9  udp4   *:111                 *:*
root     rpcbind    1196  10 udp4   *:842                 *:*
root     rpcbind    1196  11 tcp4   *:111                 *:*
root     syslogd    1188  6  udp4   *:514                 *:*
  

We should have our epair(4) interface called e0b_nfs_server addressed properly.

root@nfs_server:/ # ifconfig 
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
e0b_nfs_server: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 0e:27:dd:b3:81:88
        hwaddr 02:30:0d:9f:57:0b
        inet 10.0.10.251 netmask 0xffffff00 broadcast 10.0.10.255
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Mount /share on NFS Client

I added that NFS entry to the /etc/fstab file on the host machine.

root@host:~ # cat /etc/fstab 
#DEV         #MNT       #TYPE    #OPT  #DUMP/PASS
/dev/ada0p1  /boot/efi  msdosfs  rw    2 2
/dev/ada0p3  none       swap     sw    0 0

#DEV                #MNT  #TYPE  #OPT       #DUMP/PASS
10.0.10.251:/share  /mnt  nfs    rw,noauto  0 0

We will now attempt to mount the /share NFS export on the host machine.

root@host:/ # mount /mnt

root@host:/ # mount | grep share
10.0.10.251:/share on /mnt (nfs)

root@host:/ # cd /mnt

root@host:/mnt # :> FILE

root@host:/mnt # ls -l FILE
-rw-r--r-- 1 root root 0 2022-05-21 22:53 FILE

root@host:/mnt # rm FILE

Seems to work properly.

Here are also network interfaces on the host machine.

root@host:/ # ifconfig 
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
        ether 08:00:27:b3:81:88
        inet 10.0.10.250 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0_bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 58:9c:fc:10:ff:dd
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: e0a_nfs_server flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
        member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
e0a_nfs_server: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:27:dd:b3:81:88
        hwaddr 02:30:0d:9f:57:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


Future of NFS Server in Jails

This setup – while allowing to run the NFS server inside FreeBSD Jail with even VNET enabled has its drawbacks unfortunately. First is that it is run in userspace instead of kernel space – which means its slower. Second is that the unfsd(8) only implements NFS version 3 – so no version 4 is not possible.

freebsd-foundation-logo

Where we can go from here? Like with WiFi stuff IMHO the FreeBSD Foundation could step in to sponsor the missing bits of NFS server and VNET to make these native tools work they should. Its up to you to put the pressure on the FreeBSD Foundation when they as what are you missing from the FreeBSD UNIX system that could be improved with one of their projects. You may also join the discussion at the 251347 Bug Report of course.

I think its a big loss that native kernel space NFS server is not currently possible with VNET FreeBSD Jails.

EOF

Unbound DNS Blacklist

Today I will show you how to configure unbound(8) to block spam/malicious/malware domains at DNS level.

unbound

I will use FreeBSD for that purpose but you can use any system that unbound(8) runs on.

logo-freebsd

Earlier I used generated /etc/hosts file but that was limited in several ways. The ZSH shell will autocomplete all these blocked domains to the ssh(1)/scp(1) commands (which takes needless time and shows useless completions). Subdomains are not handled. The malicious.com is blocked but ads.malicious.com is not. You need to duplicate all those domains in the /etc/hosts file.

TL;DR

Not all people have time for my long boring stories so this is meritum of this article.

# rm -rf /var/unbound
# mkdir -p /var/unbound/conf.d
# chown -R unbound:unbound /var/unbound
# service local_unbound setup
# service local_unbound enable
# service local_unbound start
# mkdir /root/bin
# cd 
# fetch -o /root/bin/unbound-blacklist-fetch.sh \
> https://raw.githubusercontent.com/vermaden/scripts/master/unbound-blacklist-fetch.sh
# chmod +x /root/bin/unbound-blacklist-fetch.sh
# /root/bin/unbound-blacklist-fetch.sh
# service local_unbound restart
# cat << BSD >> /var/cron/tabs/root
> # FETCH FRESH unbound(8) BLACKLIST
>   0 0 * * * /root/bin/unbound-blacklist-fetch.sh
> BSD

Whole Story

The unbound(8) caching DNS resolver has been added to FreeBSD base system in 2014 with 10.0-RELEASE version so being on FreeBSD you do not need to install anything. We will start with cleaning the any existing unbound(8) configuration which relies at /var/unbound. Keep in mind that /etc/unbound links to it.

# ls -l -d /etc/unbound /var/unbound
lrwxr-xr-x 1 root    wheel   14 2019.09.21 16:23 /etc/unbound -> ../var/unbound
drwxr-xr-x 3 unbound unbound  8 2020.11.17 16:48 /var/unbound

# rm -rf /var/unbound

# mkdir -p /var/unbound/conf.d

# chown -R unbound:unbound /var/unbound

The service local_unbound setup will create all needed configuration.

Just keep in mind that this process will setup all DNS servers that you have in the /etc/resolv.conf file.

You may want to put two of your favorite DNS servers before this process.

Configuration

# cat << BSD > /etc/resolv.conf
nameserver 9.9.9.9
nameserver 1.1.1.1
BSD

# service local_unbound setup
Performing initial setup.
destination: 
Extracting forwarders from /etc/resolv.conf.
/var/unbound/forward.conf created
/var/unbound/lan-zones.conf created
/var/unbound/control.conf created
/var/unbound/unbound.conf created
/etc/resolvconf.conf created
Original /etc/resolv.conf saved as /var/backups/resolv.conf.20201115.235254

# rm /var/backups/resolv.conf.20201115.235254

# find /var/unbound
/var/unbound
/var/unbound/lan-zones.conf
/var/unbound/control.conf
/var/unbound/unbound.conf
/var/unbound/forward.conf

% find /var/unbound -ls
 12685  17  drwxr-xr-x  3  unbound  unbound    8  Nov 17 16:48  /var/unbound
 13072   1  -rw-r--r--  1  root     unbound   98  Nov 17 05:00  /var/unbound/forward.conf
 12688   9  -rw-r--r--  1  root     unbound  354  Nov 15 23:56  /var/unbound/unbound.conf
 12686   1  drwxr-xr-x  2  unbound  unbound    3  Nov 16 00:23  /var/unbound/conf.d
 12158   9  -rw-r--r--  1  root     unbound  193  Nov 15 23:56  /var/unbound/control.conf
 11732   9  -rw-r--r--  1  root     unbound  189  Nov 15 23:56  /var/unbound/lan-zones.conf

# tail -n 999 /var/unbound/*
==> /var/unbound/conf.d <==
tail: /var/unbound/conf.d: Is a directory

==> /var/unbound/control.conf <==
# This file was generated by local-unbound-setup.
# Modifications will be overwritten.
remote-control:
	control-enable: yes
	control-interface: /var/run/local_unbound.ctl
	control-use-cert: no

==> /var/unbound/forward.conf <==
# Generated by resolvconf

forward-zone:
	name: "."
	forward-addr: 9.9.9.9
	forward-addr: 1.1.1.1

==> /var/unbound/lan-zones.conf <==
# This file was generated by local-unbound-setup.
# Modifications will be overwritten.
server:
	# Unblock reverse lookups for LAN addresses
	unblock-lan-zones: yes
	insecure-lan-zones: yes

==> /var/unbound/unbound.conf <==
# This file was generated by local-unbound-setup.
# Modifications will be overwritten.
server:
	username: unbound
	directory: /var/unbound
	chroot: /var/unbound
	pidfile: /var/run/local_unbound.pid
	auto-trust-anchor-file: /var/unbound/root.key

include: /var/unbound/lan-zones.conf
include: /var/unbound/control.conf
include: /var/unbound/conf.d/*.conf

We will now enable the local_unbound service and start it. At this point without any DNS blocking configuration.

# service local_unbound enable
local_unbound enabled in /etc/rc.conf

# service local_unbound start
Starting local_unbound.

The /etc/resolv.conf will now have hour favorite DNS servers hashed/disabled and 127.0.0.1 address will be specified. You can also use sockstat(8) to check that unbound(8) is indeed listening on port 53.

# cat /etc/resolv.conf
# nameserver 9.9.9.9
# nameserver 1.1.1.1
nameserver 127.0.0.1
options edns0

% sockstat -l -4
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS 
unbound local-unbo 7362 5 udp4 127.0.0.1:53 *:*
unbound local-unbo 7362 6 tcp4 127.0.0.1:53 *:*

Test

After unbound(8) has been enabled it should now be visible that first DNS request should be longer and the second one and following requests should be very fast.

% time host ftp.freebsd.org
ftp.freebsd.org is an alias for ftp.geo.freebsd.org.
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has address 213.138.116.78
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org has IPv6 address 2001:41c8:112:8300::15:0
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org mail is handled by 0 .
host ftp.freebsd.org  0.00s user 0.01s system 1% cpu 0.501 total

% time host ftp.freebsd.org
ftp.freebsd.org is an alias for ftp.geo.freebsd.org.
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has address 213.138.116.78
ftp.geo.freebsd.org has address 139.178.72.202
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org has IPv6 address 2001:41c8:112:8300::15:0
ftp.geo.freebsd.org has IPv6 address 2604:1380:2000:9501::15:0
ftp.geo.freebsd.org mail is handled by 0 .
host ftp.freebsd.org  0.01s user 0.00s system 88% cpu 0.007 total

Yep. Works.

Blacklist

I have written a simple and short unbound-blacklist-fetch.sh to automate the process of generating up to date DNS blocked domains config.

It uses one unbound(8) source and several hosts(5) sources, then combines them in unbound(8) compatible format while removing the duplicated entries.

unbound-blacklist-script.256

We will now fetch it, put it under /root/bin directory (or use your favorite one), make it executable and start it.

# mkdir /root/bin

# fetch -o /root/bin/unbound-blacklist-fetch.sh \
> https://raw.githubusercontent.com/vermaden/scripts/master/unbound-blacklist-fetch.sh

# chmod +x /root/bin/unbound-blacklist-fetch.sh

# /root/bin/unbound-blacklist-fetch.sh

# ls -l /var/unbound/conf.d/blacklist.conf
-rw-r--r-- 1 root unbound 3003929 2020.11.16 00:23 /var/unbound/conf.d/blacklist.conf

# tail /var/unbound/conf.d/blacklist.conf
local-zone: "zyrtec.1.p2l.info" always_nxdomain
local-zone: "zyrtec.3.p2l.info" always_nxdomain
local-zone: "zyrtec.4.p2l.info" always_nxdomain
local-zone: "zyski-z-innowacji.pl" always_nxdomain
local-zone: "zytpirwai.net" always_nxdomain
local-zone: "zz.cqcounter.com" always_nxdomain
local-zone: "zzhc.vnet.cn" always_nxdomain
local-zone: "zzz.clickbank.net" always_nxdomain
local-zone: "zzz.onion.pet" always_nxdomain
local-zone: "zzzrtrcm2.com" always_nxdomain

The unbound(8) daemon already includes all /var/unbound/conf.d/*.conf files and we use that here.

You can change where the script generates blocked domains config under the # SETTINGS section directly in the script.

% grep -A 5 SETTINGS scripts/unbound-blacklist-fetch.sh 
# SETTINGS
FILE=/var/unbound/conf.d/blacklist.conf
TEMP=/tmp/unbound
TYPE=always_nxdomain
ECHO=0

After the /var/unbound/conf.d/blacklist.conf file is generated you can now restart the unbound(8) service.

# service local_unbound restart
Stopping local_unbound.
Waiting for PIDS: 87745.
Starting local_unbound.
Waiting for nameserver to start... good

We will also add that script to crontab(5) so it will fetch fresh information every day.

# cat << BSD >> /var/cron/tabs/root
> 
> # FETCH FRESH unbound(8) BLACKLIST
>   0 0 * * * /root/bin/unbound-blacklist-fetch.sh
> 
> BSD

# crontab -l | tail -4

# FETCH FRESH unbound(8) BLACKLIST
  0 0 * * * /root/bin/unbound-blacklist-fetch.sh

Test Blocked Domains

From 60000+ blocked domains I have chosen ad.track.us.org as target for verification.

% ping ad.track.us.org
ping: cannot resolve ad.track.us.org: Unknown host

% host ad.track.us.org
Host ad.track.us.org not found: 3(NXDOMAIN)

% dog ad.track.us.org
Status: NXDomain

% dog @1.1.1.1 ad.track.us.org
CNAME ad.track.us.org. 11m30s   "track.us.org."
    A track.us.org.     6m30s   185.59.208.177


unbound-test.256

As You can see the domain is successfully blocked.

The above blocking configuration does not mean that I will now disable the uBlock Origin plugin from Firefox but its a welcome addition to blocking unwanted information tools workshop.

UPDATE 1 – Reworked Script and Alternatives

After reading comments on Hacker News / Lobsters / Reddit I got a lot of good ideas how to improve my script even more.

Some people suggested that very similar functionality already exists in dns/void-zones-tools package on FreeBSD. One can also use get_unbound_adblock.sh script or lie-to-me solution.

There are also more sophisticated tools like Pi-hole which also include DHCP server and web interface for management and statistics. Unfortunately Pi-hole does not run on FreeBSD.

After reworking and adding additional sources to my unbound-blacklist-fetch.sh script its now twice the amount of blocked unwanted domains. In the first release about 60000 domains were blocked. Now its more then 120000.

Here is the distribution of data between various types of sources.

% wc -lc /tmp/unbound/lists-*
   54587 1059592 /tmp/unbound/lists-domains
  143553 4115745 /tmp/unbound/lists-hosts
   32867 1596409 /tmp/unbound/lists-unbound
  231007 6771746 total

Now the /var/unbound/conf.d/blacklist.conf before these changes.

% wc -l blacklist.conf
   60009 blacklist.conf

% ls -l /var/unbound/conf.d/blacklist.conf
-rw-r--r-- 1 root unbound 2907535 2020-11-20 00:00 /var/unbound/conf.d/blacklist.conf

… and after adding additional sources.

% wc -l blacklist.conf
  122190 blacklist.conf

% ls -l /var/unbound/conf.d/blacklist.conf
-rw-r--r-- 1 root unbound 6086623 2020-11-20 15:07 /var/unbound/conf.d/blacklist.conf

Here is also performance summary about which part takes what amount of time.

Combining various sources and generating the final config takes about 5 seconds.

Most of the time is spent in fetching the data from various sources.

UPDATE1.unbound.script.256

The script is already uploaded to the GitHub repo.

Just fetch it and enjoy πŸ™‚

UPDATE 2 – Huge Domains List Version

Thanks to Luca Castagnini from bsd.network who pointed me to https://oisd.nl/ site with HUGE list of domains that can/could/should be blocked I made another variant (or version) of the script unbound-blacklist-fetch-huge.sh with a total of 145 (!) various sources for domains to block.

It of course takes little longer to fetch and generate then the ‘casual’ version.

UPDATE2.unbound.time

Its little less then 2 minutes to fetch and generate new config while the longest part is the fetching of those 145 sources. Generation takes about 15 seconds.

These 145 sources provide more then a million domains to block.

% wc -l /tmp/unbound/* 
 551704 lists-domains
 439505 lists-hosts
  60835 lists-unbound
1052044 total

The script after removing duplicated entries makes little more then 480000 domains of it.

% wc -l /var/unbound/conf.d/blacklist.conf 
 484829 /var/unbound/conf.d/blacklist.conf

Unfortunately it comes at a price. In this HUGE variant with domains from 145 sources the unbound(8) server now uses about 150 MB of RAM.

% top -b -o res|grep -E 'RES|unbound'
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C    TIME    WCPU COMMAND
75849 unbound       1  20    0   158M   149M select   4    0:03   0.00% local-unbound

I leave up to you which version to use and which sources to choose for blocking, but as my Firefox with about 20 tabs opened takes little more then 4226 MB of RAM these additional 150 MB from unbound(8) does not hurt that much πŸ™‚

% ./FIREFOX.RAM.sh
4226 MB

% cat FIREFOX.RAM.sh 
#! /bin/sh

SUM=0

top -b -o res \
  | sed 1,10d \
  | grep firefox \
  | awk '{print $7}' \
  | tr -cd '0-9\n' \
  | while read I
    do
      SUM=$(( ${SUM} + ${I} ))
      echo ${SUM}
    done | tail -1 | tr -d '\n'
echo " MB"
One more thing related to Firefox. After checking ‘free’ memory with Firefox running and after closing it the difference was about 2.6 GB which means that above script to calculate Firefox memory usage is not a lot accurate πŸ™‚
EOF

FreeBSD Cluster with Pacemaker and Corosync

I always missed ‘proper’ cluster software for FreeBSD systems. Recently I got to run several Pacemaker/Corosync based clusters on Linux systems. I thought how to make similar high availability solutions on FreeBSD and I was really shocked when I figured out that both Pacemaker and Corosync tools are available in the FreeBSD Ports and packages as net/pacemaker2 and net/corosync2 respectively.

In this article I will check how well Pacemaker and Corosync cluster works on FreeBSD.

pacemaker

There are many definitions of a cluster. One that I like the most is that a cluster is a system that is still redundant after losing one of its nodes (is still a cluster). This means that 3 nodes is a minimum for a cluster by that definition. The two node clusters are quite problematic because of their biggest exposure to the split brain problem. That is why often in the two node clusters additional devices or systems are added to make sure that this split brain does not happen. For example one can add third node without any resources or services just as a ‘witness’ role. Other way is to add a shared disk resource that will serve the same purpose and often its a raw volume with SCSI-3 Persistent Reservation mechanism used.

Lab Setup

As usual it will be entirely VirtualBox based and it will consist of 3 hosts. To not create 3 same FreeBSD installations I used 12.1-RELEASE virtual machine image available from the FreeBSD Project directly:

There are several formats available – qcow2/raw/vhd/vmdk – but as I will be using VirtualBox I used the VMDK one.

Here is the list of the machines for the GlusterFS cluster:

  • 10.0.10.111 node1
  • 10.0.10.112 node2
  • 10.0.10.113 node3

Each VirtualBox virtual machine for FreeBSD is the default one (as suggested in the VirtualBox wizard) with 512 MB RAM and NAT Network as shown on the image below.

machine

Here is the configuration of the NAT Network on VirtualBox.

nat-network-01

nat-network-02

Before we will try connect to our FreeBSD machines we need to make the minimal network configuration inside each VM. Each FreeBSD machine will have such minimal /etc/rc.conf file as shown example for node1 host.

root@node1:~ # cat /etc/rc.conf
hostname=node1
ifconfig_em0="inet 10.0.10.111/24 up"
defaultrouter=10.0.10.1
sshd_enable=YES

For the setup purposes we will need to allow root login on these FreeBSD machines with PermitRootLogin yes option in the /etc/ssh/sshd_config file. You will also need to restart the sshd(8) service after the changes.

root@node1:~ # grep PermitRootLogin /etc/ssh/sshd_config
PermitRootLogin yes

root@node1:~ # service sshd restart

By using NAT Network with Port Forwarding the FreeBSD machines will be accessible on the localhost ports. For example the node1 machine will be available on port 2211, the node2 machine will be available on port 2212 and so on. This is shown in the sockstat utility output below.

nat-network-03-sockstat

nat-network-04-ssh

To connect to such machine from the VirtualBox host system you will need this command:

vboxhost % ssh -l root localhost -p 2211

Packages

As we now have ssh(1) connectivity we need to add needed packages. To make our VMs resolve DNS queries we need to add one last thing. We will also switch to ‘quarterly’ branch of the pkg(8) packages.

root@node1:~ # echo 'nameserver 1.1.1.1' > /etc/resolv.conf
root@node1:~ # sed -i '' s/quarterly/latest/g /etc/pkg/FreeBSD.conf

Remember to repeat these two upper commands on node2 and node3 systems.

Now we will add Pacemaker and Corosync packages.

root@node1:~ # pkg install pacemaker2 corosync2 crmsh

root@node2:~ # pkg install pacemaker2 corosync2 crmsh

root@node3:~ # pkg install pacemaker2 corosync2 crmsh

These are messages both from pacemaker2 and corosync2 that we need to address.

Message from pacemaker2-2.0.4:

--
For correct operation, maximum socket buffer size must be tuned
by performing the following command as root :

# sysctl kern.ipc.maxsockbuf=18874368

To preserve this setting across reboots, append the following
to /etc/sysctl.conf :

kern.ipc.maxsockbuf=18874368

======================================================================

Message from corosync2-2.4.5_1:

--
For correct operation, maximum socket buffer size must be tuned
by performing the following command as root :

# sysctl kern.ipc.maxsockbuf=18874368

To preserve this setting across reboots, append the following
to /etc/sysctl.conf :

kern.ipc.maxsockbuf=18874368

We need to change the kern.ipc.maxsockbuf parameter. Lets do it then.

root@node1:~ # echo 'kern.ipc.maxsockbuf=18874368' >> /etc/sysctl.conf
root@node1:~ # service sysctl restart

root@node2:~ # echo 'kern.ipc.maxsockbuf=18874368' >> /etc/sysctl.conf
root@node2:~ # service sysctl restart

root@node3:~ # echo 'kern.ipc.maxsockbuf=18874368' >> /etc/sysctl.conf
root@node3:~ # service sysctl restart

Lets check what binaries come with these packages.

root@node1:~ # pkg info -l pacemaker2 | grep bin
        /usr/local/sbin/attrd_updater
        /usr/local/sbin/cibadmin
        /usr/local/sbin/crm_attribute
        /usr/local/sbin/crm_diff
        /usr/local/sbin/crm_error
        /usr/local/sbin/crm_failcount
        /usr/local/sbin/crm_master
        /usr/local/sbin/crm_mon
        /usr/local/sbin/crm_node
        /usr/local/sbin/crm_report
        /usr/local/sbin/crm_resource
        /usr/local/sbin/crm_rule
        /usr/local/sbin/crm_shadow
        /usr/local/sbin/crm_simulate
        /usr/local/sbin/crm_standby
        /usr/local/sbin/crm_ticket
        /usr/local/sbin/crm_verify
        /usr/local/sbin/crmadmin
        /usr/local/sbin/fence_legacy
        /usr/local/sbin/iso8601
        /usr/local/sbin/pacemaker-remoted
        /usr/local/sbin/pacemaker_remoted
        /usr/local/sbin/pacemakerd
        /usr/local/sbin/stonith_admin

root@node1:~ # pkg info -l corosync2 | grep bin
        /usr/local/bin/corosync-blackbox
        /usr/local/sbin/corosync
        /usr/local/sbin/corosync-cfgtool
        /usr/local/sbin/corosync-cmapctl
        /usr/local/sbin/corosync-cpgtool
        /usr/local/sbin/corosync-keygen
        /usr/local/sbin/corosync-notifyd
        /usr/local/sbin/corosync-quorumtool

root@node1:~ # pkg info -l crmsh | grep bin
        /usr/local/bin/crm

Cluster Initialization

Now we will initialize our FreeBSD cluster.

First we need to make sure that names of the nodes are DNS resolvable.

root@node1:~ # tail -3 /etc/hosts

10.0.10.111 node1
10.0.10.112 node2
10.0.10.113 node3

root@node2:~ # tail -3 /etc/hosts

10.0.10.111 node1
10.0.10.112 node2
10.0.10.113 node3

root@node3:~ # tail -3 /etc/hosts

10.0.10.111 node1
10.0.10.112 node2
10.0.10.113 node3


Now we will generate the Corosync key.

root@node1:~ # corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /usr/local/etc/corosync/authkey.

root@node1:~ # echo $?
0

root@node1:~ # ls -l /usr/local/etc/corosync/authkey
-r--------  1 root  wheel  128 Sep  2 20:37 /usr/local/etc/corosync/authkey

Now the Corosync configuration file. For sure some examples were provided by the package maintainer.

root@node1:~ # pkg info -l corosync2 | grep example
        /usr/local/etc/corosync/corosync.conf.example
        /usr/local/etc/corosync/corosync.conf.example.udpu

We will take the second one as a base for our config.

root@node1:~ # cp /usr/local/etc/corosync/corosync.conf.example.udpu /usr/local/etc/corosync/corosync.conf

root@node1:~ # vi /usr/local/etc/corosync/corosync.conf
               /* LOTS OF EDITS HERE */

root@node1:~ # cat /usr/local/etc/corosync/corosync.conf

totem {
  version: 2
  crypto_cipher: aes256
  crypto_hash: sha256
  transport: udpu

  interface {
    ringnumber: 0
    bindnetaddr: 10.0.10.0
    mcastport: 5405
    ttl: 1
  }
}

logging {
  fileline: off
  to_logfile: yes
  to_syslog: no
  logfile: /var/log/cluster/corosync.log
  debug: off
  timestamp: on

  logger_subsys {
    subsys: QUORUM
    debug: off
  }
}

nodelist {

  node {
    ring0_addr: 10.0.10.111
    nodeid: 1
  }

  node {
    ring0_addr: 10.0.10.112
    nodeid: 2
  }

  node {
    ring0_addr: 10.0.10.113
    nodeid: 3
  }

}

quorum {
  provider: corosync_votequorum
  expected_votes: 2
}

Now we need to propagate both Corosync key and config across the nodes in the cluster.

We can use some simple tools created exactly for that like net/csync2 cluster synchronization tool for example but plain old net/rsync will serve as well.

root@node1:~ # pkg install -y rsync

root@node1:~ # rsync -av /usr/local/etc/corosync/ node2:/usr/local/etc/corosync/
The authenticity of host 'node2 (10.0.10.112)' can't be established.
ECDSA key fingerprint is SHA256:/ZDmln7GKi6n0kbad73TIrajPjGfQqJJX+ReSf3NMvc.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node2' (ECDSA) to the list of known hosts.
Password for root@node2:
sending incremental file list
./
authkey
corosync.conf
service.d/
uidgid.d/

sent 1,100 bytes  received 69 bytes  259.78 bytes/sec
total size is 4,398  speedup is 3.76

root@node1:~ # rsync -av /usr/local/etc/corosync/ node3:/usr/local/etc/corosync/
The authenticity of host 'node2 (10.0.10.112)' can't be established.
ECDSA key fingerprint is SHA256:/ZDmln7GKi6n0kbad73TIrajPjGfQqJJX+ReSf3NMvc.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node3' (ECDSA) to the list of known hosts.
Password for root@node3:
sending incremental file list
./
authkey
corosync.conf
service.d/
uidgid.d/

sent 1,100 bytes  received 69 bytes  259.78 bytes/sec
total size is 4,398  speedup is 3.76

Now lets check that they are the same.

root@node1:~ # cksum /usr/local/etc/corosync/{authkey,corosync.conf}
2277171666 128 /usr/local/etc/corosync/authkey
1728717329 622 /usr/local/etc/corosync/corosync.conf

root@node2:~ # cksum /usr/local/etc/corosync/{authkey,corosync.conf}
2277171666 128 /usr/local/etc/corosync/authkey
1728717329 622 /usr/local/etc/corosync/corosync.conf

root@node3:~ # cksum /usr/local/etc/corosync/{authkey,corosync.conf}
2277171666 128 /usr/local/etc/corosync/authkey
1728717329 622 /usr/local/etc/corosync/corosync.conf

Same.

We can now add corosync_enable=YES and pacemaker_enable=YES to the /etc/rc.conf file.

root@node1:~ # sysrc corosync_enable=YES
corosync_enable:  -> YES

root@node1:~ # sysrc pacemaker_enable=YES
pacemaker_enable:  -> YES

root@node2:~ # sysrc corosync_enable=YES
corosync_enable:  -> YES

root@node2:~ # sysrc pacemaker_enable=YES
pacemaker_enable:  -> YES

root@node3:~ # sysrc corosync_enable=YES
corosync_enable:  -> YES

root@node3:~ # sysrc pacemaker_enable=YES
pacemaker_enable:  -> YES

Lets start these services then.

root@node1:~ # service corosync start
Starting corosync.
Sep 02 20:55:35 notice  [MAIN  ] Corosync Cluster Engine ('2.4.5'): started and ready to provide service.
Sep 02 20:55:35 info    [MAIN  ] Corosync built-in features:
Sep 02 20:55:35 warning [MAIN  ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
Sep 02 20:55:35 warning [MAIN  ] Please migrate config file to nodelist.

root@node1:~ # ps aux | grep corosync
root  1695   0.0  7.9 38340 38516  -  S    20:55    0:00.40 /usr/local/sbin/corosync
root  1699   0.0  0.1   524   336  0  R+   20:57    0:00.00 grep corosync

Do the same on the node2 and node3 systems.

The Pacemaker is not yet running so that will fail.

root@node1:~ # crm status
Could not connect to the CIB: Socket is not connected
crm_mon: Error: cluster is not available on this node
ERROR: status: crm_mon (rc=102): 

We will start it now.

root@node1:~ # service pacemaker start
Starting pacemaker.

root@node2:~ # service pacemaker start
Starting pacemaker.

root@node3:~ # service pacemaker start
Starting pacemaker.

You need to give it little time to start because if you will execute crm status command right away you will get 0 nodes configured message as shown below.

root@node1:~ # crm status
Cluster Summary:
  * Stack: unknown
  * Current DC: NONE
  * Last updated: Wed Sep  2 20:58:51 2020
  * Last change:  
  * 0 nodes configured
  * 0 resource instances configured


Full List of Resources:
  * No resources

… but after a while everything is detected and works as desired.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 21:02:49 2020
  * Last change:  Wed Sep  2 20:59:00 2020 by hacluster via crmd on node2
  * 3 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * No resources

The Pacemaker runs properly.

root@node1:~ # ps aux | grep pacemaker
root      1716   0.0  0.5 10844   2396  -  Is   20:58     0:00.00 daemon: /usr/local/sbin/pacemakerd[1717] (daemon)
root      1717   0.0  5.2 49264  25284  -  S    20:58     0:00.27 /usr/local/sbin/pacemakerd
hacluster 1718   0.0  6.1 48736  29708  -  Ss   20:58     0:00.75 /usr/local/libexec/pacemaker/pacemaker-based
root      1719   0.0  4.5 40628  21984  -  Ss   20:58     0:00.28 /usr/local/libexec/pacemaker/pacemaker-fenced
root      1720   0.0  2.8 25204  13688  -  Ss   20:58     0:00.20 /usr/local/libexec/pacemaker/pacemaker-execd
hacluster 1721   0.0  3.9 38148  19100  -  Ss   20:58     0:00.25 /usr/local/libexec/pacemaker/pacemaker-attrd
hacluster 1722   0.0  2.9 25460  13864  -  Ss   20:58     0:00.17 /usr/local/libexec/pacemaker/pacemaker-schedulerd
hacluster 1723   0.0  5.4 49304  26300  -  Ss   20:58     0:00.41 /usr/local/libexec/pacemaker/pacemaker-controld
root      1889   0.0  0.6 11348   2728  0  S+   21:56     0:00.00 grep pacemaker

We can check how Corosync sees its members.

root@node1:~ # corosync-cmapctl | grep members
runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(10.0.10.111) 
runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.1.status (str) = joined
runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(10.0.10.112) 
runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.2.status (str) = joined
runtime.totem.pg.mrp.srp.members.3.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.3.ip (str) = r(0) ip(10.0.10.113) 
runtime.totem.pg.mrp.srp.members.3.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.3.status (str) = joined

… or the quorum information.

root@node1:~ # corosync-quorumtool
Quorum information
------------------
Date:             Wed Sep  2 21:00:38 2020
Quorum provider:  corosync_votequorum
Nodes:            3
Node ID:          1
Ring ID:          1/12
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
         1          1 10.0.10.111 (local)
         2          1 10.0.10.112
         3          1 10.0.10.113

The Corosync log file is filled with the following information.

root@node1:~ # cat /var/log/cluster/corosync.log
Sep 02 20:55:35 [1694] node1 corosync notice  [MAIN  ] Corosync Cluster Engine ('2.4.5'): started and ready to provide service.
Sep 02 20:55:35 [1694] node1 corosync info    [MAIN  ] Corosync built-in features:
Sep 02 20:55:35 [1694] node1 corosync warning [MAIN  ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.
Sep 02 20:55:35 [1694] node1 corosync warning [MAIN  ] Please migrate config file to nodelist.
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] Initializing transport (UDP/IP Unicast).
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] Initializing transmit/receive security (NSS) crypto: aes256 hash: sha256
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] The network interface [10.0.10.111] is now up.
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync configuration map access [0]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: cmap
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync configuration service [1]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: cfg
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: cpg
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync profile loading service [4]
Sep 02 20:55:35 [1694] node1 corosync notice  [QUORUM] Using quorum provider corosync_votequorum
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync vote quorum service v1.0 [5]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: votequorum
Sep 02 20:55:35 [1694] node1 corosync notice  [SERV  ] Service engine loaded: corosync cluster quorum service v0.1 [3]
Sep 02 20:55:35 [1694] node1 corosync info    [QB    ] server name: quorum
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] adding new UDPU member {10.0.10.111}
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] adding new UDPU member {10.0.10.112}
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] adding new UDPU member {10.0.10.113}
Sep 02 20:55:35 [1694] node1 corosync notice  [TOTEM ] A new membership (10.0.10.111:4) was formed. Members joined: 1
Sep 02 20:55:35 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:55:35 [1694] node1 corosync notice  [QUORUM] Members[1]: 1
Sep 02 20:55:35 [1694] node1 corosync notice  [MAIN  ] Completed service synchronization, ready to provide service.
Sep 02 20:58:14 [1694] node1 corosync notice  [TOTEM ] A new membership (10.0.10.111:8) was formed. Members joined: 2
Sep 02 20:58:14 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:14 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:14 [1694] node1 corosync notice  [QUORUM] This node is within the primary component and will provide service.
Sep 02 20:58:14 [1694] node1 corosync notice  [QUORUM] Members[2]: 1 2
Sep 02 20:58:14 [1694] node1 corosync notice  [MAIN  ] Completed service synchronization, ready to provide service.
Sep 02 20:58:19 [1694] node1 corosync notice  [TOTEM ] A new membership (10.0.10.111:12) was formed. Members joined: 3
Sep 02 20:58:19 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:19 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:19 [1694] node1 corosync warning [CPG   ] downlist left_list: 0 received
Sep 02 20:58:19 [1694] node1 corosync notice  [QUORUM] Members[3]: 1 2 3
Sep 02 20:58:19 [1694] node1 corosync notice  [MAIN  ] Completed service synchronization, ready to provide service.

Here is the configuration.

root@node1:~ # crm configure show
node 1: node1
node 2: node2
node 3: node3
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=2.0.4-2deceaa3ae \
        cluster-infrastructure=corosync

As we will not be configuring the STONITH mechanism we will disable it.

root@node1:~ # crm configure property stonith-enabled=false

New configuraion with STONITH disabled.

root@node1:~ # crm configure show
node 1: node1
node 2: node2
node 3: node3
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=2.0.4-2deceaa3ae \
        cluster-infrastructure=corosync \
        stonith-enabled=false

The STONITH configuration is out of scope of this article but properly configured STONITH looks like that.

stonith

First Service

We will now configure our first highly available service – a classic – a floating IP address πŸ™‚

root@node1:~ # crm configure primitive IP ocf:heartbeat:IPaddr2 params ip=10.0.10.200 cidr_netmask="24" op monitor interval="30s"

Lets check how it behaves.

root@node1:~ # crm configure show
node 1: node1
node 2: node2
node 3: node3
primitive IP IPaddr2 \
        params ip=10.0.10.200 cidr_netmask=24 \
        op monitor interval=30s
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=2.0.4-2deceaa3ae \
        cluster-infrastructure=corosync \
        stonith-enabled=false

Looks good – lets check the cluster status.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:03:35 2020
  * Last change:  Wed Sep  2 22:02:53 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::heartbeat:IPaddr2):        Stopped

Failed Resource Actions:
  * IP_monitor_0 on node3 'not installed' (5): call=5, status='complete', exitreason='Setup problem: couldn't find command: ip', last-rc-change='2020-09-02 22:02:53Z', queued=0ms, exec=132ms
  * IP_monitor_0 on node2 'not installed' (5): call=5, status='complete', exitreason='Setup problem: couldn't find command: ip', last-rc-change='2020-09-02 22:02:54Z', queued=0ms, exec=120ms
  * IP_monitor_0 on node1 'not installed' (5): call=5, status='complete', exitreason='Setup problem: couldn't find command: ip', last-rc-change='2020-09-02 22:02:53Z', queued=0ms, exec=110ms

Crap. Linuxism. The ip(8) command is expected to be present in the system. This is FreeBSD and as any UNIX system it comes with ifconfig(8) command instead.

We will have to figure something else. For now we will delete our useless IP service.

root@node1:~ # crm configure delete IP

Status after deletion.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:04:34 2020
  * Last change:  Wed Sep  2 22:04:31 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 0 resource instances configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * No resources

Custom Resource

Lets check what resources are available by stock Pacemaker installation.

root@node1:~ # ls -l /usr/local/lib/ocf/resource.d/pacemaker
total 144
-r-xr-xr-x  1 root  wheel   7484 Aug 29 01:22 ClusterMon
-r-xr-xr-x  1 root  wheel   9432 Aug 29 01:22 Dummy
-r-xr-xr-x  1 root  wheel   5256 Aug 29 01:22 HealthCPU
-r-xr-xr-x  1 root  wheel   5342 Aug 29 01:22 HealthIOWait
-r-xr-xr-x  1 root  wheel   9450 Aug 29 01:22 HealthSMART
-r-xr-xr-x  1 root  wheel   6186 Aug 29 01:22 Stateful
-r-xr-xr-x  1 root  wheel  11370 Aug 29 01:22 SysInfo
-r-xr-xr-x  1 root  wheel   5856 Aug 29 01:22 SystemHealth
-r-xr-xr-x  1 root  wheel   7382 Aug 29 01:22 attribute
-r-xr-xr-x  1 root  wheel   7854 Aug 29 01:22 controld
-r-xr-xr-x  1 root  wheel  16134 Aug 29 01:22 ifspeed
-r-xr-xr-x  1 root  wheel  11040 Aug 29 01:22 o2cb
-r-xr-xr-x  1 root  wheel  11696 Aug 29 01:22 ping
-r-xr-xr-x  1 root  wheel   6356 Aug 29 01:22 pingd
-r-xr-xr-x  1 root  wheel   3702 Aug 29 01:22 remote

Not many … we will try to modify the Dummy service into an IP changer on FreeBSD.

root@node1:~ # cp /usr/local/lib/ocf/resource.d/pacemaker/Dummy /usr/local/lib/ocf/resource.d/pacemaker/ifconfig

root@node1:~ # vi /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
               /* LOTS OF TYPING */

Because of the WordPress blogging system limitations I am forced to post this ifconfig resource as an image … but fear not – the text version is also available here – ifconfig.odt – for download.

Also the first version did not went that well …

root@node1:~ # setenv OCF_ROOT /usr/local/lib/ocf
root@node1:~ # ocf-tester -n resourcename /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
Beginning tests for /usr/local/lib/ocf/resource.d/pacemaker/ifconfig...
* rc=3: Your agent has too restrictive permissions: should be 755
-:1: parser error : Start tag expected, '<' not found
usage: /usr/local/lib/ocf/resource.d/pacemaker/ifconfig {start|stop|monitor}
^
* rc=1: Your agent produces meta-data which does not conform to ra-api-1.dtd
* rc=3: Your agent does not support the meta-data action
* rc=3: Your agent does not support the validate-all action
* rc=0: Monitoring a stopped resource should return 7
* rc=0: The initial probe for a stopped resource should return 7 or 5 even if all binaries are missing
* Your agent does not support the notify action (optional)
* Your agent does not support the demote action (optional)
* Your agent does not support the promote action (optional)
* Your agent does not support master/slave (optional)
* rc=0: Monitoring a stopped resource should return 7
* rc=0: Monitoring a stopped resource should return 7
* rc=0: Monitoring a stopped resource should return 7
* Your agent does not support the reload action (optional)
Tests failed: /usr/local/lib/ocf/resource.d/pacemaker/ifconfig failed 9 tests

But after adding 755 mode to it and making several (hundred) changes it become usable.

root@node1:~ # vi /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
             /* LOTS OF NERVOUS TYPING */
root@node1:~ # chmod 755 /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
root@node1:~ # setenv OCF_ROOT /usr/local/lib/ocf
root@node1:~ # ocf-tester -n resourcename /usr/local/lib/ocf/resource.d/pacemaker/ifconfig
Beginning tests for /usr/local/lib/ocf/resource.d/pacemaker/ifconfig...
* Your agent does not support the notify action (optional)
* Your agent does not support the demote action (optional)
* Your agent does not support the promote action (optional)
* Your agent does not support master/slave (optional)
* Your agent does not support the reload action (optional)
/usr/local/lib/ocf/resource.d/pacemaker/ifconfig passed all tests

Looks usable.

The ifconfig resource. Its pretty limited and with hardcoded IP address as for now.

ifconfig

Lets try to add new IP resource to our FreeBSD cluster.

Tests

root@node1:~ # crm configure primitive IP ocf:pacemaker:ifconfig op monitor interval="30"

Added.

Lets see what status command now shows.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:44:52 2020
  * Last change:  Wed Sep  2 22:44:44 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node1

Failed Resource Actions:
  * IP_monitor_0 on node3 'not installed' (5): call=24, status='Not installed', exitreason='', last-rc-change='2020-09-02 22:42:52Z', queued=0ms, exec=5ms
  * IP_monitor_0 on node2 'not installed' (5): call=24, status='Not installed', exitreason='', last-rc-change='2020-09-02 22:42:53Z', queued=0ms, exec=2ms

Crap. I forgot to copy this new ifconfig resource to the other nodes. Lets fix that now.

root@node1:~ # rsync -av /usr/local/lib/ocf/resource.d/pacemaker/ node2:/usr/local/lib/ocf/resource.d/pacemaker/
Password for root@node2:
sending incremental file list
./
ifconfig

sent 3,798 bytes  received 38 bytes  1,534.40 bytes/sec
total size is 128,003  speedup is 33.37

root@node1:~ # rsync -av /usr/local/lib/ocf/resource.d/pacemaker/ node3:/usr/local/lib/ocf/resource.d/pacemaker/
Password for root@node3:
sending incremental file list
./
ifconfig

sent 3,798 bytes  received 38 bytes  1,534.40 bytes/sec
total size is 128,003  speedup is 33.37

Lets stop, delete and re-add our precious resource now.

root@node1:~ # crm resource stop IP
root@node1:~ # crm configure delete IP
root@node1:~ # crm configure primitive IP ocf:pacemaker:ifconfig op monitor interval="30"

Fingers crossed.

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:45:46 2020
  * Last change:  Wed Sep  2 22:45:43 2020 by root via cibadmin on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node1

Looks like running properly.

Lets verify that its really up where it should be.

root@node1:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:2a:78:60
        inet 10.0.10.111 netmask 0xffffff00 broadcast 10.0.10.255
        inet 10.0.10.200 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

root@node2:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:80:50:05
        inet 10.0.10.112 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

root@node3:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:74:5e:b9
        inet 10.0.10.113 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

Seems to be working.

Now lets try to move it to the other node in the cluster.

root@node1:~ # crm resource move IP node3
INFO: Move constraint created for IP to node3

root@node1:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:47:31 2020
  * Last change:  Wed Sep  2 22:47:28 2020 by root via crm_resource on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node3

Switched properly to node3 system.

root@node3:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:74:5e:b9
        inet 10.0.10.113 netmask 0xffffff00 broadcast 10.0.10.255
        inet 10.0.10.200 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

root@node1:~ # ifconfig em0
em0: flags=8843 metric 0 mtu 1500
        options=81009b
        ether 08:00:27:2a:78:60
        inet 10.0.10.111 netmask 0xffffff00 broadcast 10.0.10.255
        media: Ethernet autoselect (1000baseT )
        status: active
        nd6 options=29

Now we will poweroff the node3 system to check it that IP is really highly available.

root@node2:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:49:57 2020
  * Last change:  Wed Sep  2 22:47:29 2020 by root via crm_resource on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node3

root@node3:~ # poweroff

root@node2:~ # crm status
Cluster Summary:
  * Stack: corosync
  * Current DC: node2 (version 2.0.4-2deceaa3ae) - partition with quorum
  * Last updated: Wed Sep  2 22:50:16 2020
  * Last change:  Wed Sep  2 22:47:29 2020 by root via crm_resource on node1
  * 3 nodes configured
  * 1 resource instance configured

Node List:
  * Online: [ node1 node2 ]
  * OFFLINE: [ node3 ]

Full List of Resources:
  * IP  (ocf::pacemaker:ifconfig):       Started node1

Seems that failover went well.

The crm command also colors various sections of its output.

failover

Good to know that Pacemaker and Corosync cluster runs well on FreeBSD.

Some work is needed to write the needed resource files but one with some time and determination can surely put FreeBSD into a very capable highly available cluster.

EOF

Run broot on FreeBSD

The broot file manager is quite fresh and nice approach to files and directories filtering/searching/view/manipulation/… and whatever else you call messing with files πŸ™‚

The broot tools is not yet available on the FreeBSD systems (as package or port).

This guide will show you how to built and install it on your FreeBSD system.

Here is how it looks in action.

Filter for jails.

broot-filter-jails.jpg

Filter for zfs.

broot-filter-zfs.jpg

It has ‘size mode’ when started with -s option similar to ncdu(1) tool.

broot-filter-size.jpg

You can also check the Feature Showcase section on their GitHub page – https://github.com/Canop/broot – available here.

Build

There are three steps to make it happen.

1. You need to install the rust package.

# pkg install rust

Then you need to type (as regular user) the cargo install broot command.

% cargo install broot

It will fail here:

broot-fail.jpg

You will need to apply this patch below:

% diff -u \
  /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs.ORG \
  /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs
--- /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs.ORG  2020-01-10 23:41:29.825912000 +0100
+++ /home/vermaden/.cargo/registry/src/github.com-1ecc6299db9ec823/crossterm-0.14.1/src/terminal/sys/unix.rs      2020-01-10 23:41:07.703471000 +0100
@@ -33,7 +33,7 @@
         ws_ypixel: 0,
     };
 
-    if let Ok(true) = wrap_with_result(unsafe { ioctl(STDOUT_FILENO, TIOCGWINSZ, &mut size) }) {
+    if let Ok(true) = wrap_with_result(unsafe { ioctl(STDOUT_FILENO, TIOCGWINSZ.into(), &mut size) }) {
         Ok((size.ws_col, size.ws_row))
     } else {
         tput_size().ok_or_else(|| std::io::Error::last_os_error().into())

Then type cargo install broot command again. It will now properly compile.

% cargo install broot
    Updating crates.io index
  Downloaded broot v0.11.6
  Downloaded 1 crate (1.6 MB) in 2.89s
  Installing broot v0.11.6
   Compiling libc v0.2.66
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling autocfg v0.1.7
   Compiling semver-parser v0.7.0
   Compiling autocfg v1.0.0
   Compiling proc-macro2 v1.0.7
   Compiling log v0.4.8
   Compiling scopeguard v1.0.0
   Compiling unicode-xid v0.2.0
   Compiling bitflags v1.2.1
   Compiling syn v1.0.13
   Compiling memchr v2.2.1
   Compiling arc-swap v0.4.4
   Compiling slab v0.4.2
   Compiling smallvec v1.1.0
   Compiling serde v1.0.104
   Compiling unicode-width v0.1.7
   Compiling regex-syntax v0.6.13
   Compiling ansi_term v0.11.0
   Compiling strsim v0.8.0
   Compiling vec_map v0.8.1
   Compiling id-arena v2.2.1
   Compiling custom_error v1.7.1
   Compiling glob v0.3.0
   Compiling open v1.3.2
   Compiling umask v0.1.8
   Compiling thread_local v1.0.0
   Compiling minimad v0.6.3
   Compiling lazy-regex v0.1.2
   Compiling semver v0.9.0
   Compiling lock_api v0.3.3
   Compiling crossbeam-utils v0.7.0
   Compiling crossbeam-epoch v0.8.0
   Compiling num-traits v0.2.11
   Compiling num-integer v0.1.42
   Compiling textwrap v0.11.0
   Compiling rustc_version v0.2.3
   Compiling memoffset v0.5.3
   Compiling iovec v0.1.4
   Compiling net2 v0.2.33
   Compiling dirs-sys v0.3.4
   Compiling parking_lot_core v0.7.0
   Compiling signal-hook-registry v1.2.0
   Compiling time v0.1.42
   Compiling atty v0.2.14
   Compiling users v0.9.1
   Compiling quote v1.0.2
   Compiling aho-corasick v0.7.6
   Compiling mio v0.6.21
   Compiling dirs v2.0.2
   Compiling directories v2.0.2
   Compiling parking_lot v0.10.0
   Compiling clap v2.33.0
   Compiling crossbeam-queue v0.2.1
   Compiling crossbeam-channel v0.4.0
   Compiling toml v0.5.5
   Compiling term v0.6.1
   Compiling regex v1.3.3
   Compiling signal-hook v0.1.12
   Compiling chrono v0.4.10
   Compiling crossterm v0.14.1
   Compiling simplelog v0.7.4
   Compiling crossbeam-deque v0.7.2
   Compiling thiserror-impl v1.0.9
   Compiling crossbeam v0.7.3
   Compiling thiserror v1.0.9
   Compiling termimad v0.8.9
   Compiling broot v0.11.6
    Finished release [optimized] target(s) in 4m 56s
  Installing /home/vermaden/.cargo/bin/broot
   Installed package `broot v0.11.6` (executable `broot`)
warning: be sure to add `/home/vermaden/.cargo/bin` to your PATH to be able to run the installed binaries

% echo $?
0

Install

Now go to the ~/.cargo/bin directory and copy the broot binary to some place that is set in your ${PATH} variable.

Then start new terminal (updated ${PATH} variable) and type broot command.

% cp ~/.cargo/bin/broot ~/scripts
% rehash
% broot

You will be asked if automatic setup of the br function should tool place. I agreed with y answer.

broot-first-run.jpg

Here are things generated by this process.

% find ~/.config/broot
/home/vermaden/.config/broot
/home/vermaden/.config/broot/conf.toml
/home/vermaden/.config/broot/launcher
/home/vermaden/.config/broot/launcher/installed-v1
/home/vermaden/.config/broot/launcher/bash
/home/vermaden/.config/broot/launcher/bash/br

% find ~/.local/share/broot
/home/vermaden/.local/share/broot
/home/vermaden/.local/share/broot/launcher
/home/vermaden/.local/share/broot/launcher/fish
/home/vermaden/.local/share/broot/launcher/fish/1.fish
/home/vermaden/.local/share/broot/launcher/bash
/home/vermaden/.local/share/broot/launcher/bash/1

As I use ZSH shell it also updates my ~/.zshrc file.

% tail -3 ~/.zshrc

source /home/vermaden/.config/broot/launcher/bash/br

Finished. You now have broot installed and ready to use.

broot-filter-bhyve.jpg

UPDATE 1 – Now No Patches Are Needed

Thanks to the broot author any patches are now not needed.

It builds and works out of the box.

broot-update-fixed

UPDATE 2 – Its in Ports/Packages Now

The broot file manager is now available via usual FreeBSD Ports and packages which makes this guide pointless πŸ™‚

Its available as misc/broot port.

EOF

Β 

List Block Devices on FreeBSD lsblk(8) Style

When I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few:

  • sockstat
  • gstat
  • top -b -o res
  • top -m io -o total
  • usbconfig
  • rcorder
  • beadm/bectl
  • idprio/rtprio

… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world.

Example lsblk(8) output from Linux system:

$ lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE   MOUNTPOINT
sr0                           11:0    1  1024M  0 rom
sda                            8:0    0 931.5G  0 disk
|-sda1                         8:1    0   500M  0 part   /boot
`-sda2                         8:2    0   931G  0 part
  |-vg_local-lv_root (dm-0)  253:0    0    50G  0 lvm    /
  |-vg_local-lv_swap (dm-1)  253:1    0  17.7G  0 lvm    [SWAP]
  `-vg_local-lv_home (dm-2)  253:2    0   1.8T  0 lvm    /home
sdc                            8:32   0 232.9G  0 disk
`-sdc1                         8:33   0 232.9G  0 part
  `-md1                        9:1    0 232.9G  0 raid10 /data
sdd                            8:48   0 232.9G  0 disk
`-sdd1                         8:49   0 232.9G  0 part
  `-md1                        9:1    0 232.9G  0 raid10 /data

What FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs.

# camcontrol devlist
[Samsung SSD 860 EVO mSATA 1TB RVT41B6Q]  at scbus1 target 0 lun 0 (ada0,pass0)

% geom disk list
Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 1000204886016 (932G)
   Sectorsize: 512
   Mode: r1w1e2
   descr: Samsung SSD 860 EVO mSATA 1TB
   lunid: 5002538e402b4ddd
   ident: S41PNB0K303632D
   rotationrate: 0
   fwsectors: 63
   fwheads: 1

# gpart show
=>        40  1953525088  ada0  GPT  (932G)
          40      409600     1  efi  (200M)
      409640        1024     2  freebsd-boot  (512K)
      410664         984        - free -  (492K)
      411648  1953112064     3  freebsd-zfs  (931G)
  1953523712        1416        - free -  (708K)

They provide needed information in acceptable manner but only on systems with small amount of disks. What if you would like to display a summary of all system drives contents? This is where lsblk.sh comes handy. While lsblk(8) has many interesting features like --perms/--scsi/--inverse modes I focused to provide only the basic feature – to list the system block devices and their contents. As I have long and pleasing experience with writing shell scripts such as sysutils/beadm or sysutils/automount I though that writing lsblk.sh may be a good idea. I actually ‘open-sourced’ or should I say shared that project/idea in 2016 in this thread lsblk(8) Command for FreeBSD on FreeBSD Forums but lack of time really slowed that ‘side project’ development pace. I finally got back to it to finish it.

The lsblk.sh is generally small and simple shell script which tales less then 400 SLOC.

lsblk

Here is example output of lsblk.sh command from my single disk laptop.

% lsblk.sh
DEVICE         MAJ:MIN  SIZE TYPE                      LABEL MOUNT
ada0             0:5b  932G GPT                           - -
  ada0p1         0:64  200M efi                    efiboot0 [UNMOUNTED]
  ada0p2         0:65  512K freebsd-boot           gptboot0 -
  [FREE]         -:-   492K -                             - -
  ada0p3         0:66  931G freebsd-zfs                zfs0 [ZFS]
  [FREE]         -:-   708K -                             - -


Same output in graphical window.

lolcat

Below you will find an example lsblk.sh output from server with two system SSD drives (da0/da1) and two HDD data drives (da2/da3).

# lsblk.sh
DEVICE         MAJ:MIN SIZE TYPE                      LABEL MOUNT
da0              0:be  224G GPT                           - -
  da0p1          0:15a 200M efi                    efiboot0 [UNMOUNTED]
  da0p2          0:15b 512K freebsd-boot           gptboot0 -
  [FREE]         -:-   492K -                             - -
  da0p3          0:15c 2.0G freebsd-swap              swap0 [UNMOUNTED]
  da0p4          0:15d 221G freebsd-zfs                zfs0 [ZFS]
  [FREE]         -:-   580K -                             - -
da1              0:bf  224G GPT                           - -
  da1p1          0:16a 200M efi                    efiboot1 [UNMOUNTED]
  da1p2          0:16b 512K freebsd-boot           gptboot1 -
  [FREE]         -:-   492K -                             - -
  da1p3          0:16c 2.0G freebsd-swap              swap1 [UNMOUNTED]
  da1p4          0:16d 221G freebsd-zfs                zfs1 [ZFS]
  [FREE]         -:-   580K -                             - -
da2              0:c0   11T GPT                           - -
  da2p1          0:16e  11T freebsd-zfs                   - [ZFS]
  [FREE]         -:-   1.0G -                             - -
da3              0:c1   11T GPT                           - -
  da3p1          0:16f  11T freebsd-zfs                   - [ZFS]
  [FREE]         -:-   1.0G -                             - -

Below you will find other examples from other systems I have tested lsblk.sh on.

lsblk.examples

While lsblk.sh is not the fastest script on Earth (because of all the needed parsing) it does its job quite well. If you would like to install it in your system just type the command below:

# fetch -o /usr/local/bin/lsblk https://raw.githubusercontent.com/vermaden/scripts/master/lsblk.sh
# chmod +x /usr/local/bin/lsblk
# hash -r || rehash
# lsblk

If I got time which other original Linux lsblk(8) subcommand/option/argument is worth adding to the lsblk.sh script? πŸ™‚

Regards.

UPDATE 1 – Added USAGE/HELP Information

Just added some usage information that can be displayed by specifying one of these as argument:

  • h
  • -h
  • --h
  • help
  • -help
  • --help

IMHO writing man page for such simple utility is needless. I think I will create dedicated man page when lsblk.sh tool will grow in size and options to comparable with the Linux lsblk(8) equivalent. Here is how it looks.

# lsblk.sh --help
usage:

  BASIC USAGE INFORMATION
  =======================
  # lsblk.sh [DISK]

example(s):

  LIST ALL BLOCK DEVICES IN SYSTEM
  --------------------------------
  # lsblk.sh
  DEVICE         MAJ:MIN SIZE TYPE                      LABEL MOUNT
  ada0             0:5b  932G GPT                           - -
    ada0p1         0:64  200M efi                    efiboot0 [UNMOUNTED]
    ada0p2         0:65  512K freebsd-boot           gptboot0 -
    [FREE]         -:-   492K -                             - -
    ada0p3         0:66  931G freebsd-zfs                zfs0 [ZFS]

  LIST ONLY da1 BLOCK DEVICE
  --------------------------
  # lsblk.sh da1
  DEVICE         MAJ:MIN SIZE TYPE                      LABEL MOUNT
  da1              0:80  2.0G MBR                           - -
    da1s1          0:80  2.0G freebsd                       - -
      da1s1a       0:81  1.0G freebsd-ufs                root /
      da1s1b       0:82  1.0G freebsd-swap               swap SWAP

hint(s):

  DISPLAY ALL DISKS IN SYSTEM
  ---------------------------
  # sysctl kern.disks
  kern.disks: ada0 da0 da1

Regards.

UPDATE 2 – Code Reorganization and 75% Rewrite

… at least this is what git(1) tries to tell me after commit message.

% git commit (...)
[master 12fd4aa] Rework entire flow. Split code into functions. Add many useful comments. In other words its 2.0 version.
 1 file changed, 494 insertions(+), 505 deletions(-)
 rewrite lsblk.sh (75%)

After several productive hours new incarnation of lsblk.sh is now available.

It has similar SLOC but its now smaller by a quarter … while doing more and with better accuracy. Great example why “less is more.”

% wc scripts/lsblk.sh.OLD
     491    2201   19721 scripts/lsblk.sh.OLD

% wc scripts/lsblk.sh
     494    1871   15472 scripts/lsblk.sh

Things that does not have simple solution are described below.

One of them is ‘double’ label for FAT filesystems. We have both /dev/gpt/efiboot0 label and FAT label is named EFISYS. We have to choose something here. As not all FAT filesystems have label I have chosen the GPT label.

% glabel status | grep ada0p1
  gpt/efiboot0     N/A  ada0p1
msdosfs/EFISYS     N/A  ada0p1

I was also not able to cover FUSE mounts. When you mount – for example – the /dev/da0 device as NTFS (with ntfs-3g) or exFAT (with mount.exfat) there is no visible difference in mount(8) output.

% mount -t fusefs
/dev/fuse on /mnt/ntfs (fusefs)
/dev/fuse on /mnt/exfat (fusefs)

When I mount such filesystem by my daemon (like sysutils/automount) I keep track of what device have been mounted to which directory in the /var/run/automount.state file. Then when I get the detach event for /dev/da0 device I know what to u(n)mount … but when I only have /dev/fuse device its just not possible.

… or maybe YOU know any way of extracting information from /dev/fuse (or generally from FUSE) what device is mounted where?

Now little presentation after update.

Here are various non ZFS filesystems mounted.

% mount -t nozfs
devfs on /dev (devfs, local, multilabel)
linprocfs on /compat/linux/proc (linprocfs, local)
tmpfs on /compat/linux/dev/shm (tmpfs, local)
/dev/label/ASD on /mnt/tmp (msdosfs, local)
/dev/fuse on /mnt/ntfs (fusefs)
/dev/md0s1f on /mnt/ufs.other (ufs, local)
/dev/gpt/OTHER on /mnt/fat.other (msdosfs, local)
/dev/md0s1a on /mnt/ufs (ufs, local)

… and here is how now lsblk.sh displays them.

% lsblk.sh
DEVICE         MAJ:MIN SIZE TYPE                      LABEL MOUNT
ada0             0:56  932G GPT                           - -
  ada0p1         0:64  200M efi                gpt/efiboot0 -
  ada0p2         0:65  512K freebsd-boot       gpt/gptboot0 -
  [FREE]         -:-   492K -                             - -
  ada0p3         0:66  931G freebsd-zfs                   - [ZFS]
  [FREE]         -:-   708K -                             - -
md0              0:28f 1.0G MBR                           - -
  md0s1          0:294 512M freebsd                       - -
    md0s1a       0:29a 100M freebsd-ufs                root /mnt/ufs
    md0s1b       0:29b  32M freebsd-swap         label/swap SWAP
    md0s1e       0:29c  64M freebsd-ufs                   - -
    md0s1f       0:29d 316M freebsd-ufs                   - /mnt/ufs.other
  md0s2          0:296 256M ntfs                          - -
  md0s3          0:297 256M fat32               msdosfs/ONE -
md1              0:2a4 1.0G msdosfs                   LARGE 
md2              0:298 2.0G GPT                           - -
  md2p1          0:29f 2.0G ms-basic-data         gpt/OTHER /mnt/fat.other

I used some file based memory devices for this. Now by default lsblk.sh also displays memory disks contents.

% mdconfig.sh -l
md0     vnode    1024M  /home/vermaden/FILE     
md2     vnode    2048M  /home/vermaden/FILE.GPT 
md1     vnode    1024M  /home/vermaden/FILER    

Here is how it looks in the xterm(1) terminal.

lsblk.2.0

Regards.

UPDATE 3 – Added geli(8) Support

I thought that adding geli(8) support may be useful. The latest lsblk.sh now avoids code duplication for MOUNT and LABEL detection (moved into single unified function). Also added more comments for code readability and some minor fixes … and its again smaller πŸ™‚

% wc lsblk.sh.1.0
     491    2201   19721 lsblk.sh.1.0

% wc lsblk.sh.2.0
     493    1861   15415 lsblk.sh.2.0

% wc lsblk.sh
     488    1820   15332 lsblk.sh

About 40% (according to git commit was changed this time (191 insertions and 196 deletions).

# git commit (...)
[master ec9985a] Add geli(8) support. Avoid code duplication and move MOUNT/LABEL detection into function. More comments. Minor fixes.
 1 file changed, 191 insertions(+), 196 deletions(-)

Also forgot to mention that now lsblk.sh thanks to smart optimizations (like not doing things twice and aggregating grep(1) | awk(1) pipes into single awk(1) queries) runs 3 times faster then the initial version πŸ™‚

New output with geli(8) support below.

lsblk.2.1.geli.png

Regards.

UPDATE 4 – Added fuse(8) Support

As I wrote in the UPDATE 2 keeping track of what is mounted and where under fuse(8) is very hard as all mounted devices magically become /dev/fuse after mount is done.

After little research I found that this information (what really is mounted where by using fuse(8) interface under FreeBSD) is available after mounting procfs filesystem under /proc. You just need to cat cmdline entry for all PIDs of ntfs-3g. Its not perfect but the information at least is available.

# mount -t procfs proc /proc

# ps ax | grep ntfs-3g
45995  -  Is      0:00.00 ntfs-3g /dev/md1s2 /mnt/ntfs
59607  -  Is      0:00.00 ntfs-3g /dev/md3 /mnt/ntfs.another
83323  -  Is      0:00.00 ntfs-3g /dev/md3 /mnt/ntfs.another

# pgrep ntfs-3g
59607
83323
45995

% pgrep ntfs-3g | while read I; do cat /proc/$I/cmdline; echo; done
ntfs-3g/dev/md3/mnt/ntfs.another
ntfs-3g/dev/md3/mnt/ntfs.another
ntfs-3g/dev/md1s2/mnt/ntfs

This was the code prototype that worked for fuse(8) mountpoints detection.

    if [ -e /proc/0/status ]
    then
      FUSE_MOUNTS=$(
        while read PID
        do
          cat /proc/${PID}/cmdline
          echo
        done << ________EOF
          $( pgrep ntfs-3g )
________EOF
)
      FUSE_MOUNTS=$( echo "${FUSE_MOUNTS}" | sort -u )
      FUSE_MOUNTS=$( echo "${FUSE_MOUNTS}" | sed 's|ntfs-3g||g' )
      FUSE_CHECKS=$( echo "${FUSE_MOUNTS}" | grep /dev/${TARGET}/ )
      if [ "${FUSE_CHECKS}" != "" ]
      then
        MOUNT=$( echo "${FUSE_CHECKS}" | sed "s|/dev/${TARGET}||g" )
      fi
    fi
  fi

… and I have just realized that I found new (better) way of getting that information without mounting /proc filesystem – all you need to do is to display the ntfs-3g processes with their command line arguments, for example like that:

% ps -p $( pgrep ntfs-3g | tr '\n' ',' | sed '$s/.$//' ) -o command | sed 1d
ntfs-3g /dev/md1s2 /mnt/ntfs
ntfs-3g /dev/md3 /mnt/ntfs.another
ntfs-3g /dev/md3 /mnt/ntfs.another

So after I also thought that its only for NTFS (ntfs-3g(8) process) I also added exFAT support by searching for mount.exfat PIDs as well. The fuse(8) mount point detection works now for both NTFS and exFAT filesystems … and code to support it is even shorter.

  # TRY fuse(8) MOUNTS FROM PROCESSES
  if [ "${MOUNT_FOUND}" != "1" ]
  then
    FUSE_PIDS=$( pgrep mount.exfat ntfs-3g | tr '\n' ',' | sed '$s/.$//' )
    FUSE_MOUNTS=$( ps -p "${FUSE_PIDS}" -o command | sed 1d | sort -u )
    MOUNT=$( echo "${FUSE_MOUNTS}" |  grep "/dev/${TARGET} " | awk '{print $3}' )
  fi

I also changed how MAJOR and MINOR numbers are displayed – from HEX to DEC – as it is on Linux. The FreeBSD’s ls(1) from Base System displays these as HEX – for example you will get 0x2af value:

% ls -l /dev/md4
crw-rw----  1 root  operator  0x2af 2019.09.29 05:18 /dev/md4

But do the same with GNU equivalent by using gls(1) from FreeBSD Ports (from sysutils/coreutils package) and it shows MAJOR and MINOR in DEC values. The gls(1) is just ls(1) from the Linux world but as ls(1) name is already ‘taken’ by FreeBSD’s Base System tool the FreeBSD developers/maintainers add ‘g’ letter (for GNU) to distinguish them.

% gls -l /dev/md4
crw-rw---- 1 root 2, 175 2019-09-29 05:18 /dev/md4

… and they are also easier/faster to get with stat(1) tool.

  MAJ=$( stat -f "%Hr" /dev/${DEV} )
  MIN=$( stat -f "%Lr" /dev/${DEV} )

Latest lsblk.sh looks like that now.

lsblk.2.3.fuse.NTFS.exFAT

… that is why I did not (yet) added lsblk.sh to the FreeBSD Ports. Several new versions with important features span across just two days πŸ™‚

Regards.

UPDATE 5 – Another 69% Rewrite

After messing with gpart(8) more I found that using its -p flag which is a game changer. The difference is that with -p flag it displays names along partitions – its no longer needed to find the PREFIX and ‘create’ partition names.

Default gpart(8) output.

# gpart show md0
=>     63  2097089  md0  MBR  (1.0G)
       63  1048576    1  freebsd  (512M)
  1048639   524288    2  ntfs  (256M)
  1572927   524225    3  fat32  (256M)

Output of gpart(8) with -p flag.

# gpart show -p md0
=>     63  2097089    md0  MBR  (1.0G)
       63  1048576  md0s1  freebsd  (512M)
  1048639   524288  md0s2  ntfs  (256M)
  1572927   524225  md0s3  fat32  (256M)

That discovery implicated a quite large rewrite of lsblk.sh. The git commit estimates this as 69% code rewrite.

# git commit (...)
(...)
 1 file changed, 487 insertions(+), 501 deletions(-)
 rewrite lsblk.sh (69%)

The latest lsblk.sh has now these features:

  • Previous bugs fixed.
  • Detects exFAT labels.
  • Is now 20% faster.
  • Has less 10% SLOC.
  • Has less 15% of code.
  • Handles bsdlabel(8) on entire device properly.
  • Handles exFAT on entire device properly.

The difference in code is shown below.

# wc lsblk.sh
     487    1791   13705 lsblk.sh

# wc lsblk.sh.OLD
     544    1931   16170 lsblk.sh.OLD

Latest lsblk.sh looks as usual but I now use ‘-‘ instead of ‘[UNMOUNTED]‘ one.

lsblk.2.5.gpart.exfat

UPDATE 6 – New Updated and Fixed Version

The lsblk.sh has been updated to 3.4 version – also already updated in the FreeBSD Ports tree – in the sysutils/lsblk port.
The Changelog for this version is below:
  • Add sysctl -n kern.disks to disk listing.
  • Reset LABEL in __gpart_present function.
  • Fix behavior with [bootme] and [bootonce] flags from gpart(8).
  • Disable GPTID for label display.
  • Add -d|–disks option to list entire disks only.
Keep in mind that lsblk.sh uses diskinfo(8) which – to function properly – need you to be in the operator group. You can add yourself to that group like that:
# pw groupmod operator -m yourself

… or by editing the /etc/group file.

Here is example output.

lsblk.UPDATE6

UPDATE 7 – More Fixes

The lsblk.sh has been updated to 3.5 version – also already updated in the FreeBSD Ports tree – in the sysutils/lsblk port.
The Changelog for this version is below:
  • Remove control sequences and colors from output when listing disks.
  • Use diskinfo(8) only for md(4) disks as geom(4) does not support them.
  • Add new comments and rework some of the older ones.
  • Add additional checks for SIZE gathering and printing.
  • Properly print exFAT filesystem label when on entire device without partitions.
  • Fix display of NTFS-3G mountpoints.
  • Check automount(8) /var/run/automount.state for fusefs(5) filesystems.
Presence in the operator group is only needed for size for md(4)disks. All other disks size is gathered using geom(8)command now.
EOF

SMB/CIFS on FreeBSD

If you use FreeBSD/Illumos/Linux (or other UNIX/Unix-like system) there is big chance that you do not like – to say the least – the Windows world, but sometimes there is need to share some files with the Windows world. This is where Samba project comes handy. Today I would like to share minimalistic and simple Samba configuration and also a way to access SMB/CIFS shares from a FreeBSD machine.

samba_logo.png

On the naming side CIFS (Common Internet File System) is just particular version/dialect of the SMB (Server Message Block) protocol.

As usual I will use FreeBSD as a server. For the setup I used FreeBSD 12.0-RELEASE virtual machine image available from the project location:

There are several formats available – qcow2/raw/vhd/vmdk – but as I will be using VirtualBox I used the VMDK one.

The main FreeBSD configuration file on the server can be as small and simple as the one bellow.

# cat /etc/rc.conf
hostname="samba"
ifconfig_em0="inet 10.0.10.40/24"
defaultrouter="10.0.10.1"
sshd_enable="YES"

You of course do not need SSH to server SMB/CIFS shares with Samba.

Serve SMB/CIFS Share on FreeBSD with Samba

There are several versions of Samba available on FreeBSD, but if you do not have exact reason to use the older version then just go ahead with the latest one.

# pkg search samba
p5-Samba-LDAP-0.05_2           Manage a Samba PDC with an LDAP Backend
p5-Samba-SIDhelper-0.0.0_3     Create SIDs based on G/UIDs
samba-nsupdate-9.13.3_1        nsupdate utility with GSS-TSIG support
samba46-4.6.16_1               Free SMB/CIFS and AD/DC server and client for Unix
samba47-4.7.12                 Free SMB/CIFS and AD/DC server and client for Unix
samba48-4.8.7                  Free SMB/CIFS and AD/DC server and client for Unix

First You will need to add Samba package.

# pkg install samba48

Then we need to create configuration file for Samba. I will assume here that you would like to share two things as examples. The /data directory with write permissions only to my vermaden user and also my home directory /home/vermaden with read permissions for me and all people on my vermaden group. The so called public read is disabled entirely. Only after passing user and password the access will be possible to these shares. I also added several performance related options. Below is the /usr/local/etc/smb4.conf configuration file.

# cat /usr/local/etc/smb4.conf
[global]
workgroup          = workgroup
netbios name       = smb
server string      = samba
security           = user
max smbd processes = 3
encrypt passwords  = yes
socket options     = TCP_NODELAY IPTOS_LOWDELAY IPTOS_THROUGHPUT SO_KEEPALIVE SO_RCVBUF=65536 SO_SNDBUF=65536
aio read size      = 16384
aio write size     = 16384
strict locking     = no
strict sync        = no

# DISABLE PRINTING
load printers           = no
disable spoolss         = yes
show add printer wizard = no

[data]
  path       = /data
  public     = no
  writable   = yes
  browsable  = no
  write list = vermaden

[vermaden]
  path       = /home/vermaden
  public     = no
  writable   = no
  browsable  = no
  write list = @vermaden

We will also need vermaden user, let’s create one with pw(8) command.

First the vermaden group with GID of 1000. The -N flag just shows what will be done instead of doing actual changes to the system. Let’s try that and then execute the command without the -N flag to actually add the group.

# pw groupadd -n vermaden -g 1000 -N
vermaden:*:1000:
# pw groupadd -n vermaden -g 1000
# pw groupshow vermaden
vermaden:*:1000:

As we have the group its time to create vermaden user with UID of 1000. Like with group let’s first try with -N flag to check what will be made.

# pw useradd -n vermaden -c '' -u 1000 -g 1000 -m -N
vermaden:*:1000:1000::0:0::/home/vermaden:/bin/sh
# pw useradd -n vermaden -c '' -u 1000 -g 1000 -m
# pw usershow vermaden
vermaden:*:1000:1000::0:0::/home/vermaden:/bin/sh

Let’s verify our vermaden user again.

# id vermaden
uid=1000(vermaden) gid=1000(vermaden) groups=1000(vermaden)
# su - vermaden
By pressing "Scroll Lock" you can use the arrow keys to scroll backward
through the console output.  Press "Scroll Lock" again to turn it off.
Don't have a "Scroll Lock" key? The "Pause / Break" key acts alike.

Now let’s create password for this new vermaden user.

# passwd vermaden
Changing local password for vermaden
New Password:
Retype New Password:

Now we need to add the vermaden user with pdbedit command from the Samba package.

# which pdbedit
/usr/local/bin/pdbedit

# pkg which `which pdbedit`
/usr/local/bin/pdbedit was installed by package samba48-4.8.7

# pdbedit -a -u vermaden
new password:
retype new password:
Unix username:        vermaden
NT username:
Account Flags:        [U          ]
User SID:             S-1-5-21-1751207453-560213463-1759912891-1000
Primary Group SID:    S-1-5-21-1751207453-560213463-1759912891-513
Full Name:
Home Directory:       \\smb\vermaden
HomeDir Drive:
Logon Script:
Profile Path:         \\smb\vermaden\profile
Domain:               SMB
Account desc:
Workstations:
Munged dial:
Logon time:           0
Logoff time:          9223372036854775807 seconds since the Epoch
Kickoff time:         9223372036854775807 seconds since the Epoch
Password last set:    Fri, 21 Dec 2018 16:49:29 UTC
Password can change:  Fri, 21 Dec 2018 16:49:29 UTC
Password must change: never
Last bad password   : 0
Bad password count  : 0
Logon hours         : FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF

To list all users with the pdbedit command use the -L argument.

# pdbedit -L
vermaden:1000:

We now need to add Samba to the FreeBSD system services automatic startup.

# sysrc samba_server_enable=YES
samba_server_enable:  -> YES

# sysrc samba_server_enable
samba_server_enable: YES

# cat /etc/rc.conf
hostname="samba"
ifconfig_em0="inet 10.0.10.40/24"
defaultrouter="10.0.10.1"
sshd_enable="YES"
samba_server_enable="YES"

Now we can start the Samba service.

# service samba_server start
Performing sanity check on Samba configuration: OK
Starting nmbd.
Starting smbd.

Let’s check which Samba daemons listen on which ports.

# sockstat -l -4
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
root     smbd       599   33 tcp4   *:445                 *:*
root     smbd       599   34 tcp4   *:139                 *:*
root     nmbd       595   15 udp4   *:137                 *:*
root     nmbd       595   16 udp4   *:138                 *:*
(...)

Now let’s try to access the /data share from the Windows system.

Open explorer.exe on Windows machine and type //smb/data into location field and then type smb\vermaden as username.

bsd-share-01

You should be able to access the share now as shown below.

bsd-share-02

Let’s put some text into that test.txt file.

bsd-share-03.png

Let’s verify that it works on the FreeBSD side.

# cat /data/test.txt
Input from Windows.

So we are able to access/modify files from FreeBSD machine on the Windows world.

Access SMB/CIFS Share from FreeBSD

Let’s try the other way around.

By default there are several shares already served on Windows.

C:\>net share

Share name   Resource                        Remark

-------------------------------------------------------------------------------
C$           C:\                             Default share
IPC$                                         Remote IPC
ADMIN$       C:\Windows                      Remote Admin
Users        C:\Users
The command completed successfully.


C:\>

You can share a directory from Windows by using graphical interface as shown below.

win-share-01

… or by using CLI interface within cmd.exe interpreter with net commands.

win-share-02

win-share-03

win-share-04

The test share is now exported for vuser user with FULL access rights which means read/write in the Windows world.

Here are the same commands in text so you may copy/paste them as needed.

C:\Windows\system32>cd \

C:\>mkdir asd

C:\>net share test=C:\asd /grant:vuser,FULL
test was shared successfully.


C:\>net share

Share name   Resource                        Remark

-------------------------------------------------------------------------------
C$           C:\                             Default share
IPC$                                         Remote IPC
ADMIN$       C:\Windows                      Remote Admin
test         C:\asd
Users        C:\Users
The command completed successfully.


C:\>

Let’s try to mount it using the mount_smbfs command on FreeBSD system. The 10.0.10.4 address is the IP of the Windows machine.

# mount_smbfs -I 10.0.10.4 //vuser@vbox/test /mnt
Password:
#

# mount
/dev/gpt/rootfs on / (ufs, local, soft-updates)
devfs on /dev (devfs, local, multilabel)
//VUSER@VBOX/TEST on /mnt (smbfs)

It also works the other way.

After your job is done you may remove the test share also with net command as shown below.

win-share-05.png

… and also the same commands in text so you may copy/paste them as needed.

C:\>net share test /delete
test was deleted successfully.


C:\>net share

Share name   Resource                        Remark

-------------------------------------------------------------------------------
C$           C:\                             Default share
IPC$                                         Remote IPC
ADMIN$       C:\Windows                      Remote Admin
Users        C:\Users
The command completed successfully.


C:\>

This sentence concludes this article πŸ˜‰

UPDATE 1

The SMB/CIFS on FreeBSD article was featured in the BSD Now 279 – Future of ZFS episode.

Thanks for mentioning!

EOF

Β 

The Power to Serve – FreeBSD Power Management

This is the motto of the FreeBSD operating system – The Power to Serve – which also greatly fits for the topic of this article. Decade ago (yes time flies) I even made a wallpaper with this motto – still available on the DeviatArt page.

freebsd_the_power_to_serve_small.jpg

Time for FreeBSD article covering its power management features. It also applies to FreeBSD Desktop series but its not limited to it. Popular opinion seems to be that FreeBSD is so server oriented that it lacks any power management mechanisms. Nothing more far from the truth. While less important on the desktop (but will still lower your electricity bill) or servers it is desirable to properly configure power management on laptops to so they will have longer battery life and will run more quiet.

I write this as the FreeBSD Handbook does not cover all that information in the 11.13. Power and Resource Management chapter. The FreeBSD on Laptops article part 4. Power Management is from the ancient times of FreeBSD 10.1-RELEASE. There is some information on the FreeBSD Wiki page but parts of it are outdated.

FreeBSD offers many mechanisms in the power management department:

  • power off devices without attached driver
  • scale CPU frequency and power
  • supports CPU sleep states (C1/C1E/C2/C3/…)
  • enabling/disabling Turbo Mode available in most CPUs
  • per USB device power management options
  • SATA/AHCI channels/controllers power management
  • suspend/resume support (along with using laptop lid for it)
  • support for vendor specific tools that help to measure power management
  • tools and ACPI support for fan speed control
  • tools and ACPI support for setting screen brightness
  • battery capacity status and running time estimation
  • network interfaces power saving options
  • support for AMD PowerNow!
  • support for Intel (Enhanced) SpeedStep
  • support for Intel Speed Shift
  • support for AMD Turbo Core
  • support for Intel Turbo Boost

One word about different files for the settings in the FreeBSD system:

  • /etc/rc.conf – does not require reboot just daemons reloading
  • /etc/sysctl.conf – does not require reboot – you can set them at runtime
  • /boot/loader.conf – these settings REQUIRE reboot

Here is the Table of Contents (non-clickable) for the article.

  • Information
    • Battery
    • Battery Wear
    • CPU
    • lscpu(1)
    • dmesg(8)
  • CPU Frequency Scaling
    • powerd(8)
    • powerdxx(8)
    • C-States
    • CPU Turbo Mode
  • USB Devices
  • SATA/AHCI Power Management
  • Devices without Driver
    • Nvidia Optimus
  • Suspend and Resume
  • Network Interfaces
  • Vendor Tools
  • DTrace
  • Other
    • ZFS
    • Applications
  • Hardware
  • UPDATE 1 – Graphics Card Power Saving
  • UPDATE 2 – AMD CPU Temperatures
  • UPDATE 3 – Suspend/Resume Tips
  • UPDATE 4 – Intel Speed Shift

Information

Let’s start by describing where to get needed information about current CPU speed, used C-states, current power management modes for USB devices, battery capacity and remaining time, etc.

Battery

To get battery information you can use the acpiconf(8) tool. This is the acpiconf(8) output for my main battery (in the ThinkPad T420s laptop) with AC power attached.

% acpiconf -i 0
Design capacity:        44000 mWh
Last full capacity:     37930 mWh
Technology:             secondary (rechargeable)
Design voltage:         11100 mV
Capacity (warn):        1896 mWh
Capacity (low):         200 mWh
Low/warn granularity:   1 mWh
Warn/full granularity:  1 mWh
Model number:           45N1037
Serial number:          28608
Type:                   LION
OEM info:               SANYO
State:                  high
Remaining capacity:     100%
Remaining time:         unknown
Present rate:           0 mW
Present voltage:        12495 mV

… and with AC power detached.

% acpiconf -i 0
Design capacity:        44000 mWh
Last full capacity:     37930 mWh
Technology:             secondary (rechargeable)
Design voltage:         11100 mV
Capacity (warn):        1896 mWh
Capacity (low):         200 mWh
Low/warn granularity:   1 mWh
Warn/full granularity:  1 mWh
Model number:           45N1037
Serial number:          28608
Type:                   LION
OEM info:               SANYO
State:                  high
Remaining capacity:     100%
Remaining time:         2:31
Present rate:           0 mW
Present voltage:        12492 mV

Now as AC power is detached from the laptop the Remaining time: field will show you remaining time estimation for this single battery shows as 2:31 here (two hours and thirty one minutes).

Below is acpiconf(8) output for my secondary battery (in ThinkPad T420s ultrabay instead of DVD drive).

% acpiconf -i 1
Design capacity:        31320 mWh
Last full capacity:     24510 mWh
Technology:             secondary (rechargeable)
Design voltage:         10800 mV
Capacity (warn):        1225 mWh
Capacity (low):         200 mWh
Low/warn granularity:   1 mWh
Warn/full granularity:  1 mWh
Model number:           45N1041
Serial number:            260
Type:                   LiP
OEM info:               SONY
State:                  high
Remaining capacity:     100%
Remaining time:         unknown
Present rate:           0 mW
Present voltage:        12082 mV

… and with AC power detached.

% acpiconf -i 1
Design capacity:        31320 mWh
Last full capacity:     24510 mWh
Technology:             secondary (rechargeable)
Design voltage:         10800 mV
Capacity (warn):        1225 mWh
Capacity (low):         200 mWh
Low/warn granularity:   1 mWh
Warn/full granularity:  1 mWh
Model number:           45N1041
Serial number:            260
Type:                   LiP
OEM info:               SONY
State:                  discharging
Remaining capacity:     98%
Remaining time:         1:36
Present rate:           14986 mW
Present voltage:        11810 mV

With AC power detached it shows the Remaining time: as 1:36 for the secondary battery.

So its total 4:07 time on battery estimated. The same time in minutes (247) will be shown in the sysctl(8) value named hw.acpi.battery.time as shown below.

% sysctl hw.acpi.battery.time
hw.acpi.battery.time: 247

You can also get more ‘complete’ battery information with below sysctl(8) values under hw.acpi.battery MIB.

% sysctl hw.acpi.battery
hw.acpi.battery.info_expire: 5
hw.acpi.battery.units: 2
hw.acpi.battery.state: 1
hw.acpi.battery.time: 247
hw.acpi.battery.life: 99

The hw.acpi.battery.time will show you ‘-1‘ value if you have AC power attached.

% sysctl hw.acpi.battery
hw.acpi.battery.info_expire: 5
hw.acpi.battery.units: 2
hw.acpi.battery.state: 0
hw.acpi.battery.time: -1
hw.acpi.battery.life: 100

Battery Wear

As time passes by batteries lose their ‘design’ capacity. After 1-2 years such battery can have only 70% or less of its original efficiency.

All the information needed to check that is provided by the acpiconf(8) command with Design capacity: and Last full capacity: values. I have made a battery-capacity.sh script that will tell you what the current battery efficiency is. Here is how it looks in action.

% battery-capacity.sh 0
Battery '0' model '45N1037' has efficiency: 86%

% battery-capacity.sh 1
Battery '1' model '45N1041' has efficiency: 78%

Here is the battery-capacity.sh script itself.

#! /bin/sh

if [ ${#} -ne 1 ]
then
  echo "usage: ${0##*/} BATTERY"
  exit
fi

if acpiconf -i ${1} 1> /dev/null 2> /dev/null
then
  DATA=$( acpiconf -i ${1} )
  MAX=$( echo "${DATA}" | grep '^Design\ capacity:'     | awk -F ':' '{print $2}' | tr -c -d '0-9' )
  NOW=$( echo "${DATA}" | grep '^Last\ full\ capacity:' | awk -F ':' '{print $2}' | tr -c -d '0-9' )
  MOD=$( echo "${DATA}" | grep '^Model\ number:'        | awk -F ':' '{print $2}' | awk '{print $1}' )
  echo -n "Battery '${1}' model '${MOD}' has efficiency: "
  printf '%1.0f%%\n' $( bc -l -e "scale = 2; ${NOW} / ${MAX} * 100" -e quit )
else
  echo "NOPE: Battery '${1}' does not exists on this system."
  echo "INFO: Most systems has only '0' or '1' batteries."
  exit 1
fi

CPU

To get information about current CPU’s you will have to use dev.cpu MIB or dev.cpu.0 for the first physical CPU core.

% sysctl dev.cpu.0
dev.cpu.0.cx_method: C1/hlt C2/io
dev.cpu.0.cx_usage_counters: 412905 0
dev.cpu.0.cx_usage: 100.00% 0.00% last 290us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/1 C2/3/104
dev.cpu.0.freq_levels: 2501/35000 2500/35000 2200/29755 2000/26426 1800/23233 1600/20164 1400/17226 1200/14408 1000/11713 800/9140
dev.cpu.0.freq: 800
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=none _UID=0
dev.cpu.0.%location: handle=\_PR_.CPU0
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU

If you load the coretemp(4) kernel module with kldload(8) command you will get additional temperature information.

Below is same sysctl(8) dev.cpu.0 MIB with coretemp(4) kernel module loaded.

% sysctl dev.cpu.0
dev.cpu.0.temperature: 49.0C
dev.cpu.0.coretemp.throttle_log: 0
dev.cpu.0.coretemp.tjmax: 100.0C
dev.cpu.0.coretemp.resolution: 1
dev.cpu.0.coretemp.delta: 51
dev.cpu.0.cx_method: C1/hlt C2/io
dev.cpu.0.cx_usage_counters: 16549 0
dev.cpu.0.cx_usage: 100.00% 0.00% last 1489us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/1 C2/3/104
dev.cpu.0.freq_levels: 2501/35000 2500/35000 2200/29755 2000/26426 1800/23233 1600/20164 1400/17226 1200/14408 1000/11713 800/9140
dev.cpu.0.freq: 800
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=none _UID=0
dev.cpu.0.%location: handle=\_PR_.CPU0
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU

Let me describe some most useful ones.

CPU core temperature.
dev.cpu.0.temperature: 49.0C

CPU supported C-states (C1 and C2 for this CPU).
dev.cpu.0.cx_supported: C1/1/1 C2/3/104

CPU statistics for C-states usage (only C1 state been used).
dev.cpu.0.cx_usage_counters: 16549 0
dev.cpu.0.cx_usage: 100.00% 0.00% last 1489us

CPU maximum (most deep) C state enabled.
dev.cpu.0.cx_lowest: C1

CPU supported frequency levels with power usage after the ‘/‘ character. The 2500/35000 can be read as 2.5 GHz frequency with 35 W power usage and 2501 is the Turbo Mode. The lowest is 800 MHz with about 9 W usage.
dev.cpu.0.freq_levels: 2501/35000 2500/35000 2200/29755 2000/26426 1800/23233 1600/20164 1400/17226 1200/14408 1000/11713 800/9140

CPU current frequency (will vary when You use powerd(8) or powerdxx(8) daemon).
dev.cpu.0.freq: 800

The hw.acpi.thermal.tz0.temperature MIB will also show you current thermal zone temperature.

% sysctl hw.acpi.thermal.tz0.temperature
hw.acpi.thermal.tz0.temperature: 49.1C

To check how many cores you have use these commands.

% grep FreeBSD/SMP /var/run/dmesg.boot
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)

% sysctl kern.smp.cpus
kern.smp.cpus: 2

If my description does not feel useful then you should also check the -d flag for sysctl(8) command as shown below.

% sysctl -d dev.cpu.0.freq
dev.cpu.0.freq: Current CPU frequency

lscpu(1)

There is also third party tool called lscpu(8) that will describe your CPU features and model. You will have to add it from packages.

# pkg install lscpu

To make lscpu(8) work the cpuctl(4) kernel module is needed.

Here is how it looks for my dual core CPU.

# kldload cpuctl
# lscpu
Architecture:            amd64
Byte Order:              Little Endian
Total CPU(s):            2
Thread(s) per core:      2
Core(s) per socket:      2
Socket(s):               0
Vendor:                  GenuineIntel
CPU family:              6
Model:                   42
Model name:              Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
Stepping:                7
L1d cache:               32K
L1i cache:               32K
L2 cache:                256K
L3 cache:                3M
Flags:                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 cflsh ds acpi mmx fxsr sse sse2 ss htt tm pbe sse3 pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline aes xsave osxsave avx syscall nx rdtscp lm lahf_lm

dmesg(8)

Also dmesg(8) command (or /var/run/dmesg.boot file after longer uptime) covers your CPU model and features information.

% grep CPU /var/run/dmesg.boot
CPU: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz (2491.97-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0:  on acpi0
coretemp0:  on cpu0

CPU Frequency Scaling

For CPU scaling feature you may use the powerd(8) daemon available in the FreeBSD base system or powerdxx(8) from the FreeBSD Ports or packages. The powerdxx(8) daemon aims to better scale multicore systems and not turning all cores to high state when there is moderate load on the system but some people may prefer that approach to have full power available when they do anything and to save power when they do nothing. Thus powerd(8) is not better then powerdxx(8) or vice versa. They are just different so that gives you more options for your needs.

No matter which one you will choose it has to be configured in the /etc/rc.conf file.

powerd(8)

Here are the options for powerd(8) daemon.

powerd_enable=YES
powerd_flags="-n adaptive -a hiadaptive -b adaptive -m 800 -M 1600"

The -n option of for the unknown state – if for some reason the powerd(8) will not be able to determine if you are running on the AC power or battery. The -a is for AC power and -b for running on the battery. The adaptive setting is less ‘aggressive’ so its more battery time friendly. The hiadaptive is more aggressive this its preferred when you are running on AC power. The -m option sets minimum CPU frequency to be used and -M the maximum. Both in MHz units. Check powerd(8) man page for more details.

powerdxx(8)

First you will need to install it.

# pkg install powerdxx

Then its options are identical as those of powerd(8) daemon.

powerdxx_enable=YES
powerdxx_flags="-n adaptive -a hiadaptive -b adaptive -m 800 -M 1600"

Check the powerdxx(8) section above for the flags/parameters description.

Decade ago CPU frequency scaling on FreeBSD was not that ‘easy’ as it is now, you may check my old HOWTO: FreeBSD CPU Scaling and Power Saving in that topic from 2008.

C-States

The C-states can be configured in the /etc/rc.conf file with these options.

  • performance_cx_lowest
  • economy_cx_lowest

The economy_cx_lowest parameter is for running on battery and performance_cx_lowest parameter is for running on AC power. Both are set using the /etc/rc.d/power_profile script used by rc(8) subsystem. It sets the hw.acpi.cpu.cx_lowest parameter which sets/controls all dev.cpu.*.cx_lowest values. You can also track the changes in the /var/log/messages file when you attach/detach the AC power.

% tail -f /var/log/messages
Nov 28 13:14:42 t420s power_profile[48231]: changed to 'economy'
Nov 28 13:14:46 t420s power_profile[56835]: changed to 'performance'

Usually I jest use these values.

performance_cx_lowest=C1
economy_cx_lowest=Cmax

These settings above are generally sufficient for most systems. To check which C-states your CPU supports get the value of dev.cpu.0.cx_supported MIB.

% sysctl dev.cpu.0.cx_supported
dev.cpu.0.cx_supported: C1/1/1 C2/3/104

My CPU supports only C1 and C2 but yours may support more. I remember once when using some old Core 2 Duo laptop that the C2 state had quite ‘noticeable’ delay when getting back from C1 (running) state to C2 (sleep) state so following setting is needed. You do not use the performance_cx_lowest and economy_cx_lowest parameters. You set the first core to C1 and all other cores to C2. This way even on battery you have fully responsive system and all other cores may sleep and save energy.

For example if You would have 4 cores and your maximum (deepest) supported C-state would be C3, then you would put these into the /etc/sysctl.conf file.

% grep cx_lowest /etc/sysctl.conf
dev.cpu.0.cx_lowest=C1
dev.cpu.1.cx_lowest=C3
dev.cpu.2.cx_lowest=C3
dev.cpu.3.cx_lowest=C3

CPU Turbo Mode

There are two ways to enable Turbo mode. One way is to set powerd(8) or powerdxx(8) daemon with maximum frequency set above nominal CPU speed. For example if you have CPU described as dual-core 2.3 GHz then set the maximum speed with -M flag to 4000 for example (which would mean 4GHz). If you do not use CPU frequency scaling daemon then you will use dev.cpu.0.freq parameter with highest (first) value from the dev.cpu.0.freq_levels MIB.

Supported CPU frequency levels on my system.

% sysctl dev.cpu.0.freq_levels 
dev.cpu.0.freq_levels: 2501/35000 2500/35000 2200/29755 2000/26426 1800/23233 1600/20164 1400/17226 1200/14408 1000/11713 800/9140

The highest value (left) is 2501/35000 so I need to set dev.cpu.0.freq parameter with this value to use Turbo Mode. You need to only use the ‘frequency’ value part because if you paste it with power requirements description it will fail.

# sysctl dev.cpu.0.freq=2501/35000
sysctl: invalid integer '2501/35000'

This is how it should be used.

# sysctl dev.cpu.0.freq=2501
dev.cpu.0.freq: 800 -> 2501

USB Devices

To list attached USB devices use the usbconfig(8) tool.

% usbconfig
ugen1.1:  at usbus1, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen2.1:  at usbus2, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.1:  at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen2.2:  at usbus2, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.2:  at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)
ugen0.3:  at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
ugen2.3:  at usbus2, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA)

You will see that pwr parameter (short for power) will show you current power setting which can be:

  • ON
  • OFF
  • SAVE

To set new USB power option for the ugen1.1 device also use the usbconfig(8) tool with the power_save parameter in the following way.

# usbconfig -u 1 -a 1 power_save

The USB power management does not have dedicated config file on FreeBSD so we will put them into universal /etc/rc.local file which is being run at the end of the start-up process managed by the rc(8) subsystem. Here is the added content with exception for the ‘Lenovo USB Receiver‘ which is my wireless mouse.

% grep -A 10 POWER /etc/rc.local
# POWER SAVE USB DEVICES
usbconfig \
  | grep -v 'Lenovo USB Receiver' \
  | awk '{print $1}' \
  | sed 's|ugen||'g \
  | tr -d : \
  | awk -F '.' '{print $1 " " $2 }' \
  | while read U A
    do
      usbconfig -u ${U} -a ${A} power_save 2> /dev/null
    done

It’s good idea to NOT save power for mouse or tracked devices because you will probably find it annoying to have to wait about a second each time you would like to use it. I use a for loop to set power saving for all USB devices except wireless USB mouse (identified as ‘Lenovo USB Receiver‘ device).

SATA/AHCI Power Management

FreeBSD offers AHCI channels power management via acpich(4) driver. These power management settings can be set at boot using the hint.ahcich.*.pm_level parameter in the /boot/loader.conf file. I use configuration up to 8 channels while I only have three.

% grep ahcich /var/run/dmesg.boot
ahcich0:  at channel 0 on ahci0
ahcich1:  at channel 1 on ahci0
ahcich4:  at channel 4 on ahci0
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0

That is because settings for non-existent devices are harmless and will not display any error messages but you will not have to use different settings for various systems which saves time. This is the hint.ahcich.*.pm_level description from the ahci(4) man page.

  hint.ahcich.X.pm_level

    controls SATA interface Power Management for the specified channel,
    allowing some power to be saved at the cost of additional command latency.

    Some controllers, such as ICH8, do not implement modes 2 and 3 with NCQ
    used. Because of artificial entering latency, performance degradation in
    modes 4 and 5 is much smaller then in modes 2 and 3.

Possible power management options are:

  • 0 – interface Power Management is disabled (default)
  • 1 – device is allowed to initiate PM state change, host is passive
  • 2 – host initiates PARTIAL PM state transition every time port becomes idle
  • 3 – host initiates SLUMBER PM state transition every time port becomes idle
  • 4 – driver initiates PARTIAL PM state transition 1ms after port becomes idle
  • 5 – driver initiates SLUMBER PM state transition 125ms after port becomes idle

Here are my setting from the /boot/loader.conf file.

# AHCI POWER MANAGEMENT FOR EVERY USED CHANNEL (ahcich 0-7)
  hint.ahcich.0.pm_level=5
  hint.ahcich.1.pm_level=5
  hint.ahcich.2.pm_level=5
  hint.ahcich.3.pm_level=5
  hint.ahcich.4.pm_level=5
  hint.ahcich.5.pm_level=5
  hint.ahcich.6.pm_level=5
  hint.ahcich.7.pm_level=5

Devices without Driver

FreeBSD has power saving option to not power devices that does not have attached driver. Its called hw.pci.do_power_nodriver and you can set it in the /boot/loader.conf file. Here is its description from then pci(4) man page.

  hw.pci.do_power_nodriver (Defaults to 0)

    Place devices into a low power state (D3) when
    a suitable device driver is not found.

It can be set to one of the following values:

  • 0 – All devices are left fully powered (defaults).
  • 1 – Like ‘2‘ except that storage controllers are also not powered down.
  • 2 – Powers down most devices (display/memory/peripherals not powered down).
  • 3 – Powers down all PCI devices without a device driver.

Here is my setting from the /boot/loader.conf file.

# POWER OFF DEVICES WITHOUT ATTACHED DRIVER
  hw.pci.do_power_nodriver=3

The pciconf(8) utility will show you what devices are in your system and which driver is attached to it. If no driver is attached you will see none*@ for such devices, as none0@ below. You can also check man page for most drivers like em(4) man page for em0 device or xhci(4) page for xhci0 device.

% pciconf -l
hostb0@pci0:0:0:0:      class=0x060000 card=0x21d217aa chip=0x01048086 rev=0x09 hdr=0x00
vgapci0@pci0:0:2:0:     class=0x030000 card=0x21d217aa chip=0x01268086 rev=0x09 hdr=0x00
none0@pci0:0:22:0:      class=0x078000 card=0x21d217aa chip=0x1c3a8086 rev=0x04 hdr=0x00
em0@pci0:0:25:0:        class=0x020000 card=0x21ce17aa chip=0x15028086 rev=0x04 hdr=0x00
ehci0@pci0:0:26:0:      class=0x0c0320 card=0x21d217aa chip=0x1c2d8086 rev=0x04 hdr=0x00
hdac0@pci0:0:27:0:      class=0x040300 card=0x21d217aa chip=0x1c208086 rev=0x04 hdr=0x00
pcib1@pci0:0:28:0:      class=0x060400 card=0x21d217aa chip=0x1c108086 rev=0xb4 hdr=0x01
pcib2@pci0:0:28:1:      class=0x060400 card=0x21d217aa chip=0x1c128086 rev=0xb4 hdr=0x01
pcib3@pci0:0:28:3:      class=0x060400 card=0x21d217aa chip=0x1c168086 rev=0xb4 hdr=0x01
pcib4@pci0:0:28:4:      class=0x060400 card=0x21d217aa chip=0x1c188086 rev=0xb4 hdr=0x01
ehci1@pci0:0:29:0:      class=0x0c0320 card=0x21d217aa chip=0x1c268086 rev=0x04 hdr=0x00
isab0@pci0:0:31:0:      class=0x060100 card=0x21d217aa chip=0x1c4f8086 rev=0x04 hdr=0x00
ahci0@pci0:0:31:2:      class=0x010601 card=0x21d217aa chip=0x1c038086 rev=0x04 hdr=0x00
ichsmb0@pci0:0:31:3:    class=0x0c0500 card=0x21d217aa chip=0x1c228086 rev=0x04 hdr=0x00
iwn0@pci0:3:0:0:        class=0x028000 card=0x11118086 chip=0x42388086 rev=0x3e hdr=0x00
sdhci_pci0@pci0:5:0:0:  class=0x088000 card=0x21d217aa chip=0xe8221180 rev=0x07 hdr=0x00
xhci0@pci0:13:0:0:      class=0x0c0330 card=0x01941033 chip=0x01941033 rev=0x04 hdr=0x00

You can also use -v flag to get more detailed information.

% pciconf -l -v
(...)
xhci0@pci0:13:0:0:      class=0x0c0330 card=0x01941033 chip=0x01941033 rev=0x04 hdr=0x00
    vendor     = 'NEC Corporation'
    device     = 'uPD720200 USB 3.0 Host Controller'
    class      = serial bus
    subclass   = USB

Nvidia Optimus

If for some reason your BIOS/UEFI firmware does not allow you to disable Nvidia discrete graphics card you may use this script to disable it so it will not drain power from your system. It requires the acpi_call(4) kernel module which is provided by the acpi_call package.

# mkdir /root/bin
# cd /root/bin
# fetch https://people.freebsd.org/~xmj/turn_off_gpu.sh
# pkg install acpi_call
# kldload acpi_call
# chmod +x /root/bin/turn_off_gpu.sh
# /root/bin/turn_off_gpu.sh

You may add it to the /etc/rc.local file after the USB power saving options with this entry.

# DISABLE NVIDIA CARD
  /root/bin/turn_off_gpu.sh

It will store the working ACPI call in the /root/.gpu_method file and execute it each next time.

Suspend and Resume

The biggest enemies of suspend/resume mechanism are bugs in your BIOS/UEFI firmware for your hardware. Sometimes disabling Bluetooth helps – that is the option for ThinkPad T420s for example. To check which suspend modes are supported on your system check the hw.acpi.supported_sleep_state MIB from sysctl(8) subsystem.

% sysctl hw.acpi.supported_sleep_state
hw.acpi.supported_sleep_state: S3 S4 S5

To enter ACPI S3 sleep state (suspend) you can use acpiconf(8) tool or zzz(8) tool.

# zzz

… or with acpiconf(8) tool.

# acpiconf -s 3

Its exactly the same as stated in the zzz(8) man page.

You can also set sysctl(8) value that everytime you close your laptop lid your system will go to sleep. To achieve that put hw.acpi.lid_switch_state=S3 into the /etc/sysctl.conf file. No matter if you put you hardware to sleep by command or by closing the lid your laptop will resume after opening the lid. Of course if you haven’t closed the lid after the zzz(8) command you will either have to close and open the lid or push the power button to resume. Of course you may also suspend/resume desktops or even your backup server if it has its purpose. It’s not limited to laptops only.

There are also dedicated kernel modules for various vendor ACPI subsystems. Here they are:

  • /boot/kernel/acpi_asus_wmi.ko
  • /boot/kernel/acpi_asus.ko
  • /boot/kernel/acpi_dock.ko
  • /boot/kernel/acpi_fujitsu.ko
  • /boot/kernel/acpi_hp.ko
  • /boot/kernel/acpi_ibm.ko
  • /boot/kernel/acpi_panasonic.ko
  • /boot/kernel/acpi_sony.ko
  • /boot/kernel/acpi_toshiba.ko
  • /boot/kernel/acpi_video.ko
  • /boot/kernel/acpi_wmi.ko

For example if you have IBM/Lenovo ThinkPad the you will use the acpi_ibm.ko kernel module.

# kldload acpi_ibm

After loading each module you will get new sysctl(8) values for your use. For example related to fan speed, keyboard backlit or screen brightness. Below is new dev.acpi_ibm section in sysctl(8) after loading the acpi_ibm(4) kernel module.

% sysctl dev.acpi_ibm
dev.acpi_ibm.0.handlerevents: NONE
dev.acpi_ibm.0.mic_led: 0
dev.acpi_ibm.0.fan: 0
dev.acpi_ibm.0.fan_level: 0
dev.acpi_ibm.0.fan_speed: 0
dev.acpi_ibm.0.wlan: 1
dev.acpi_ibm.0.bluetooth: 0
dev.acpi_ibm.0.thinklight: 0
dev.acpi_ibm.0.mute: 0
dev.acpi_ibm.0.volume: 0
dev.acpi_ibm.0.lcd_brightness: 0
dev.acpi_ibm.0.hotkey: 1425
dev.acpi_ibm.0.eventmask: 134217727
dev.acpi_ibm.0.events: 1
dev.acpi_ibm.0.availmask: 134217727
dev.acpi_ibm.0.initialmask: 2060
dev.acpi_ibm.0.%parent: acpi0
dev.acpi_ibm.0.%pnpinfo: _HID=LEN0068 _UID=0
dev.acpi_ibm.0.%location: handle=\_SB_.PCI0.LPC_.EC__.HKEY
dev.acpi_ibm.0.%driver: acpi_ibm
dev.acpi_ibm.0.%desc: IBM ThinkPad ACPI Extras
dev.acpi_ibm.%parent: 

Here are descriptions of more interesting ones.

This one will turn the LED light on the Microphone mute button.
dev.acpi_ibm.0.mic_led

Select if you want to manage CPU fan (0) or leave it to the manufacturer defaults (1).
dev.acpi_ibm.0.fan

If CPU fan is enabled, set its speed.
dev.acpi_ibm.0.fan_level

This one will tell you how fast the CPU fan is spinning (in RPMs).
dev.acpi_ibm.0.fan_speed

Enable/disable WiFi (if its enabled in BIOS).
dev.acpi_ibm.0.wlan

Enable/disable Bluetooth (if its enabled in BIOS).
dev.acpi_ibm.0.bluetooth

Enable/disable ThinkLight.
dev.acpi_ibm.0.thinklight

Mute/unmute speakers.
dev.acpi_ibm.0.mute

Speakers volume.
dev.acpi_ibm.0.volume

Screen brightness.
dev.acpi_ibm.0.lcd_brightness

For most of the cases its not needed to use them as you will probably just use the vendor defined keyboard shortcuts (probably with Fn key) or vendor specific dedicated buttons. Sometimes you want to create/use your own setup or need custom keyboard shortcuts, or you want to control the fan speed depending on the CPU temperature other way then your vendor predefined it. This is when these dedicated ACPI kernel modules are most useful.

For example I recently thought that my CPU fan seems to be little louder then it should be so I created custom cron(8) based acpi-thinkpad-fan.sh script to use lower fan speeds or even lower quieter speeds when CPU temperature is low enough.

I will post it here. Maybe you will find it useful for your purposes. To describe it shortly it disables the fan when CPU temperature is below 50 (C) degrees, it sets it to level ‘1’ if its between 50 (C) and 60 (C) degrees and sets it to level ‘3’ when temperature reaches more then 60 (C) degrees.

#! /bin/sh

if ! kldstat | grep -q acpi_ibm.ko
then
  doas kldload acpi_ibm
fi

doas sysctl dev.acpi_ibm.0.fan=0 1> /dev/null 

TEMP=$( sysctl -n hw.acpi.thermal.tz0.temperature | awk -F'.' '{print $1}' )

if [ ${TEMP} -lt 50 ]
then
  doas sysctl dev.acpi_ibm.0.fan_level=0 1> /dev/null
  exit 0
fi

if [ ${TEMP} -lt 60 ]
then
  doas sysctl dev.acpi_ibm.0.fan_level=1 1> /dev/null
  exit 0
fi

if [ ${TEMP} -ge 60 ]
then
  doas sysctl dev.acpi_ibm.0.fan_level=3 1> /dev/null
  exit 0
fi

… and here is its crontab(5) entry:

% crontab -l
# ACPI/IBM/FAN
* * * * * ~/scripts/acpi-thinkpad-fan.sh

Network Interfaces

There is also ifconfig(8) option to save power if a driver supports such feature, its called powersave and its used like that.

# ifconfig wlan0 powersave

I use it in my network.sh network management script described broadly in the FreeBSD Network Management with network.sh article.

Vendor Tools

There are also vendor tools available on FreeBSD like powermon(8) for example. Remember that it requires cpuctl(4) kernel module to work.

# pkg install powermon
# kldload cpuctl
# powermon
                  Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
                      (Arch: Sandy Bridge, Limit: 44W)



   5.11W [=======>                                                           ]



 Package:           Uncore:             x86 Cores:          GPU:
 Current: 5.11W     Current: 3.17W      Current: 1.73W      Current: 0.21W
 Total: 98.33J      Total: 60.86J       Total: 33.49J       Total: 3.98J

DTrace

The dynamic tracing framework that like ZFS found its way from Solaris/Illumos to FreeBSD may be also useful weapon in the battle for more time on your battery.

First add the dtrace-toolkit package.

# pkg install dtrace-toolkit

Your system stops saving energy or wakes CPU up because something needs to be run/done. To check what is run on your system you mostly run ps(1) or top(1) utilities but that will not show you what exactly is being started or how often something is being run. This is where DTrace comes with help.

We will use the /usr/share/dtrace/toolkit/execsnoop script from the dtrace-toolkit package. It will print EVERY COMMAND that is being run with all its arguments.It will remain silent when no commands are run, be advised.

Here is example output for my dzen2 toolbar update.

# /usr/local/share/dtrace-toolkit/execsnoop 
  UID    PID   PPID ARGS
 1000  97748  97509 /usr/local/bin/zsh -c ~/scripts/dzen2-update.sh > ~/.dzen2-fifo
 1000  97748      1 /bin/sh /home/vermaden/scripts/dzen2-update.sh
 1000  99157  97748 sysctl -n kern.smp.cpus
 1000    311  97748 ps ax -o %cpu,rss,command -c
 1000   3118   1521 awk -v SMP=200 /\ idle$/ {printf("%.1f%%",SMP-$1)}
 1000   4462  97748 date +%Y/%m/%d/%a/%H:%M
 1000   4801  97748 sysctl -n dev.cpu.0.freq
 1000   6009  97748 sysctl -n hw.acpi.thermal.tz0.temperature
 1000   6728  97748 sysctl -n vm.stats.vm.v_inactive_count
 1000   7043  97748 sysctl -n vm.stats.vm.v_free_count
 1000   7482  97748 sysctl -n vm.stats.vm.v_cache_count
 1000  10363   8568 bc -l
 1000  10863  10363 dc -x
 1000  13143   7773 grep --color -q ^\.
 1000  13798  97748 /bin/sh /home/vermaden/scripts/__conky_if_ip.sh
 1000  15089  14235 ifconfig -u
 1000  16439  14235 grep -v 127.0.0.1
 1000  17738  14235 grep -c inet 
 1000  19069  18612 ifconfig -l -u
 1000  19927  18612 sed s/lo0//g
 1000  20772  13798 ifconfig wlan0
 1000  23388  21410 grep ssid
 1000  24588  13798 grep -q "
 1000  25965  25282 awk /ssid/ {print $2}
 1000  27917  27217 awk /inet / {print $2}
 1000  29941  97748 /bin/sh /home/vermaden/scripts/__conky_if_gw.sh
 1000  32808  31412 route -n -4 -v get default
 1000  34012  31412 awk END{print $2}
 1000  34895  97748 /bin/sh /home/vermaden/scripts/__conky_if_dns.sh
 1000  36118  34895 awk /^nameserver/ {print $2; exit} /etc/resolv.conf
 1000  37628  97748 /bin/sh /home/vermaden/scripts/__conky_if_ping.sh dzen2
 1000  38829  37628 ping -c 1 -s 0 -t 1 -q 9.9.9.9
 1000  42079  41566 mixer -s vol
 1000  42177  41566 awk -F : {printf("%s",$2)}
 1000  44434  43254 zfs list -H -d 0 -o name,avail
 1000  45866  43254 awk {printf("%s/%s ",$1,$2)}
 1000  47004  97748 /bin/sh /home/vermaden/scripts/__conky_battery_separate.sh dzen2
 1000  48282  47004 sysctl -n hw.acpi.battery.units
 1000  49494  47004 sysctl -n hw.acpi.battery.life
 1000  49948  47004 sysctl -n hw.acpi.acline
 1000  52073  51441 acpiconf -i 0
 1000  53055  51441 awk /^State:/ {print $2}
 1000  53981  53186 acpiconf -i 0
 1000  55354  53186 awk /^Remaining capacity:/ {print $3}
 1000  55968  55631 acpiconf -i 1
 1000  57187  55631 awk /^State:/ {print $2}
 1000  58405  57471 acpiconf -i 1
 1000  59201  57471 awk /^Remaining capacity:/ {print $3}
 1000  60961  59252 bsdgrep -v -E (COMMAND|idle)$
 1000  63534  59252 head -3
 1000  62194  59252 sort -r -n
 1000  64629  59252 awk {printf("%s/%d%%/%.1fGB ",$3,$1,$2/1024/1024)}
 1000  64634  93198 tail -1 /home/vermaden/.dzen2-fifo

Lots of processes just to update the information on the top of the screen. That is why I refresh dzen2 information only every 5 minutes and if I want exact information and system status for current moment I just ‘click’ on then dzen2 bar to run all these commands and refresh itself.

This way using DTrace you will know if something unwanted does not steal you precious battery time. You may find such dzen2 config in my FreeBSD Desktop – Part 13 – Configuration – Dzen2 article.

Other

ZFS

By default ZFS will commit transaction group every 5 seconds and that is good default setting for the vfs.zfs.txg.timeout parameter. You may want to increase it a little if needed. To 10 for example. I say about that parameter mostly because lots of guides advice to set it to 1 for various performance reasons but keep in mind that setting it to 1 will prevent your disk (and CPU) from going to sleep thus draining more battery life.

If you want to mess with vfs.zfs.txg.timeout value set it in the /boot/loader.conf file.

Applications

To get more time on battery used applications are also crucial. For example Thunar uses less CPU time then Caja or Nautilus. The Geany text editor uses less CPU resources and memory then Scite or Gedit editors, even GVim takes more resources. Not to mention that custom Openbox/Fluxbox/${YOUR_FAVORITE_WM} window manager based setup will consume a lot less CPU time then entire Gnome or Mate environment.

Hardware

It’s sometimes possible to literally buy more battery time. For example when you want to buy new SSD for you laptop then pick not the fastest one but the most power efficient one. You will probably not feel the performance difference anyway but you will appreciate more battery time.

Most RAM modules come with 1.5V current voltage but there is chance that your laptop may support low power DDR modules with 1.35V current thus increasing your battery time. Also keep in mind that each RAM stick uses about 0.5-1.0W of power so using single 8 GB RAM stick will provide you more battery time the the same 8 GB of memory using two 4 GB RAM modules. This also have performance drawback because with single RAM module you will not be able to use dual channel technology so you will limit you RAM speed. Some laptops have even 4 RAM slots (like ThinkPad W520 for example) so without losing anything you should use two 8 GB RAM sticks instead of four 4 GB RAM sticks for longer battery life.

It is sometimes possible to swap your DVD drive to internal secondary battery. Examples of such laptops are Dell Latitude D630, ThinkPad T420s or ThinkPad T500/W500. Sometimes vendors offer entire slice battery that will stick to the bottom of your laptop like slice battery for ThinkPad X220 or T420/T520/W520 laptops or for the 1st generation of ThinkPad X1 laptop.

Hope that this information will help you squeeze some battery time (or at least save some power) on FreeBSD πŸ™‚

UPDATE 1 – Graphics Card Power Saving

If You have the graphics/drm-kmod package installed you probably use the latest i915kms.ko kernel module.

To set maximum power management for integrated Intel graphics cards put these into the /boot/loader.conf file.

# INTEL DRM WITH graphics/drm-kmod PACKAGE (NEW)
# SKIP UNNECESSARY MODE SETS AT BOOT TIME 
  compat.linuxkpi.fastboot=1
# USE SEMAPHORES FOR INTER RING SYNC
  compat.linuxkpi.semaphores=1
# ENABLE POWER SAVING RENDER C-STATE 6
  compat.linuxkpi.enable_rc6=7
# ENABLE POWER SAVING DISPLAY C-STATES
  compat.linuxkpi.enable_dc=2
# ENABLE FRAME BUFFER COMPRESSION FOR POWER SAVINGS
  compat.linuxkpi.enable_fbc=1

In the past these settings below were used but they are not present anymore.

# INTEL DRM WITH graphics/drm-kmod PACKAGE (OLD)
  drm.i915.enable_rc6=7
  drm.i915.semaphores=1
  drm.i915.intel_iommu_enabled=1

UPDATE 2 – AMD CPU Temperatures

While the coretemp(4) kernel module is used for Intel CPUs the amdtemp(4) kernel module will provide additional temperature information for AMD CPUs.

UPDATE 3 – Suspend/Resume Tips

The biggest enemies of suspend/resume subsystem are bugs in the BIOS/UEFI firmware. Sometimes disabling the Bluetooth helps – that is the option for the Lenovo ThinkPad T420s for example. On the Lenovo ThinkPad X240 it is disabling the TPM (Trusted Platform Module).

UPDATE 4 – Intel Speed Shift

As I run FreeBSD on very decent but also pretty old 2011 ThinkPad W520 this option is a mystery for me – but for those who run more fresh systems may benefit from it.

With the introduction of Intel Skylake (6th Generation) CPUs the FreeBSD operating system is able to utilize the Intel Speed Shift technology. You can read more about it in the hwpstate_intel(4) man page. To make use of it include the following line in the /boot/loader.conf file.

machdep.hwpstate_pkg_ctrl=0

Then in the /etc/sysctl.conf file you would add a line for each CPU thread (not core) with your desired settings.

dev.hwpstate_intel.N.epp=Y

The N here stands for the thread number. On a dual core CPU with 4 threads you would have 4 such lines. On a octa core CPU with 8 threads you would have 8 lines.

The Y value means:

  • 0 – maximum performance
  • 50 – balanced (default)
  • 100 – maximum power savings

To get the most power savings you would use 100 value for all threads as shown below.

dev.hwpstate_intel.0.epp=100
dev.hwpstate_intel.1.epp=100
dev.hwpstate_intel.2.epp=100
dev.hwpstate_intel.3.epp=100
dev.hwpstate_intel.4.epp=100
dev.hwpstate_intel.5.epp=100
dev.hwpstate_intel.6.epp=100
dev.hwpstate_intel.7.epp=100

You could of course use full performance for a single thread, keep one balanced thread and save power on the rest of the threads as shown below.

dev.hwpstate_intel.0.epp=0
dev.hwpstate_intel.1.epp=50
dev.hwpstate_intel.2.epp=100
dev.hwpstate_intel.3.epp=100
dev.hwpstate_intel.4.epp=100
dev.hwpstate_intel.5.epp=100
dev.hwpstate_intel.6.epp=100
dev.hwpstate_intel.7.epp=100

… and when you will be running on AC instead on battery then you may prefer to use that unlimited power with all values of 0 for all threads.

unlimited-power

Settings below.

dev.hwpstate_intel.0.epp=0
dev.hwpstate_intel.1.epp=0
dev.hwpstate_intel.2.epp=0
dev.hwpstate_intel.3.epp=0
dev.hwpstate_intel.4.epp=0
dev.hwpstate_intel.5.epp=0
dev.hwpstate_intel.6.epp=0
dev.hwpstate_intel.7.epp=0

Ideally you would use performance settings when on AC power and run in power save mode when running on battery. You can achieve that with this simple script below running in cron(8) daemon.

#! /bin/sh

case $( sysctl -n hw.acpi.acline ) in

  (0) # BATTERY
    doas sysctl dev.hwpstate_intel.0.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.1.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.2.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.3.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.4.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.5.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.6.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.7.epp=100 1> /dev/null 2> /dev/null
    ;;

  (1) # AC
    doas sysctl dev.hwpstate_intel.0.epp=0   1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.1.epp=50  1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.2.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.3.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.4.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.5.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.6.epp=100 1> /dev/null 2> /dev/null
    doas sysctl dev.hwpstate_intel.7.epp=100 1> /dev/null 2> /dev/null
    ;;

esac

… and now the crontab(5) entry. I am also assuming here that you have needed doas(1) permissions. Modify the script if you useΒ sudo(8) instead.

% crontab -l
# INTEL/SPEED/SHIFT
  * * * * * ~/scripts/acpi-intel-speed-shift.sh

Unfortunately as I do not own laptop with Intel Speed Shift CPU so I do not know if running the 0 and 50 for the first two threads works best. Maybe its better to switch all threads to 100 to save more power. I will let you know when I will get such laptop in the future πŸ™‚

EOF

FreeBSD Desktop – Part 2.1 – Install FreeBSD 12

This article is an update/rewrite to the already published FreeBSD Desktop – Part 2 – Install. With the upcoming introduction of the FreeBSD 12.0-RELESE version new possibilities arise when it comes to installation. I already talked/showed that method in my ZFS Boot Environments Reloaded at NLUUG presentation but to make it more available and obvious part of my FreeBSD Desktop series I write about it again in dedicated article entry.

You may want to check other articles in the FreeBSD Desktop series on the FreeBSD Desktop – Global Page where you will find links to all episodes of the series along with table of contents for each episode’s contents.

Now (in FreeBSD 12.x) it is possible to install FreeBSD on GELI encrypted root on ZFS pool without any additional partitions or filesystems. No longer separate UFS or ZFS boot pool /boot filesystem is needed. And what is even more appealing such setup is supported both on UEFI and BIOS (also refereed as Legacy or CSM) systems. Such setup is also compatible with both new bectl(8) utility and the old proven beadm(8) tool. It is also nice that to make such setup you only need to choose the Auto ZFS option from the bsdinstall(8) so you will not have to do it by hand. I advice using GPT (BIOS+UEFI) as it will support both system types so when you are running BIOS system now and will move the disk to other system that boots with UEFI it will also just work out of the box.

The FreeBSD 12.0 is currently at the RC1 stage so we will use that one for below examples of such setup. The 12.0-RELEASE is expected to arise before Christmas if no significant problems or bugs will be found on the road to RC2 and RC3 editions.

For the record here is the FreeBSD 12.0-RC1 Availability information page and aggregated FreeBSD 12.0-RELEASE Release Notes for the upcoming new major FreeBSD version, but it is not yet complete/ready.

I will only show one install process that will work for both UEFI and BIOS systems – the crucial option here is GPT (BIOS+UEFI) to select (which is also the default one). The other option that You need to select is Yes for the Encryption part and also select the SWAP size. You may as well do not use swap and enter ‘0‘ here which means that SWAP partition will not be created. You may as well create ZFS ZVOL partition for SWAP on ZFS pool later or just create a file like /SWAP and enable it as SWAP. No matter which SWAP option you will choose if your system swaps then you are too low on memory and neither of these methods are better or worse then.

freebsd-install-01.png

freebsd-install-02.png

freebsd-install-03.png

One last thing about the default FreeBSD (no matter if 11.x or 12.x) ZFS dataset/filesystem layout. I showed it on my ZFS Boot Environments/ZFS Boot Environments Reloaded presentations but without any text comment as I talked it live.

By default both /var and /usr filesystems are part of the Boot Environment. They are protected and snapshoted during the beadm create newbe process (or by bectl(8) also). Its appears that /var and /usr are separate processes when you type zfs list commend as shown on the slide below.

zroot-layout-01.png

… but when you check the canmount parameter for all ZFS datasets, then it become obvious that /usr and /var are ’empty’ datasets (not mounted).

zroot-layout-02.png

… and also confirmation from theΒ df(1) tool.

zroot-layout-03.png

I asked FreeBSD Developers what is the reason for such construct and its for the mountpoint inheritance purposes. For example when zroot/usr has mountpoint set to /usr then when you create zroot/usr/local dataset, then it will automatically get the /usr/local for the mountpoint parameter by inheritance. At the first sight it may be misleading (I also got caught) but it makes sense when you think about it.

The only filesystems that are NOT included for the Boot Environment protection are these:

  • /usr/home
  • /usr/ports
  • /usr/src
  • /var/audit
  • /var/crash
  • /var/log
  • /var/mail
  • /var/tmp

While in most cases it is not needed to protect these in the Boot Environment protection if you want to also protect these type these two comments to move all the /usr/* and /var/* datasets/filesystems into the Boot Environment pool/ROOT/dataset. It will work on a running system without need for reboot, just make sure you use -u flag.

# zfs rename -u zroot/usr zroot/ROOT/default/usr
# zfs rename -u zroot/var zroot/ROOT/default/var

Now grab that FreeBSD ISO and install it the best possible way up to date πŸ™‚

You will probably want to get amd64 version which is suitable for both 64-bit AMD and Intel systems.

EOF

Β 

Silent Fanless FreeBSD Server – DIY Backup

I already once wrote about this topic at the Silent Fanless FreeBSD Desktop/Server article. To my pleasant surprise BSD NOW Episode 253: Silence of the Fans featured my article for which I am very grateful. Today I would like to show another practical example of such setup and with more hands on approach along with real power usage measurements with power meter. I also got more power efficient ASRock N3150B-ITX motherboard with only 6W TDP which includes 4-core Celeron N3150 CPU and also nice small Supermicro SC101i Mini ITX case. Keep in mind that ASRock also made very similar N3150-ITX motherboard (no ‘B’ in model name) with different ports/connectors that may better suit your needs better.

You may also check the follow up Silent Fanless FreeBSD Server – Redundant Backup article.

Build

Here is how the Supermicro SC101i case looks like with ASRock N3150B-ITX motherboard installed.

silent-backup-case-external.jpg

silent-backup-case-back.jpg

One thing that surprised me very much was the hard disk cost. The internal Seagate 4TB ST4000LM024 2.5 SATA drive costs about $180-190 but the same disk sold as Maxtor M3 4TB 2.5 disk in external case with Maxtor brand (which is owned by Seagate anyway) and USB 3.0 port costs half of that – about $90-100. At least in Europe/Poland location.

I think you do already know where I am going with my thoughts. I will use an external Maxtor M3 4TB 2.5 drive and connect it via the USB 3.0 port in this setup. While SATA III provides theoretical throughput of 6Gbps the USB 3.0 provides 5Gbps theoretical throughput. The difference can be important for low latency high throughput SSD drives that approach 580MB/s speed but not for traditional rotational disks moving gently at 5400RPM.

The maximum performance I was able to squeeze from this Maxtor M3 4TB 2.5 USB 3.0 drive was 90MB/s write speed and 120MB/s read speed using pv(1) tool, and that was at the beginning of the disk. These speeds will drop to about 70MB/s and 90MB/s at the end of the disk respectively for write and read operations. We are not even approaching SATA I standard here which tops at 1.5Gbps. Thus it will not make a difference or not a significant one for sure for such storage.

At first I wanted to make a hole on the motherboard end steel plate (somewhere beside the back ports) with drill to get outside with USB cable from the case and attach it to one of the USB 3.0 ports at the back of the motherboard but fortunately I got better idea. This motherboard has connector for internal USB 3.0 (so called front panel USB on the case) so I bought Akyga AK-CA-57 front panel cable with USB 3.0 port and connected everything inside the case.

This is the Akyga AK-CA-57 USB 3.0 cable.

silent-backup-usb-akyga-cable-AK-CA-57.jpg

If I was going to install two USB 3.0 disks using this method I would use one of these cables instead:

The only problem can be more physical one – will it blend will it fit? Fortunately I was able to find a way to fit it in the case and there is even space for the second disk. As this will be my offsite backup replacement which is only 3rd stage/offsite backup I do not need to create redundant mirror/RAID1 protection but it’s definitely possible with two Maxtor M3 4TB 2.5 USB 3.0 drives.

The opened Supermicro SC101i case with ASRock N3150B-ITX motherboard inside and attached Pico PSU looks like that.

silent-backup-mobo-case.jpg

With attached Akyga AK-CA-57 USB 3.0 cable things get little narrow, but with proper cable lay you will still be able to fit another internal 2.5 SATA disk or external 2.5 USB 3.0 disk.

silent-backup-mobo-case-blue.jpg

I attached Akyga AK-CA-57 cable to this USB 3.0 connector on the motherboard.

silent-backup-mobo-case-usb.jpg

Case with Maxtor M3 4TB disk. The disk placement required little modifications.

silent-backup-mobo-case-blue-disk.jpg

I created custom disk holders using steel plates I got from window mosquito net set for my home but you should be able to get something similar in any hardware shop. I modified them a little with pliers.

silent-backup-handles

I also ‘silenced’ the disk vibrations with felt stickers.

silent-backup-silence.jpg

The silenced disk in the Supermicro SC101i case.

silent-backup-mobo-case-blue-disk-silence.jpg

Ancestor

Before this setup I used Raspberry Pi 2B with external Western Digital 2TB 2.5 USB 3.0 disk but the storage space requirements become larger so I needed to increase that. It was of course with GELI encryption and ZFS with enabled LZ4 compression on top. The four humble ARM32 cores and soldered 1GB of RAM was able to squeeze whooping 5MB/s read/write experience from this ZFS/GELI setup but that was not hurting me as I used rsync(1) for differential backups and the Internet connection to that box was limited to about 1.5MB/s. I would still use that setup but it just won’t boot with that larger Maxtor M3 4TB disk because it requires more power and I already used stronger 5V 3.1A charger then 5V 2.0A suggested by vendor. Even the safe_mode_gpio=4 and max_usb_current=1 options at /boot/msdos/config.txt did not help.

Cost

The complete setup price tops at $220 total. Here are the parts used.

PRICE  COMPONENT
  $59  CPU/Motherboard ASRock N3150B-ITX Mini-ITX
  $14  RAM Crucial 4GB DDR3L 1.35V
  $13  PSU 12V 7.5A 90W Pico (internal)
   $2  PSU 12V 2.5A 30W Leader Electronics (external)
  $29  Supermicro SC101i (used)
   $3  Akyga AK-CA-57 USB 3.0 Cable
   $3  SanDisk Fit 16GB USB 2.0 Drive (system)
  $95  Maxtor M3 4TB 2.5 USB 3.0 Drive (data)
 $220  TOTAL

PSU

In earlier Silent Fanless FreeBSD Desktop/Server article I used quite large 90W PSU from FSP Group. From the PSUs that I owned only ThinkPad W520/W530 bricks can compete in size with this beast. As this motherboard will use very little power (details lower) it will require a lot smaller PSU. As the FSP Group PSU has IEC C14 slot it also requires additional IEC C13 power cable which makes it even bigger solution. The new 12V 2.5A 30W is very compact and also costs fraction of the 90W FSP Group gojira.

New Leader Electronics PSU label.

silent-backup-psu-ext-label.jpg

Below you can see the comparison for yourself.

silent-backup-psu-compare

I also got cheaper and less powerful Pico PSU which now tops as 12V 7.5A 90W power.

silent-backup-psu-pico-12V-90W.jpg

Power Consumption

This is where it gets really interesting. I measured the power consumption with power meter.

silent-backup-power-meter.jpg

Idle

When this box is booted without any media attached it uses only 7.5W of power idling. While the system was idle with SanDisk 16GB USB 2.0 drive (on which FreeBSD was installed) it used about 8.0W of power. When booted with Maxtor M3 4TB disk inside and SanDisk 16GB USB 2.0 drive attached it run idle at about 8.5W of power.

Load

As I do not need full CPU speed I limited the CPU speed in powerd(8) options to 1.2Ghz. With this limit set the fully loaded system with all 4 cores busy at 100% and two dd(8) processes for read both boot SanDisk 16GB drive and Maxtor M3 4TB disk and with GELI enabled ZFS pool doing scrub operation in progress and additional two find(1) processes for both disks it would not pass the 13.9W barrier. Without CPU limitation (that means Intel Turbo Boost enabled) the system used 16.0W of power at most.

Summary of power usage for this box.

 POWER  TYPE  CONFIGURATION
 7.5 W  IDLE  System
 8.0 W  IDLE  System + SanDisk 16GB drive
 8.5 W  IDLE  System + SanDisk 16GB drive + Maxtor M3 4TB drive + CPU 1.2 Ghz limit
 8.5 W  IDLE  System + SanDisk 16GB drive + Maxtor M3 4TB drive
13.9 W  LOAD  System + SanDisk 16GB drive + Maxtor M3 4TB drive + CPU 1.2 Ghz limit
16.0 W  LOAD  System + SanDisk 16GB drive + Maxtor M3 4TB drive

For comparision the Raspberry Pi 2B with 16GB MicroSD card attached used only 1.5W but we all know how slow it is. When used with Western Digital 2TB 2.5 USB 3.0 drive it used about 2.2W at idle state.

Configuration for Low Power Consumption

Below are FreeBSD configuration files used in this box to lower the power consumption.

The /etc/sysctl.conf file.

# ANNOYING THINGS
  vfs.usermount=1
  kern.coredump=0
  hw.syscons.bell=0
  kern.vt.enable_bell=0

# LIMIT ZFS ARC EFFICIENTLY
  kern.maxvnodes=32768

# ALLOW UPGRADES IN JAILS
  security.jail.chflags_allowed=1

# ALLOW RAW SOCKETS IN JAILS
  security.jail.param.allow.raw_sockets=1
  security.jail.allow_raw_sockets=1

# RANDOM PID
  kern.randompid=12345

# PERFORMANCE/ALL SHARED MEMORY SEGMENTS WILL BE MAPPED TO UNPAGEABLE RAM 
  kern.ipc.shm_use_phys=1

# MEMORY OVERCOMMIT SEE tuning(7)
  vm.overcommit=2

# NETWORK/DO NOT SEND RST ON SEGMENTS TO CLOSED PORTS
  net.inet.tcp.blackhole=2

# NETWORK/DO NOT SEND PORT UNREACHABLES FOR REFUSED CONNECTS
  net.inet.udp.blackhole=1

# NETWORK/ENABLE SCTP BLACKHOLING blackhole(4) FOR MORE DETAILS
  net.inet.sctp.blackhole=1

# NETWORK/MAX SIZE OF AUTOMATIC RECEIVE BUFFER (2097152) [4x]
  net.inet.tcp.recvbuf_max=8388608

# NETWORK/MAX SIZE OF AUTOMATIC SEND BUFFER (2097152) [4x]
  net.inet.tcp.sendbuf_max=8388608

# NETWORK/MAXIMUM SOCKET BUFFER SIZE (5242880) [3.2x]
  kern.ipc.maxsockbuf=16777216

# NETWORK/MAXIMUM LISTEN SOCKET PENDING CONNECTION ACCEPT QUEUE SIZE (128) [8x]
  kern.ipc.soacceptqueue=1024

# NETWORK/DEFAULT tcp MAXIMUM SEGMENT SIZE (536) [2.7x]
  net.inet.tcp.mssdflt=1460

# NETWORK/MINIMUM TCP MAXIMUM SEGMENT SIZE (216) [6x]
  net.inet.tcp.minmss=1300

# NETWORK/LIMIT ON SYN/ACK RETRANSMISSIONS (3)
  net.inet.tcp.syncache.rexmtlimit=0

# NETWORK/USE TCP SYN COOKIES IF THE SYNCACHE OVERFLOWS (1)
  net.inet.tcp.syncookies=0

# NETWORK/ENABLE TCP SEGMENTATION OFFLOAD (1)
  net.inet.tcp.tso=0

# NETWORK/ENABLE IP OPTIONS PROCESSING ([LS]SRR, RR, TS) (1)
  net.inet.ip.process_options=0

# NETWORK/ASSIGN RANDOM ip_id VALUES (0)
  net.inet.ip.random_id=1

# NETWORK/ENABLE SENDING IP REDIRECTS (1)
  net.inet.ip.redirect=0

# NETWORK/IGNORE ICMP REDIRECTS (0)
  net.inet.icmp.drop_redirect=1

# NETWORK/ASSUME SO_KEEPALIVE ON ALL TCP CONNECTIONS (1)
  net.inet.tcp.always_keepalive=0

# NETWORK/DROP TCP PACKETS WITH SYN+FIN SET (0)
  net.inet.tcp.drop_synfin=1

# NETWORK/RECYCLE CLOSED FIN_WAIT_2 CONNECTIONS FASTER (0)
  net.inet.tcp.fast_finwait2_recycle=1

# NETWORK/CERTAIN ICMP UNREACHABLE MESSAGES MAY ABORT CONNECTIONS IN SYN_SENT (1)
  net.inet.tcp.icmp_may_rst=0

# NETWORK/MAXIMUM SEGMENT LIFETIME (30000) [0.27x]
  net.inet.tcp.msl=8192

# NETWORK/ENABLE PATH MTU DISCOVERY (1)
  net.inet.tcp.path_mtu_discovery=0

# NETWORK/EXPIRE TIME OF TCP HOSTCACHE ENTRIES (3600) [2x]
  net.inet.tcp.hostcache.expire=7200

# NETWORK/TIME BEFORE DELAYED ACK IS SENT (100) [0.2x]
  net.inet.tcp.delacktime=20

The /boot/loader.conf file.

# BOOT OPTIONS
  autoboot_delay=1
  boot_mute=YES

# MODULES FOR BOOT
  zfs_load=YES

# DISABLE HYPER THREADING
  machdep.hyperthreading_allowed=0

# REDUCE NUMBER OF SOUND GENERATED INTERRUPTS
  hw.snd.latency=7

# RACCT/RCTL RESOURCE LIMITS
  kern.racct.enable=1

# PIPE KVA LIMIT | 320 MB
  kern.ipc.maxpipekva=335544320

# NUMBER OF SEGMENTS PER PROCESS
  kern.ipc.shmseg=1024

# LARGE PAGE MAPPINGS
  vm.pmap.pg_ps_enabled=1

# SHARED MEMORY
  kern.ipc.shmmni=1024
  kern.ipc.shmseg=1024

# ZFS TUNING
  vfs.zfs.prefetch_disable=1
  vfs.zfs.cache_flush_disable=1
  vfs.zfs.vdev.cache.size=16M
  vfs.zfs.arc_min=32M
  vfs.zfs.arc_max=128M
  vfs.zfs.txg.timeout=1

# NETWORK MAX SEND QUEUE SIZE
  net.link.ifqmaxlen=2048

# POWER OFF DEVICES WITHOUT ATTACHED DRIVER
  hw.pci.do_power_nodriver=3

# AHCI POWER MANAGEMENT FOR EVERY USED CHANNEL (ahcich 0-7)
  hint.ahcich.0.pm_level=5
  hint.ahcich.1.pm_level=5
  hint.ahcich.2.pm_level=5
  hint.ahcich.3.pm_level=5
  hint.ahcich.4.pm_level=5
  hint.ahcich.5.pm_level=5
  hint.ahcich.6.pm_level=5
  hint.ahcich.7.pm_level=5

# GELI THREADS
  kern.geom.eli.threads=2
  kern.geom.eli.batch=1

The /etc/rc.conf file.

# NETWORK
  hostname=offsite.local
  background_dhclient=YES
  extra_netfs_types=NFS
  defaultroute_delay=3
  defaultroute_carrier_delay=3

# MODULES/COMMON/BASE
  kld_list="${kld_list} aesni geom_eli"
  kld_list="${kld_list} fuse coretemp sem cpuctl ichsmb cc_htcp"
  kld_list="${kld_list} libiconv cd9660_iconv msdosfs_iconv udf_iconv"

# POWER
  performance_cx_lowest=C1
  economy_cx_lowest=Cmax
  powerd_enable=YES
  powerd_flags="-n adaptive -a hiadaptive -b adaptive -m 400 -M 1200"

# DAEMONS | yes
  zfs_enable=YES
  nfs_client_enable=YES
  syslogd_flags='-s -s'
  sshd_enable=YES

# DAEMONS | no
  sendmail_enable=NONE
  sendmail_submit_enable=NO
  sendmail_outbound_enable=NO
  sendmail_msp_queue_enable=NO

# FS
  fsck_y_enable=YES
  clear_tmp_enable=YES
  clear_tmp_X=YES
  growfs_enable=YES

# OTHER
  keyrate=fast
  font8x14=vgarom-8x14
  virecover_enable=NO
  update_motd=NO
  devfs_system_ruleset=desktop
  hostid_enable=NO

USB Boot Drive

I was not sure if I should use USB 2.0 drive or USB 3.0 drive for FreeBSD system so I got both versions from SanDisk and tested their performance with pv(1) and diskinfo(8) tools. The pv(1) utility had options enabled shown below and for diskinfo(8) the -c and -i parameters were used.

% which pv
pv: aliased to pv -t -r -a -b -W -B 1048576

The dmesg(8) information for the SanDisk Fit USB 2.0 16GB drive.

# dmesg | tail -6
da0 at umass-sim0 bus 0 scbus3 target 0 lun 0
da0:  Removable Direct Access SPC-4 SCSI device
da0: Serial Number 4C530001100609104091
da0: 40.000MB/s transfers
da0: 15060MB (30842880 512 byte sectors)
da0: quirks=0x2

The dmesg(8) information for the SanDisk Fit USB 3.0 16GB drive.

# dmesg | tail -6
da0 at umass-sim0 bus 0 scbus3 target 0 lun 0
da0:  Removable Direct Access SPC-4 SCSI device
da0: Serial Number 4C530 001070202100093
da0: 40.000MB/s transfers
da0: 14663MB (30031250 512 byte sectors)
da0: quirks=0x2

There is also noticeable size difference as the USB 2.0 version has additional 400 MB of space!

By the way … the SanDisk Fit USB 3.0 16GB came with this sticker inside the box – a serial number for the RescuePRO Deluxe software – which I will never use. Not because its bad or something but because I have no such needs. You may take it … of course unless someone else did not took it already πŸ™‚

silent-backup-license.jpg

Below are the results of the benchmarks, I tested them in both USB 2.0 and USB 3.0 ports.


                   DRIVE  USB  pv/READ  pv/WRITE  diskinfo/OVERHEAD  diskinfo/IOPS
SanDisk Fit USB 2.0 16GB  2.0   29MB/s     5MB/s   0.712msec/sector           2521
SanDisk Fit USB 2.0 16GB  3.0   33MB/s     5MB/s   0.799msec/sector           2441
SanDisk Fit USB 3.0 16GB  2.0   35MB/s     9MB/s   0.618msec/sector           1920
SanDisk Fit USB 3.0 16GB  3.0   91MB/s    11MB/s   0.567msec/sector           1588

What is also interesting is that while USB 2.0 version has lower throughput it has more IOPS then the newer USB 3.0 incarnation of the SanDisk Fit drive. I also did other more real life test. I checked how long would it take to boot FreeBSD system installed on each of them from the loader(8) screen to the login: prompt. The difference is 5 seconds. Details are shown below.

 TIME  DRIVE
  28s  SanDisk Fit USB 3.0 16GB
  33s  SanDisk Fit USB 2.0 16GB

With such small ~15% difference I will use SanDisk Fit USB 2.0 16GB as it sticks out little less outside from the slot as shown below.

silent-backup-usb-drives.jpg

Cloud Storage Prices Comparison

The Tarsnap“online backups for the truly paranoid” – costs $0.25/GB/month. The price in Tarsnap is for data transmitted after deduplication and compression but that does not change much here. For my data the compressratio property from ZFS dataset is at 3% (1.03). When I estimate deduplication savings with zdb -S pool command I get additional 1% of the savings (1.01). Lets assume that with both deduplication and compression it would take 5% (1.05) savings. That would lower the Tarsnap price to $0.2375/GB/month.

The Backblaze B2 Cloud Storage – storage costs $0.005/GB/month.

Our single 4TB disk solution costs $230 for lets say 3 years. You can expect disk failure after that period but it may serve you as well for another 3 years. Now as we know the cloud storage prices lets calculate price for 4TB data stored for 3 years in these cloud services.

Self Solution Electricity Cost

We also need to calculate how much energy our build solution would consume. Currently 1kWh of power costs about $0.20 in Europe/Poland (rounded up). This means that running computer with 1000W power usage for 1 hour would cost you $0.20 on electricity bill. Our solution idles at 8.5W and uses 13.9W when fully loaded. It will be idle for most of the time so I will assume that it will use 10W on average here. That would cost us $0.002 for 10W device running for 1 hour.

Below you will also find calculations for 1 day (24x multiplier), 1 year (another 365.25x multiplier) and 3 years (another 3x multiplier).

  COST  TIME
$0.002  1 HOUR
$0.048  1 DAY
$17.53  1 YEAR
$52.60  3 YEARS

Our total 3 years electricity cost is $282.60 for building and then running the system non-stop. We can also implement features like Wake On LAN to limit that power usage even more for example.

Here are these cloud storage service providers prices.


PROVIDER     PRICE  DATA  TIME
Tarsnap    $0.2375   1GB  1 Month
Backblaze  $0.0050   1GB  1 Month

The price for 1 month of keeping 4TB of data on these providers looks as follows.


PROVIDER   PRICE  DATA  TIME
Tarsnap     $973   4TB  1 Month
Backblaze    $20   4TB  1 Month

For just 1 month the Tarsnap is 4 TIMES more expensive the keeping the backup on your self computer with 4TB disk. The Backblaze service is at 1/10 cost which is still reasonable.

Lets compare prices for 3 years of 4TB storage.


PROVIDER    PRICE  DATA  TIME
Tarsnap    $35021   4TB  3 Years
Backblaze    $737   4TB  3 Years

After 3 years the Backblaze solutions is about 2.5 TIMES more expensive then our personal setup, but if you really do not want to create your solution the difference for 3 years is not that big. The Tarsnap is out of bounds here being more then 120 TIMES more expensive then self hosted solution. Remember that I also did not included costs for transferring the data into or from the cloud storage. That would make cloud storage costs even bigger depending how often you would want to pull/push your data.

EOF