Monthly Archives: March 2018

FreeBSD Desktop – Part 1 – Simplified Boot

This is the first post in the FreeBSD Desktop series.

You may want to check other articles in the FreeBSD Desktop series on the FreeBSD Desktop – Global Page where you will find links to all episodes of the series along with table of contents for each episode’s contents.

The default FreeBSD boot process is quite verbose with a lot of debugging information along with kernel messages. We may divide that boot process into several β€˜screens’ or stages. First thing You see is the β€˜BIOS’ screen of the computer manufacturer. SecondΒ  thing is the FreeBSD BTX Loader. The third one is the FreeBSD Boot Menu with eventual ZFS Boot Environments if You use ZFS for root filesystem and other options to select like Single User Mode for example. The 4th one is the system boot along with kernel messages in non-native resolution. In the middle of that stage screen switches to native resolution and continues to display kernel messages and services leading to the text prompt with login: at the end. There comes optional fifth screen which may be graphically started (x11) login manager like slim or gdm.

This verbose information is usually useful for servers but not that much for laptops and/or desktop/workstation systems. The UNIX philosophy is to not β€˜say’ anything to stdout if everything is OK, so stdout/stderr should only be used when something is wrong … like on AMIGA, if anything is wrong then I want to see big red sign like [GURU MEDITAION] but if everything is ok, shut the … slience is golden πŸ™‚

guru-meditation

I really like Sun Solaris 10 approach here, that it boots with minimal information like its version and hostname into the login: prompt in less then 10 lines. The image below is from the first Sun Solaris 10 boot, so it includes additional OpenSSH server key generation information.

sun-solaris-10-boot-first

Unfortunately – despite what Oracle says – Oracle Solaris is dying, I gathered most of the information here – Oracle just killed Solaris/SPARC/ZFS teams – https://forums.freebsd.org/threads/62320/ – on FreeBSD Forums. The recent Oracle Solaris 11.4 release process along with public beta will not change that. Oracle Solaris will be kept in maintenance mode for the rest of its life, which is set by Oracle to 2034 currently. Pity because even BSD bits recently found its way into it Solaris, for example the OpenBSD PF firewall, there are some differences – Comparing PF in Oracle Solaris to IP Filter and to OpenBSD Packet Filter – https://docs.oracle.com/cd/E37838_01/html/E60993/pfovw-comparall.html – but there are differences between OpenBSD PF and FreeBSD PF too.

Back to FreeBSD – according to the project website – https://freebsd.org/ – β€œFreeBSD is an operating system used to power modern servers, desktops, and embedded platforms” so why not tune the boot process to be more appealing on laptops/desktops? Below are the stages of the default FreeBSD boot process up to the login: prompt.

stage0-BIOS.jpg

stage1-BTX-Loader.jpg

stage2-Boot-Menu.jpg

stage3-NOMOD-Non-Native-Boot.jpg

stage4a-NOMOD-Native-Boot-A.jpg.jpg

stage4b-NOMOD-Native-Boot-B.jpg

Not very lean to my standards. But with one parameter in /boot/loader.conf and 5 slightly silenced startup scripts its whole a lot better. Here are the modifications needed.

First add the boot_mute=YES option to the /boot/loader.conf file.

As we are here, You may as well add autoboot_delay=2 parameter to the /boot/loader.conf file to speed up boot process by 8 seconds. Default delay is 10 seconds.

% grep boot_mute /boot/loader.conf
boot_mute=YES
%

Next we will need to modify these startup scripts.

  • /etc/rc.d/ldconfig
  • /etc/rc.d/netif
  • /etc/rc.d/nfsclient
  • /etc/rc.d/random
  • /etc/rc.d/routing

Here is the summary of the changes. In most cases its just adding 1> /dev/null or 1> /dev/null 2> /dev/null to not display unneeded information at boot process.

% grep -n -E '(1|2)> /dev/null' /etc/rc.d/* | grep -E 'routing|netif|ldconfig'
/etc/rc.d/ldconfig:40: check_startmsgs && echo 'ELF ldconfig path:' ${_LDC} 1> /dev/null
/etc/rc.d/ldconfig:60: echo '32-bit compatibility ldconfig path:' ${_LDC} 1> /dev/null
/etc/rc.d/netif:260: /sbin/ifconfig ${ifn} 1> /dev/null 2> /dev/null
/etc/rc.d/routing:70: eval static_${_a} delete $_if 1> /dev/null 2> /dev/null
/etc/rc.d/routing:97: static_$2 add $3 1> /dev/null 2> /dev/null
/etc/rc.d/routing:104: static_$2 add $3 add $3 1> /dev/null 2> /dev/null

The only exception is the /etc/rc.d/random which requires little more love.

% grep -n -A 8 'random_start()' /etc/rc.d/random
45:random_start()
46-{
47-
48-   # if [ ${harvest_mask} -gt 0 ]; then
49-   #       echo -n 'Setting up harvesting: '
50-   #       ${SYSCTL} kern.random.harvest.mask=${harvest_mask} > /dev/null
51-   #       ${SYSCTL_N} kern.random.harvest.mask_symbolic
52-   # fi
53-

Here are diff(1) patches if that way will be easier for you.

% diff -rq ~/CLEAN-FreeBSD-11.1-RELEASE/etc/rc.d /etc/rc.d | column -t
Files  ~/CLEAN-FreeBSD-11.1-RELEASE/etc/rc.d/ldconfig   and  /etc/rc.d/ldconfig   differ
Files  ~/CLEAN-FreeBSD-11.1-RELEASE/etc/rc.d/netif      and  /etc/rc.d/netif      differ
Files  ~/CLEAN-FreeBSD-11.1-RELEASE/etc/rc.d/nfsclient  and  /etc/rc.d/nfsclient  differ
Files  ~/CLEAN-FreeBSD-11.1-RELEASE/etc/rc.d/random     and  /etc/rc.d/random     differ
Files  ~/CLEAN-FreeBSD-11.1-RELEASE/etc/rc.d/routing    and  /etc/rc.d/routing    differ
% diff -u ./rc.d/ldconfig /etc/rc.d/ldconfig
--- ./rc.d/ldconfig     2017-07-21 04:11:06.000000000 +0200
+++ /etc/rc.d/ldconfig  2017-12-18 09:12:18.190074000 +0100
@@ -37,7 +37,7 @@
                                _LDC="${_LDC} ${i}"
                        fi
                done
-               check_startmsgs && echo 'ELF ldconfig path:' ${_LDC}
+               check_startmsgs && echo 'ELF ldconfig path:' ${_LDC} 1> /dev/null
                ${ldconfig} -elf ${_ins} ${_LDC}
 
                case `sysctl -n hw.machine_arch` in
@@ -57,7 +57,7 @@
                                fi
                        done
                        check_startmsgs &&
-                           echo '32-bit compatibility ldconfig path:' ${_LDC}
+                           echo '32-bit compatibility ldconfig path:' ${_LDC} 1> /dev/null
                        ${ldconfig} -32 -m ${_ins} ${_LDC}
                        ;;
                esac

% diff -u ./rc.d/netif /etc/rc.d/netif
--- ./rc.d/netif        2017-07-21 04:11:06.000000000 +0200
+++ /etc/rc.d/netif     2017-11-30 17:32:11.394251000 +0100
@@ -257,7 +257,7 @@
                esac
                if check_startmsgs; then
                        for ifn in ${_ok}; do
-                               /sbin/ifconfig ${ifn}
+                               /sbin/ifconfig ${ifn} 1> /dev/null 2> /dev/null
                        done
                fi
        fi
% diff -u ./rc.d/nfsclient /etc/rc.d/nfsclient
--- ./rc.d/nfsclient    2017-07-21 04:11:06.000000000 +0200
+++ /etc/rc.d/nfsclient 2017-12-18 09:15:38.200376000 +0100
@@ -44,7 +44,7 @@
        # successfully notified about a previous client shutdown.
        # If there is no /var/db/mounttab, we do nothing.
        if [ -f /var/db/mounttab ]; then
-               rpc.umntall -k
+               rpc.umntall -k 2> /dev/null
        fi
 }
 load_rc_config $name
% diff -u ./rc.d/random /etc/rc.d/random
--- ./rc.d/random       2017-07-21 04:11:06.000000000 +0200
+++ /etc/rc.d/random    2018-01-09 13:32:18.439347000 +0100
@@ -45,13 +45,13 @@
 random_start()
 {
 
-       if [ ${harvest_mask} -gt 0 ]; then
-               echo -n 'Setting up harvesting: '
-               ${SYSCTL} kern.random.harvest.mask=${harvest_mask} > /dev/null
-               ${SYSCTL_N} kern.random.harvest.mask_symbolic
-       fi
+       # if [ ${harvest_mask} -gt 0 ]; then
+       #       echo -n 'Setting up harvesting: '
+       #       ${SYSCTL} kern.random.harvest.mask=${harvest_mask} > /dev/null
+       #       ${SYSCTL_N} kern.random.harvest.mask_symbolic
+       # fi
 
-       echo -n 'Feeding entropy: '
+       echo -n 'Feeding entropy:'
 
        if [ ! -w /dev/random ] ; then
                warn "/dev/random is not writeable"

% diff -u ./rc.d/routing /etc/rc.d/routing
--- ./rc.d/routing      2017-07-21 04:11:06.000000000 +0200
+++ /etc/rc.d/routing   2017-12-18 09:22:16.604428000 +0100
@@ -67,7 +67,7 @@
        ""|[Aa][Ll][Ll]|[Aa][Nn][Yy])
                for _a in inet inet6 atm; do
                        afexists $_a || continue
-                       eval static_${_a} delete $_if
+                       eval static_${_a} delete $_if 1> /dev/null 2> /dev/null
                        # When $_if is specified, do not flush routes.
                        if ! [ -n "$_if" ]; then
                                eval routing_stop_${_a}
@@ -94,14 +94,14 @@
        _ret=0
        case $1 in
        static)
-               static_$2 add $3
+               static_$2 add $3 1> /dev/null 2> /dev/null
                _ret=$?
                ;;
        options)
                options_$2
                ;;
        doall)
-               static_$2 add $3
+               static_$2 add $3 add $3 1> /dev/null 2> /dev/null
                _ret=$?
                options_$2
                ;;

Now lets see how FreeBSD boots now after the modifications.

stage0-BIOS.jpg

stage1-BTX-Loader.jpg

stage2-Boot-Menu.jpg

stage3-MOD-Non-Native-Boot.jpg.jpg

stage4a-MOD-Native-Boot-A.jpg.jpg

stage4b-MOD-Native-Boot-B.jpg

Its definitely not perfect, but a lot better in my taste.

Now lets login to desktop πŸ™‚

I prefer not to use a login manager so I have an alias named x to xinit command. This way after I login I type x press [ENTER] and x11 desktop is started.

% which x
x: aliased to xinit ~/.xinitrc -- -dpi 75 -nolisten tcp 1> /dev/null 2> /dev/null

stage4c-MOD-Native-Boot-C.jpg

stage5-X11.jpg

UPDATE 1 – FreeBSD 12.x

I recently tried FreeBSD 12.0-RC* versions and there is one ‘talkative’ script that also could be ‘silenced’ a little.

Its the /etc/rc.d/devmatch scrtipt.

Here is the needed patch to make it great again nice and clean again.

% diff -u /home/vermaden/rc-devmatch devmatch 
--- /home/vermaden/rc-devmatch        2018-11-27 17:49:53.573514000 +0100
+++ devmatch    2018-11-27 17:50:11.955342000 +0100
@@ -65,7 +65,7 @@
                case "#${x}#" in
                *"#${m}#"*) continue ;;
                esac
-               echo "Autoloading module: ${m}"
+               # echo "Autoloading module: ${m}"
                kldload -n ${m}
        done
        devctl thaw

UPDATE 2 – The drm-kmod Silencing

Recently to get support for newer GPUs the drm-kmod meta port/package is needed. The thing is that if you add the /boot/modules/i915kms.ko (for Intel GPUs) to the kld_list parameter it will display following error message from the kernel even with boot_mute=YES in the /boot/loader.conf file.

Loading kernel modules:
Dec 16 11:08:03 t420s kernel: Failed to add WC MTRR for [0xe0000000-0xefffffff]: -28; performance may suffer

The syslogd is guilty here with its default configuration in the /etc/syslog.conf file. To make it silent (not print pointless messages) make this change in the /etc/syslog.conf file.

% diff -u /root/syslog.conf /etc/syslog.conf
--- /root/syslog.conf   2018-12-18 11:49:48.204878000 +0100
+++ /etc/syslog.conf    2018-12-18 11:49:55.681504000 +0100
@@ -5,7 +5,7 @@
 #      separators. If you are sharing this file between systems, you
 #      may want to use only tabs as field separators here.
 #      Consult the syslog.conf(5) manpage.
-*.err;kern.warning;auth.notice;mail.crit                       /dev/console
+# *.err;kern.warning;auth.notice;mail.crit                       /dev/console
 *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err  /var/log/messages
 security.*                                                     /var/log/security
 auth.info;authpriv.info                                        /var/log/auth.log

Now it will not print these pointless messages.

This applies both to 11.2-RELEASE and 12.0-RELEASE versions.

UPDATE 3 – Silence the Services Starting Messages

Thanks to the vmisev suggestion we can silence the FreeBSD boot process even more.

Just add rc_startmsgs=NO to your /etc/rc.conf file and reboot to see effects.

Here is already silenced boot process by my earlier settings.

rc_startmsgs_YES.jpg

After adding rc_startmsgs=NO to the /etc/rc.conf file the boot messages are cut in half.

rc_startmsgs_NO.jpg

Now its very close to Solaris/Illumos provides πŸ™‚

EOF

Valuable News – 2018/03/25

The IBM POWER9 chip is really big, check below by yourself:
https://twitter.com/ServeTheHome/status/976536374714155008

Interesting discussion about lack of Linux backward compatibility:
https://twitter.com/voretaq7/status/976223721475239937

Firefox Quantum (57+) with abandoned addons forces people to use insecure Firefox 56.x or outdated and slower Firefox ESR 52.x:
https://utcc.utoronto.ca/~cks/space/blog/web/Firefox56AddonWorry

FreeBSD defaults are not the best, here are some thoughts (updated) about them:
https://vez.mrsk.me/freebsd-defaults.txt

Great post by Cullum Smith about running FreeBSD on a laptop is available here:
https://www.c0ffee.net/blog/freebsd-on-a-laptop

If You ever wondered about weirdest hacks in production, You can check them here:
https://news.ycombinator.com/item?id=16669768

A list of all FreeBSD’s Ports maintainers is available here:
https://portscout.freebsd.org/

The Subtle Art of Not Giving a Fuck by Mark Manson:
https://markmanson.net/not-giving-a-fuck

EOF.

.
.
.
.
.

FreeBSD Network Management with network.sh Script

When You use only one connection on FreeBSD, then the best practice is to just put its whole configuration into the /etc/rc.conf file, for example typical server redundant connection would look like that one below.

cloned_interfaces="lagg0"
ifconfig_igb0="-lro -tso -vlanhwtag mtu 9000 up"
ifconfig_igb1="-lro -tso -vlanhwtag mtu 9000 up"
ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 up"
ifconfig_lagg0_alias0="inet 10.254.17.2/24"

If You must use more then one connection and You often switch between them, sometimes several times a day, then using the main FreeBSD’s config file is not the most convenient way for such operations.

For laptops where You often switch between WWAN (usually 3G connection) and WLAN (typical WiFi connection) and even LAN cable.

You can of course use graphical NetworkMgr from GhostBSD project which is described as “Python GTK3 network manager for FreeBSD, GhostBSD, TrueOS and DragonFlyBSD. NetworkMgr support both netif and OpenRC network” citing the project site – https://github.com/GhostBSD/networkmgr – it is also available in FreeBSD Ports and as package – net-mgmt/networkmgr.

GhostBSD-networkmgr

What I miss in NetworkMgr is the WWAN connection management, DNS management, optional random MAC generation and network shares unmount at disconnect from network. With my solution – network.sh – you still need to edit /etc/wpa_supplicant.conf and /etc/ppp/ppp.conf files by hand so it’s also not a perfect solution for typical desktop usage, but you do not edit these files every day.

As I use WWAN, WLAN and LAN connections on my laptop depends on the location I wrote a script to automate this connection management in a deterministic and convenient way, at least for me.

It can also set DNS to some safe/nologging providers or even a random safe DNS and generate legitimate MAC address for both LAN and WLAN if needed, even with real OUI first three octets if You also have additional network.sh.oui.txt file with them inside.

Here is the network.sh script help message.

% network.sh help 
USAGE:
  network.sh TYPE [OPTIONS]

TYPES:
  lan
  wlan
  wwan
  dns

OPTIONS:
  start
  start SSID|PROFILE
  stop
  example

EXAMPLES:
  network.sh lan start
  network.sh lan start IP.IP.IP.IP/MASK
  network.sh lan start IP.IP.IP.IP/MASK GW.GW.GW.GW
  network.sh lan restart
  network.sh wlan start
  network.sh wlan start HOME-NETWORK-SSID
  network.sh wwan example
  network.sh dns onic
  network.sh dns udns
  network.sh dns nextdns
  network.sh dns cloudflare
  network.sh dns ibm
  network.sh dns random
  network.sh dns IP.IP.IP.IP
  network.sh doas
  network.sh sudo
  network.sh status

If You run network.sh with appreciate arguments to start network connection it will display on the screen what commands it would run to achieve that. It also makes use of sudo(8) or doas(1) assuming that You are in the network group. To add yourself into the network group type this command below.

# pw groupmod network -m yourself

The network.sh doas command will print what rights it needs to work without root privileges, same for network.sh sudo command, an example below.

% network.sh doas
  # pw groupmod network -m YOURUSERNAME
  # cat /usr/local/etc/doas.conf
  permit nopass :network as root cmd /etc/rc.d/netif args onerestart
  permit nopass :network as root cmd /usr/sbin/service args squid onerestart
  permit nopass :network as root cmd dhclient
  permit nopass :network as root cmd ifconfig
  permit nopass :network as root cmd killall args -9 dhclient
  permit nopass :network as root cmd killall args -9 ppp
  permit nopass :network as root cmd killall args -9 wpa_supplicant
  permit nopass :network as root cmd ppp
  permit nopass :network as root cmd route
  permit nopass :network as root cmd tee args -a /etc/resolv.conf
  permit nopass :network as root cmd tee args /etc/resolv.conf
  permit nopass :network as root cmd umount
  permit nopass :network as root cmd wpa_supplicant

The network.sh script does not edit /usr/local/etc/doas.conf or /usr/local/etc/sudoers files, You have to put these lines there by yourself. An example doas setup for network.sh script is below.

# pkg install -y doas

# cat >> /usr/local/etc/doas.conf << __EOF
permit nopass :network as root cmd /etc/rc.d/netif args onerestart
permit nopass :network as root cmd /usr/sbin/service args squid onerestart
permit nopass :network as root cmd dhclient
permit nopass :network as root cmd ifconfig
permit nopass :network as root cmd killall args -9 dhclient
permit nopass :network as root cmd killall args -9 ppp
permit nopass :network as root cmd killall args -9 wpa_supplicant
permit nopass :network as root cmd ppp
permit nopass :network as root cmd route
permit nopass :network as root cmd tee args -a /etc/resolv.conf
permit nopass :network as root cmd tee args /etc/resolv.conf
permit nopass :network as root cmd umount
permit nopass :network as root cmd wpa_supplicant
__EOF
# 

# pw groupmod network -m yourself

The network.sh script upon disconnect would also forcefully unmount all network shares.

The idea is that it does only one connection type at a time, When You type network.sh lan start and then type network.sh wlan start, then it will reset entire FreeBSD network stack to defaults (to settings that are in /etc/rc.conf file) and then connect to WiFi in a ‘clean network environment’ as I could say. As I use 3 different methods of connecting to various networks I do not have any network settings in the /etc/rc.conf file, but You may prefer for example to have DHCP for local LAN enabled if that is more convenient for You.

The settings are on the beginning of the network.sh script, You should modify them to your needs and hardware that You own.

# SETTINGS
LAN_IF=em0
LAN_RANDOM_MAC=0
WLAN_IF=wlan0
WLAN_PH=iwn0
WLAN_RANDOM_MAC=0
WLAN_COUNTRY=PL
WLAN_REGDOMAIN=NONE
WWAN_IF=tun0
WWAN_PROFILE=WWAN
NAME=${0##*/}
NETFS="nfs,smbfs,fusefs.sshfs"
TIMEOUT=16
DELAY=0.5
SUDO_WHICH=0
SUDO=0
DOAS_WHICH=0
DOAS=1
ROOT=0

You can specify other NETFS filesystems that You want to forcefully unmount during network stop or set different physical WLAN adapter (WLAN_PH option), like ath0 for Atheros chips. similar for LAN interface which also defaults to Intel based network card with em0 driver (LAN_IF option).

If you want to disable random MAC address for LAN with LAN_RANDOM_MAC=0 and enable generation of random MAC address for WiFi networks with WLAN_RANDOM_MAC=1 option.

You should also decide if You want to use sudo (SUDO option) or doas (DOAS option).

Here is network.sh script.

Here is example of all network connections stop.

% network.sh stop
doas killall -9 wpa_supplicant
doas killall -9 ppp
doas killall -9 dhclient
doas ifconfig wlan0 destroy
doas ifconfig em0 down
echo | doas tee /etc/resolv.conf
doas /etc/rc.d/netif onerestart
%

Here is example of WLAN (or should I say WiFi) network connection start.

% network.sh wlan start
__network_reset()
__net_shares_umount()
doas killall -9 wpa_supplicant
doas killall -9 ppp
doas killall -9 dhclient
doas ifconfig em0 down
doas ifconfig wlan0 down
echo | doas tee /etc/resolv.conf
doas /etc/rc.d/netif restart
doas ifconfig wlan0 up
doas ifconfig wlan0 scan
doas ifconfig wlan0 ssid -
doas wpa_supplicant -i wlan0 -c /etc/wpa_supplicant.conf -s -B
__wlan_wait_associated()
doas dhclient -q wlan0
__dns_check_gateway()
echo | doas tee /etc/resolv.conf
echo 'nameserver 10.0.0.1' | doas tee -a /etc/resolv.conf
__dns_check()
__squid_restart()
doas ifconfig wlan0 powersave

Here is example od DNS change.

% network.sh dns ibm
echo | doas tee /etc/resolv.conf
echo 'nameserver 9.9.9.9' | doas tee -a /etc/resolv.conf

If You have any problems with the network.sh script then let me know, I will try to fix them ASAP.

If You are more into OpenBSD then FreeBSD then Vincent Delft wrote nmctlNetwork Manager Control tool for OpenBSD – available here – http://vincentdelft.be/post/post_20171023.

Ther is also another OpenBSD project by Aaron Poffenberger for network management – netctl – cli network-location manager for OpenBSD – available here – https://github.com/akpoff/netctl.

UPDATE 1 – Connect to Open/Unsecured WiFi Network

Recently when I was attending the Salt workshop during NLUUG Autumn Conference 2018 at Utrecht, Nederlands I wanted to connect to open unsecured WiFi network called 'Utrecht Hotel'. My phone of course attached to it instantly but on the other hand FreeBSD was not able to connect to it. As it turns out if you want to enable wpa_supplicant(8) to connect to open unsecured network a separate /etc/wpa_supplicant.conf option is needed (on option for all open unsecured
networks – no need to create such rule for each open/unsecured network).

Its these lines in the /etc/wpa_supplicant.conf file:

% grep -C 2 key_mgmt=NONE /etc/wpa_supplicant.conf

network={
        key_mgmt=NONE
        priority=0
}

I also modified the network.sh to contain that information in the examples section and also made little fix to always reset the previously set/forced SSID during earlier usage.

# ifconfig wlan0 ssid -

Now the network.sh should be even more pleasant to use.

UPDATE 2 – Openbox Integration

In on of the FreeBSD Desktop series articles I described how to setup Openbox window manager – FreeBSD Desktop – Part 12 – Configuration – Openbox – available here.

Below is an example of integration of that network.sh script with Openbox window manager.

network.sh.openbox.menu.jpg

… and here is the code used in the ~/.config/openbox/menu.xml file.

network.sh.openbox.menu.code

UPDATE 3 – Updated Status Page

I have jest added reworked status page to the network.sh script.

Its already updated in the GitHub ‘network’ repository:
https://github.com/vermaden/scripts/blob/master/network.sh

Here is how it looks.

network.sh.status.png

UPDATE 4 – Major Rework

After using network.sh script for a while I saw some needed changes. Time has come and I finally made them. I also find a problem when already about creating wlan0 virtual device from physical device (like iwn0).

When you start network.sh script for the first time and wlan0 is not yet created then the problem does not exists but when wlan0 already exists then network.sh waited for whopping 22 seconds on this single command. Now network.sh checks if the wlan0 device already exists which allows now WiFi connection in less then 3 seconds.

Before.

#DOAS# permit nopass :network as root cmd ifconfig
#SUDO# %network ALL = NOPASSWD: /sbin/ifconfig *
${CMD} ifconfig ${WLAN_IF} create wlandev ${WLAN_PH} 2> /dev/null
echo ${CMD} ifconfig ${WLAN_IF} create wlandev ${WLAN_PH}

After.

if ! ifconfig ${WLAN_IF} 1> /dev/null 2> /dev/null
then
  #DOAS# permit nopass :network as root cmd ifconfig
  #SUDO# %network ALL = NOPASSWD: /sbin/ifconfig *
  ${CMD} ifconfig ${WLAN_IF} create wlandev ${WLAN_PH} 2> /dev/null
  echo ${CMD} ifconfig ${WLAN_IF} create wlandev ${WLAN_PH}
fi

I used gnomon to benchmark the script execution.

Here is its simple installation process.

# pkg install -y npm
# npm install -g gnomon

Here is how it performed before the optiomization. About 25 seconds.

network.sh.SLOW.CREATE.before

And here is how it performs now. About 3 seconds.

network.sh.SLOW.CREATE.after

At first I suspected the /etc/rc.d/netif FreeBSD startup script but the real enemy was the tart ifconfig wlan0 create wlandev iwn0 command.

I also made it more verbose to better know where the time is wasted.

network.sh.more.verbous

Its now possible to set static IP and gateway in LAN mode and static IP in DNS mode.

network.sh.static.lan

The complete summary of changes and improvements is here:

  • Static IP address and gateway on LAN now possible.
  • Specify DNS by IP address.
  • Simplified __random_mac() function.
  • Fixed __wlan_wait_associated() function.
  • Removed unneded call for “create wlandev” in WLAN mode.
  • Other minor fixes.
  • WiFi (re)connection now possible under 3 seconds instead of 25+ seconds.

I also created a dedicated GitHub repository for network.sh script.

EOF

Read HFS filesystem on FreeBSD

Looking at the picture below from BSDCan 2008 FreeBSD Developer Summithttps://people.freebsd.org/~bz/200805DevSummit/ – You may get impression that FreeBSD developers does not use FreeBSD on their desktops and/or laptops, which was often true in 2008. Ten years later this seem to have changed and more FreeBSD developers use FreeBSD. At least more then in 2008 πŸ˜‰

BSDCan-2008-FreeBSD-Developer-Summit

For those ones that still use Mac OS X on their desktops/laptops instead of FreeBSD there is now possibility to mount HFS+ filesystem volumes under FreeBSD in read only thanks to FUSE subsystem. The project that allows this – FUSE driver for HFS+ filesystemshttps://github.com/0x09/hfsfuse – states that its “Created for use on FreeBSD which lacks a native driver (…)”.

Thanks to Tobias Kortkamp this driver is now available in the FreeBSD Ports tree as sysutils/fusefs-hfsfuse port.

The installation is simple:

# pkg install fusefs-hfsfuse

My buddy that uses Mac OS X created a new HFS+ volume for me and put a file on it, lets mount it for a test.

# gpart show da0
=>      34  15356093  da0  GPT  (7.3G)
        34         6       - free -  (3.0K)
        40    409600    1  efi  (200M)
    409640  14684336    2  apple-hfs  (7.0G)
  15093976    262151       - free -  (128M)

Lets verify that with file(1) command.

# file -s /dev/da0p2
/dev/da0p2: Macintosh HFS Extended version 4 data last mounted by: 'HFSJ', created: Wed Mar 14 16:59:41 2018, last modified: Wed Mar 14 15:00:44 2018, last checked: Wed Mar 14 14:59:41 2018, block size: 4096, number of blocks: 1835542, free blocks: 1827692

Yep, that’s the one, lets mount it.

# /usr/local/bin/hfsfuse --force -o noatime /dev/da0p2 /mnt

# mount
sys/ROOT/default on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
/dev/fuse on /mnt (fusefs.hfs, local, noatime, read-only, synchronous)

Now lets try to list and access its contents.

# ls /mnt
new-document.pdf
# cp /mnt/new-document.pdf ~
# echo $?
0
# ls ~/new-document.pdf
new-document.pdf
# mupdf new-document.pdf

Everything seems to work flawlessly. If time permits I would update sysutils/automount port to add HFS filesystem support for convenient automount.

EOF

Why Bareos forked from Bacula?

There are lots of reasons actually.

The situation between Bareos and Bacula is very similar to the situation of Nextcloud and OwnCloud. Bacula have open source free Community version and Enterprise paid version. In 2012 then Bacula Enterprise version was still open source but since 2012 its not. Some features from free Community version has been CUT/MOVED into Bacula Enterprise version only. People sent patches for Bacula Community edition but they were not merged for YEARS, so some developers got sick a tired of that situation and they created a fork named Bareos. All these reasons are described in details here: https://www.bareos.org/en/faq/why_fork.html

Currently Bareos has better web interface Bareos Webui – https://www.bareos.org/en/bareos-webui.html – which allows You to make RESTORE from any client to any client on that web interface. Bareos Webui also provides direct console to Bareos Server so You do not have to login using SSH separately. Bacula Web – http://www.bacula-web.org/home.html – on the other side provides monitoring read only capabilities, so You still have to use bconsole directly with Bacula.

What currently is implemented in Bareos is described here:
http://doc.bareos.org/master/html/bareos-manual-main-reference.html#WhatIsImplemented

Bareos has also grown features that are not available on Bacula, like storing backups on S3 object storage. You do not have to store backups in any public cloud directly, You may for example setup your own highly available distributed S3 storage on FreeBSD using Minio: https://docs.minio.io/docs/distributed-minio-quickstart-guide

Bareos also tries toΒ  bring some consistency and order to the configuration files:

# find /usr/local/etc/bareos -type d
/usr/local/etc/bareos
/usr/local/etc/bareos/bconsole.d
/usr/local/etc/bareos/bareos-dir.d
/usr/local/etc/bareos/bareos-dir.d/messages
/usr/local/etc/bareos/bareos-dir.d/client
/usr/local/etc/bareos/bareos-dir.d/catalog
/usr/local/etc/bareos/bareos-dir.d/director
/usr/local/etc/bareos/bareos-dir.d/pool
/usr/local/etc/bareos/bareos-dir.d/jobdefs
/usr/local/etc/bareos/bareos-dir.d/job
/usr/local/etc/bareos/bareos-dir.d/schedule
/usr/local/etc/bareos/bareos-dir.d/counter
/usr/local/etc/bareos/bareos-dir.d/console
/usr/local/etc/bareos/bareos-dir.d/storage
/usr/local/etc/bareos/bareos-dir.d/profile
/usr/local/etc/bareos/bareos-dir.d/fileset
/usr/local/etc/bareos/bareos-fd.d
/usr/local/etc/bareos/bareos-fd.d/client
/usr/local/etc/bareos/bareos-fd.d/director
/usr/local/etc/bareos/bareos-fd.d/messages
/usr/local/etc/bareos/bareos-sd.d
/usr/local/etc/bareos/bareos-sd.d/storage
/usr/local/etc/bareos/bareos-sd.d/ndmp
/usr/local/etc/bareos/bareos-sd.d/director
/usr/local/etc/bareos/bareos-sd.d/autochanger
/usr/local/etc/bareos/bareos-sd.d/device
/usr/local/etc/bareos/bareos-sd.d/messages

While Bacula model is split between pay and have everything in Bacula Enterprise or get free crippled Bacula Community the Bareos model is that You get EVERYTHING for free and you can buy subscription or integration/training services. With Bacula its also not possible to use it after Your subscription ends, its like with Oracle Solaris, You have to pay always just to use it, with Red Hat Linux You can buy the subscription once, and as it ends You can still use that Red Hat Linux system as long as You like, with Oracle Solaris or Bacula Enterprise You have to always pay.

Similar situation was between Nextcloud and OwnCloud. There was paid Enterprise OwnCloud version and crippled Community version of OwnCloud, some developers got sick an tired, some even left the OwnCloud company, even the creator or OwnCloud. They started Nextcloud with same model as Bareos, You get EVERYTHING for free and you can optionally buy subscription or integration/training services.

… and one more thing, You can actually get support/subscription for Bareos Server on FreeBSD from Bareos team. Its a custom contract currently, not listed on their site, but if You contact them they can provide that support for FreeBSD. Not possible with Bacula even with Bacula Enterprise version.

Bacula is dead, long live Bareos!

OwnCloud is dead, long live Nextcloud!

EOF

Valuable News – 2018/03/16

Its not a secret that I value the dragonflydigest.com articles like In Other BSDs and Lazy Reading series for providing great summary about weekly BSD news and other interesting stuff not always related to the BSD systems.

So I decided that I will also post valuable and interesting news that I found on the Internet once in a while. Here are my types for this ‘release’.

There is new FreeBSD variant on the block, welcome NomadBSD! Its project site is available on GitHub and its basically a live USB pendrive image intended for desktop usage.

In 1992 there was USL versus BSDi lawsuit about over intellectual property related to the Unix operating system, which was of course settled out of court in 1994 after the judge expressed doubt in the validity of USL’s intellectual property. Now | Linux beats legal threat from one of its own developers with one of the long term Linux kernel developers goes about various companies using Linux and made about $2 million in the process already.

Derek Sivers made a review list Books I’ve read about interesting books he read.

Not so recently FreeBSD’s Bhyve was ported to Illumos, now You can run OmniOSce as Bhyve guest on Illumos.

If You are a sentiment AMIGA person, there is nice article about Amiga Consciousness.

EOF.

.
.
.
.
.

Ghost in the Shell – Part 1

I wanted to post this earlier, but the busy daily life does not help πŸ˜‰

This will be first article in the series about efficient working in the shell environment. There are actually a lot articles and blog posts about efficient working in the terminal, but a lot of them are biased towards very specific uses, like hints only for Bash shell or only for specific terminal emulator. For example Moving efficiently in the CLI.

These series are about universal knowledge that would work on most shells and environments. Lets start with hint that I use many times a day that saves a lot time for not having to type …

You may want to check other articles in the Ghost in the Shell series on the Ghost in the Shell – Global Page where you will find links to all episodes of the series along with table of contents for each episode’s contents.

Recall Last Argument of Previous Command

Imagine most simple scenario, creating directory and entering it. Typically its like that:

% mkdir clear-place-for-new-work
% cd clear-place-for-new-work
%

The longer the name, the bigger the chance that You would type mkdir, then hit the [UP] arrow, then [HOME] or [CTRL]+[A] keys and then put cd in the place of mkdir.

With the use of !$ You can recall last argument of the precious command, so it will now look like that.

% mkdir clear-place-for-new-work
% cd !$
cd clear-place-for-new-work
%

Faster isn’t it?

Swap First Occurrence of a Word

The upper example can be used for the next advice as well. By typing ^fromwhat^towhat in the terminal You will swap the first occurrence of word fromwhat word to towhat word in the previous command, lets see how its working.

% mkdir clear-place-for-new-work
% ^mkdir^cd
cd clear-place-for-new-work
%

It still takes more time to write then using the !$ so its useful mostly when there are short things to swap, like numbers, for example ^3^4 to ‘move’ from one target to another. … or also if You can not recall to the last argument of previous command.

There and Back Again

A lot people does not know, that You can go back to previous working directory with dash. Lets assume that You need to get to /tmp directory for one command and get back to where You were to continue the work. Here is an example.

% pwd
/usr/local/etc/bareos/bareos-dir.d/jobdefs
% cd /tmp
% pwd
/tmp% (do needed work in /tmp dir)
% cd -
/usr/local/etc/bareos/bareos-dir.d/jobdefs
% pwd
/usr/local/etc/bareos/bareos-dir.d/jobdefs

You can even create entire directory stack with pushd/popd commands if needed, check Wikipedia article on that for more information. You can also use ${OLDPWD} variable. Useful with umount command for example.

% pwd
/media/backup-pendrive-key
% cd ~
% umount $OLDPWD
% pwd
/home/vermaden

Repeat Command from History

With exclamation mark (!) You can re-invoke the command from history with all its arguments (which sometimes can be risky). For example.

% !pkg
pkg update -f
(runs actual command)
%

Its better to first check what arguments have been used in that command, that is where :p comes handy. Here is its example usage.

% !pkg:p
pkg update -f
(just prints command without running it)
% !pkg
pkg update -f
(runs actual command)
%

Now, as arguments are known its safe to re-invoke the command with arguments. When this can be dangerous? Can ls command can be dangerous, that depends what You have on Your history, check the example below.

% ls | while read I; do rm -f ${I}; done

This command first lists the contents of the current working directory with ls command, then the output is piped to the while loop which invokes rm -f command for each item listed by ls command, which efficiently removes all non-hidden files in current working directory … which probably is not what we mean by typing !ls on the command prompt ;). That is why its valuable to first check what arguments were used with !ls:p syntax.

Enough for now, I will write more parts with more hints on how to efficiently work in the shell/terminal environment.

UPDATE 1

The Ghost in the Shell – Part 1 article was included in the BSD Now 241 – Bowling in the LimeLight episode.

Thanks for mentioning!

UPDATE 2

About Recall last argument of previous command section … there is also $_ that does similar thing as !$ but there is little difference. The !$ is ‘line oriented’ while $_ is ‘previous command oriented’. Below is an example that shows the difference in the behavior.

The $! takes value from last command in ‘previous line’ which means that '-l' value will be used from line 001 and not 'asd' from the current line 002 from previously executed command.

001 % ls -l
002 % echo asd; ls !$ | tail -2
echo asd; ls -l | tail -2
asd
// ls output //

The $_ takes value from last executed command, thus it points at 'asd' used on line 002 and not at '-l' used at previous 001 line.

001 % ls -l
002 % echo asd; ls $_ | tail -2
asd
ls: asd: No such file or directory

On BASH shell there is also [ALT]-[.] shortcut that switches between $! from previous lines. To achieve the same shortcut on ZSH use this line below in ZSH config.

bindkey '\e.' insert-last-word

Thank you Zachery Purnell for pointing that out.

EOF