Tag Archives: xterm

Desktop Environments Resource Usage Comparison

Some of them use more RAM. Some less. Today in a rather simplified benchmark I will check some popular desktop environments for their RAM usage. I recently came to see some more or less old comparisons of various desktop environments RAM usage.

2019 – https://youtu.be/fo45bo1jvZI
2020 – https://youtu.be/RrvJOXypAbk
2022 – https://itvision.altervista.org/linux-desktop-environments-system-usage.html [edit]

They were focused on difference between XFCE and KDE/Plasma environments. I am used to idea that XFCE is smaller and lighter of the two – so it should be also lighter on resources – but these two movies state that they RAM usage is similar and sometimes even KDE/Plasma is lighter. These results seemed strange to me so I wanted to test them under latest FreeBSD 13.1-RELEASE UNIX system.

Example XFCE on FreeBSD desktop screenshot from the XFCE Cupertino Way article.

xfce-ghostbsd

Upon some popular demand I also added GNOME (the 42 version) to the comparison.

Today we will test these desktop environments:

XFCE (4.16)
MATE (1.26)
KDE/Plasma (5.24)
Openbox (3.6)
GNOME (42)

We all know that Openbox is just a window manager but I wanted to include it here just from comparison.

Test Environment and Process

To save time I used VirtualBox virtual machine for the purpose of these simplified benchmarks. For that purpose he created VM had:

1 x CPU
8 GB RAM
128 MB GPU Memory
30 GB Disk

After installing the vanilla FreeBSD 13.1-RELEASE I switched to the latest pkg(8) repository. Then I added needed packages:

# pkg install xorg xfce kde5 mate openbox dzen2 tint2 xbindkeys xterm geany gnome

All of the desktop environments and their dependencies were installed on that test machine. The main FreeBSD config at /etc/rc.conf file had following contents.

% cat /etc/rc.conf
hostname="freebsd"
ifconfig_em0="DHCP"
sshd_enable="YES"
moused_enable="YES"
powerd_enable="YES"
dumpdev="AUTO"
zfs_enable="YES"
dbus_enable="YES"

The only thing I added after installation was the dbus service startup. I did not changed any settings in these environments. The were compared at their default settings.

The test was rather simple and naive but these were the tasks that I done on each of them.

Run gstat(8) command in terminal application.
Display /etc/ in file manager with scroll to end of display of dir.
Open /etc/ssh/moduli file in text editor with scroll to end of file.

These were different for various environments:

XFCE

xfce4-terminal
thunar
mousepad

MATE

mate-terminal
caja
pluma

KDE/Plasma

konsole
dolphin
kate

Openbox

xterm
caja
geany

GNOME

gnome-terminal
nautilus
gedit

I powered off that FreeBSD machine before each test – so each test looked like:

boot cold FreeBSD system
login into system (in text console)
type xinit(1) command
do the 3 defined tasks

Each desktop environment had different ~/.xinitrc file. Below you will find their contents.

% cat ~/.xinitrc.xfce
. /usr/local/etc/xdg/xfce4/xinitrc

% cat ~/.xinitrc.mate
exec ck-launch-session mate-session

% cat ~/.xinitrc.kde
exec ck-launch-session startplasma-x11

% cat ~/.xinitrc.openbox
dzen2 &
tint2 &
xbindkeys &
exec openbox

% cat ~/.xinitrc.gnome
exec gnome-session

Each of them were started like that:

% xinit ~/.xinitrc.xfce

% xinit ~/.xinitrc.mate

% xinit ~/.xinitrc.kde

% xinit ~/.xinitrc.openbox

% xinit ~/.xinitrc.gnome

RAM Usage Results

To be honest I was surprised by the results.

Clean Text Console FreeBSD

The text console of FreeBSD 13.1-RELEASE system used about 97 MB of RAM. That result is the sum of the RES column from the top(1) command.

Below you will find the top(1) output for FreeBSD text console only system.

% top -b -o res 1000
last pid:   871;  load averages:  1.92,  0.90,  0.36; battery: 99%  up 0+00:01:09    00:34:01
28 processes:  2 running, 26 sleeping
CPU:  2.0% user,  0.0% nice,  3.7% system,  0.2% interrupt, 94.0% idle
Mem: 18M Active, 21M Inact, 138M Wired, 40K Buf, 7746M Free
ARC: 43M Total, 18M MFU, 23M MRU, 335K Header, 1556K Other
     20M Compressed, 61M Uncompressed, 3.09:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  852 vermaden      1  20    0    21M  9492K RUN      0:00   0.00% sshd
  849 root          1  33    0    21M  9300K select   0:00   0.00% sshd
  799 root          1  22    0    21M  8208K select   0:00   0.00% sshd
  817 root          1  20    0    18M  7140K select   0:00   0.00% sendmail
  820 smmsp         1  52    0    18M  6704K pause    0:00   0.00% sendmail
  749 messagebus    1  52    0    14M  3648K select   0:00   0.00% dbus-daemon
  853 vermaden      1  20    0    13M  3256K wait     0:00   0.00% sh
  871 vermaden      1  20    0    14M  3220K RUN      0:00   0.00% top
  846 vermaden      1  52    0    13M  3208K ttyin    0:00   0.00% sh
  838 root          1  25    0    13M  3100K wait     0:00   0.00% login
  463 _dhcp         1  52    0    13M  2828K select   0:00   0.00% dhclient
  668 root          1  20    0    13M  2748K select   0:00   0.00% syslogd
  830 root          1  52    0    13M  2736K wait     0:00   0.00% sh
  402 root          1  52    0    13M  2708K select   0:00   0.00% dhclient
  399 root          1  52    0    13M  2632K select   0:00   0.00% dhclient
  802 root          1  20    0    13M  2516K nanslp   0:00   0.00% cron
  831 root          1  52    0    13M  2440K piperd   0:00   0.00% logger
  754 root          1  52    0    13M  2380K select   0:00   0.00% moused
  837 root          1  52    0    13M  2316K select   0:00   0.00% logger
  842 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  845 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  843 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  844 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  841 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  839 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  840 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  833 root          1  52    0    12M  2080K nanslp   0:00   0.00% sleep
  464 root          1  20    0    11M  1540K select   0:00   0.00% devd

XFCE

Next one is XFCE and it used about 1548 MB of RAM.

Below you will find the top(1) output for XFCE.

% top -b -o res 1000
last pid:  1076;  load averages:  0.58,  0.84,  0.51; battery: 99%  up 0+00:07:06    00:31:07
71 processes:  2 running, 69 sleeping
CPU:  7.6% user,  0.1% nice,  6.5% system,  1.1% interrupt, 84.8% idle
Mem: 292M Active, 337M Inact, 389M Wired, 56K Buf, 6897M Free
ARC: 240M Total, 98M MFU, 133M MRU, 1762K Header, 7212K Other
     194M Compressed, 461M Uncompressed, 2.37:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  945 vermaden      3  20    0   344M   251M select   0:11   0.00% Xorg
 1010 vermaden      4  20    0   311M   121M select   0:01   0.00% kgpg
 1004 vermaden      5  20    0   196M   100M select   0:04   0.00% xfwm4
 1008 vermaden      4  20    0   130M    92M select   0:01   0.00% xfdesktop
  948 vermaden      4  20    0   172M    76M select   0:04   0.00% xfce4-session
 1012 vermaden      6  40   19   160M    63M select   0:00   0.00% tumblerd
 1064 vermaden      5  21    0    89M    59M select   0:05   0.00% mousepad
 1013 vermaden      3  20    0   130M    52M select   0:00   0.00% kalendarac
 1007 vermaden      4  24    0    75M    51M select   0:04   0.00% thunar
 1006 vermaden      4  20    0    75M    48M select   0:04   0.00% xfce4-panel
 1056 vermaden      4  20    0    69M    42M select   0:01   0.00% xfce4-terminal
 1020 vermaden      4  20    0    65M    41M select   0:00   0.00% wrapper-2.0
 1021 vermaden      4  20    0    65M    41M select   0:00   0.00% wrapper-2.0
 1022 vermaden      4  20    0    52M    32M select   0:00   0.00% wrapper-2.0
 1005 vermaden      4  20    0    49M    30M select   0:02   0.00% xfsettingsd
 1019 vermaden      4  20    0    46M    30M select   0:00   0.00% wrapper-2.0
 1027 vermaden      3  40   19   256G    29M select   0:00   0.00% baloo_file
 1009 vermaden      4  20    0    46M    28M select   0:00   0.00% xfce4-power-manager
  975 polkitd       7  20    0  2125M    27M select   0:01   0.00% polkitd
 1029 vermaden      4  20    0    45M    27M select   0:00   0.00% xfce4-notifyd
  977 vermaden      5  20    0    49M    26M select   0:01   0.00% mate-screensaver
  983 root          7  20    0    64M    16M select   0:01   0.00% bsdisks
  981 vermaden      5  20    0    27M    11M select   0:00   0.00% gvfs-udisks2-volume
 1067 vermaden      5  20    0    24M    10M select   0:00   0.00% gvfsd-network
 1038 vermaden      4  20    0    27M    10M select   0:00   0.00% gvfsd-trash
 1070 vermaden      4  20    0    24M    10M select   0:00   0.00% gvfsd-dnssd
 1063 vermaden      4  20    0    24M    10M select   0:00   0.00% gvfsd-computer
  865 vermaden      1  20    0    21M  9492K RUN      0:00   0.00% sshd
 1042 vermaden      2  22    0    86M  9440K select   0:00   0.00% pulseaudio
  862 root          1  28    0    21M  9264K select   0:00   0.00% sshd
  979 vermaden      4  32    0    24M  8836K select   0:00   0.00% gvfsd
  973 vermaden      4  20    0    21M  8712K select   0:00   0.00% at-spi2-registryd
  966 vermaden      5  20    0    21M  8296K select   0:00   0.00% at-spi-bus-launcher
  972 root         16  20    0    24M  8256K select   0:00   0.00% console-kit-daemon
  815 root          1  22    0    21M  8208K select   0:00   0.00% sshd
  991 vermaden      5  20    0    21M  7948K select   0:00   0.00% gvfs-gphoto2-volume
 1044 root          4  22    0    20M  7916K select   0:00   0.00% accounts-daemon
 1040 vermaden      4  20    0    19M  7460K select   0:00   0.00% gvfsd-metadata
 1017 root          4  20    0    19M  7452K select   0:00   0.00% upowerd
  988 vermaden      5  20    0    19M  7208K select   0:00   0.00% gvfs-mtp-volume-mon
  833 root          1  20    0    18M  7140K select   0:00   0.00% sendmail
 1066 vermaden      4  20    0    19M  7004K select   0:00   0.00% dconf-service
  969 vermaden      4  20    0    19M  6936K select   0:00   0.00% xfconfd
  998 vermaden      1  21    0    18M  6900K select   0:00   0.00% ssh-agent
  836 smmsp         1  52    0    18M  6576K pause    0:00   0.00% sendmail
  960 vermaden      1  20    0    14M  4580K select   0:01   0.00% dbus-daemon
 1003 vermaden      1  20    0    16M  4116K select   0:00   0.00% gpg-agent
  765 messagebus    1  20    0    14M  4100K select   0:00   0.00% dbus-daemon
  955 vermaden      1  23    0    15M  3912K select   0:00   0.00% dbus-launch
  967 vermaden      1  20    0    14M  3812K select   0:01   0.00% dbus-daemon
 1058 vermaden      1  20    0    14M  3772K nanslp   0:00   0.00% gstat
 1076 vermaden      1  20    0    14M  3464K RUN      0:00   0.00% top
 1057 vermaden      1  28    0    13M  3276K wait     0:00   0.00% sh
  866 vermaden      1  20    0    13M  3256K wait     0:00   0.00% sh
  941 vermaden      1  20    0    13M  3212K wait     0:00   0.00% sh
  854 root          1  20    0    13M  3136K wait     0:00   0.00% login
  944 vermaden      1  20    0    14M  3096K wait     0:00   0.00% xinit
  479 _dhcp         1  52    0    13M  2828K select   0:00   0.00% dhclient
  684 root          1  20    0    13M  2748K select   0:00   0.00% syslogd
  418 root          1   4    0    13M  2708K select   0:00   0.00% dhclient
  415 root          1  49    0    13M  2632K select   0:00   0.00% dhclient
  818 root          1  20    0    13M  2516K nanslp   0:00   0.00% cron
  770 root          1  20    0    13M  2404K select   0:00   0.00% moused
  855 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  858 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  861 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  859 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  860 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  856 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  857 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  480 root          1  20    0    11M  1540K select   0:00   0.00% devd

MATE

Next one is MATE and it used about 1361 MB of RAM.

This is first strange thing for me. Keep in mind that MATE is a fork of GNOME 2 which was expected to be heavy compared to light XFCE … at least more then a decade ago. Seems that now MATE developers are doing better job then XFCE devs 🙂

Below you will find the top(1) output for MATE.

% top -b -o res 1000
last pid:   966;  load averages:  1.75,  1.02,  0.43; battery: 99%  up 0+00:01:53    00:40:42
66 processes:  2 running, 64 sleeping
CPU: 25.7% user,  0.0% nice,  8.9% system,  0.4% interrupt, 65.0% idle
Mem: 279M Active, 269M Inact, 381M Wired, 56K Buf, 6986M Free
ARC: 230M Total, 88M MFU, 131M MRU, 1753K Header, 8250K Other
     183M Compressed, 435M Uncompressed, 2.37:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  857 vermaden      3  23    0   344M   246M select   0:08   7.96% Xorg
  922 vermaden      4  20    0   311M   122M select   0:01   0.00% kgpg
  869 vermaden      5  20    0   172M    77M select   0:01   0.00% mate-session
  913 vermaden      6  20    0    92M    68M select   0:03   0.00% caja
  961 vermaden      5  29    0    91M    59M select   0:07  15.97% pluma
  951 vermaden      5  20    0    86M    55M select   0:01   0.00% mate-terminal
  919 vermaden      3  20    0   130M    52M select   0:00   0.00% kalendarac
  911 vermaden      5  20    0    74M    49M select   0:01   0.00% mate-panel
  941 vermaden      5  20    0    70M    45M select   0:00   0.00% notification-area-a
  902 vermaden      5  20    0    70M    44M select   0:01   0.00% marco
  917 vermaden      4  20    0    66M    43M select   0:00   0.00% mate-volume-control
  899 vermaden      6  20    0    60M    38M select   0:02   0.00% mate-settings-daemo
  939 vermaden      5  20    0    60M    38M select   0:00   0.00% clock-applet
  927 vermaden      5  20    0    57M    37M select   0:00   0.00% wnck-applet
  921 vermaden      5  20    0    55M    35M select   0:00   0.00% mate-power-manager
  915 vermaden      5  20    0    50M    32M select   0:00   0.00% mate-screensaver
  864 polkitd       7  20    0  2125M    27M select   0:00   0.00% polkitd
  914 vermaden      4  20    0    44M    26M select   0:00   0.00% polkit-mate-authent
  883 root          7  52    0    64M    16M select   0:00   0.00% bsdisks
  881 vermaden      5  20    0    27M    12M select   0:00   0.00% gvfs-udisks2-volume
  962 vermaden      5  20    0    24M    11M select   0:00   0.00% gvfsd-network
  965 vermaden      4  20    0    24M    10M select   0:00   0.00% gvfsd-dnssd
  954 vermaden      4  20    0    24M    10M select   0:00   0.00% gvfsd-computer
  929 vermaden      4  20    0    26M    10M select   0:00   0.00% gvfsd-trash
  852 vermaden      1  20    0    21M  9480K RUN      0:00   0.00% sshd
  931 vermaden      2  21    0    86M  9396K select   0:00   0.00% pulseaudio
  849 root          1  30    0    21M  9300K select   0:00   0.00% sshd
  879 vermaden      4  28    0    24M  9180K select   0:00   0.00% gvfsd
  901 vermaden      4  20    0    21M  8860K select   0:00   0.00% at-spi2-registryd
  895 vermaden      5  20    0    21M  8272K select   0:00   0.00% gvfs-gphoto2-volume
  862 root         16  20    0    24M  8244K select   0:00   0.00% console-kit-daemon
  799 root          1  23    0    21M  8208K select   0:00   0.00% sshd
  875 vermaden      5  20    0    21M  8128K select   0:00   0.00% at-spi-bus-launcher
  956 vermaden      4  20    0    19M  7704K select   0:00   0.00% gvfsd-metadata
  893 vermaden      5  20    0    19M  7544K select   0:00   0.00% gvfs-mtp-volume-mon
  924 root          4  20    0    19M  7524K select   0:00   0.00% upowerd
  817 root          1  20    0    18M  7140K select   0:00   0.00% sendmail
  897 vermaden      4  20    0    19M  6936K select   0:00   0.00% dconf-service
  820 smmsp         1  52    0    18M  6700K pause    0:00   0.00% sendmail
  912 vermaden      1  20    0    17M  4892K piperd   0:00   0.00% libgtop_server2
  873 vermaden      1  20    0    14M  4164K select   0:00   0.00% dbus-daemon
  860 vermaden      1  22    0    17M  4124K wait     0:00   0.00% ck-launch-session
  876 vermaden      1  20    0    14M  4004K select   0:01   0.00% dbus-daemon
  749 messagebus    1  20    0    14M  3984K select   0:00   0.00% dbus-daemon
  872 vermaden      1  20    0    15M  3912K select   0:00   0.00% dbus-launch
  953 vermaden      1  20    0    14M  3708K nanslp   0:00   0.00% gstat
  966 vermaden      1  20    0    14M  3392K RUN      0:00   0.00% top
  853 vermaden      1  20    0    13M  3248K wait     0:00   0.00% sh
  846 vermaden      1  21    0    13M  3212K wait     0:00   0.00% sh
  952 vermaden      1  38    0    13M  3208K wait     0:00   0.00% sh
  838 root          1  25    0    13M  3100K wait     0:00   0.00% login
  856 vermaden      1  20    0    14M  3096K wait     0:00   0.00% xinit
  463 _dhcp         1  52    0    13M  2828K select   0:00   0.00% dhclient
  668 root          1  20    0    13M  2748K select   0:00   0.00% syslogd
  402 root          1  52    0    13M  2708K select   0:00   0.00% dhclient
  399 root          1  52    0    13M  2632K select   0:00   0.00% dhclient
  802 root          1  20    0    13M  2516K nanslp   0:00   0.00% cron
  754 root          1  20    0    13M  2404K select   0:00   0.00% moused
  839 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  845 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  841 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  843 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  842 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  844 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  840 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  464 root          1  20    0    11M  1540K select   0:00   0.00% devd

KDE/Plasma

Next one is KDE/Plasma and without surprise (at least for me) it uses more RAM then other desktop environments – about 2843 MB of RAM – that is more then twice as much as MATE and almost twice as much as XFCE.

Below you will find the top(1) output for KDE/Plasma.

% top -b -o res 1000
last pid:  1075;  load averages:  2.10,  1.56,  0.79; battery: 99%  up 0+00:05:22    00:38:14
67 processes:  2 running, 65 sleeping
CPU: 30.8% user,  0.1% nice,  8.8% system,  0.2% interrupt, 60.0% idle
Mem: 530M Active, 316M Inact, 441M Wired, 56K Buf, 6633M Free
ARC: 272M Total, 119M MFU, 139M MRU, 2012K Header, 12M Other
     211M Compressed, 514M Uncompressed, 2.44:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  935 vermaden     12  21    0   588M   298M select   0:22   0.00% plasmashell
  874 vermaden      3  23    0   344M   241M select   0:15   9.96% Xorg
 1065 vermaden      7  30    0   365M   190M select   0:12  15.97% kate
  918 vermaden      5  31    0   376M   179M select   0:22  15.97% kwin_x11
 1035 vermaden      8  20    0   320M   156M select   0:03   0.00% dolphin
 1029 vermaden      3  20    0   312M   150M select   0:02   0.00% konsole
  959 vermaden      4  20    0   314M   143M select   0:01   0.00% kgpg
 1063 vermaden      5  52    0   304M   140M select   0:02   0.00% kioslave5
 1073 vermaden      4  20    0   303M   139M select   0:01   0.00% kioslave5
  916 vermaden     11  20    0   179M    84M select   0:02   0.00% kded5
  958 vermaden      3  20    0   147M    67M select   0:01   0.00% kalendarac
  944 vermaden      4  20    0   174M    63M select   0:01   0.00% DiscoverNotifier
  941 vermaden      6  20    0   130M    58M select   0:01   0.00% polkit-kde-authenti
  920 vermaden      4  20    0   131M    58M select   0:01   0.00% ksmserver
  940 vermaden      7  20    0   118M    56M select   0:01   0.00% org_kde_powerdevil
  942 vermaden      3  20    0   128M    56M select   0:01   0.00% kaccess
  922 vermaden      3  20    0   127M    55M select   0:01   0.00% kglobalaccel5
  968 vermaden      7  20    0   104M    47M select   0:01   0.00% kactivitymanagerd
  905 vermaden      3  20    0   127M    45M select   0:00   0.00% klauncher
  901 vermaden      3  26    0   113M    43M select   0:00   0.00% plasma_session
  904 vermaden      1  20    0   112M    41M select   0:00   0.00% kdeinit5
  885 vermaden      3  38    0   113M    41M select   0:00   0.00% startplasma-x11
 1041 vermaden      2  42    0    88M    37M select   0:00   0.00% kioslave5
 1069 vermaden      1  23    0   256G    37M select   0:00   0.00% kioslave5
 1039 vermaden      1  36    0   256G    37M select   0:00   0.00% kioslave5
 1027 vermaden      1  36    0   256G    36M select   0:00   0.00% kioslave5
  997 vermaden      3  28    0    86M    35M select   0:00   0.00% kioslave5
  943 vermaden      3  40   19   256G    32M select   0:00   0.00% baloo_file
  945 vermaden      4  20    0    87M    32M select   0:00   0.00% gmenudbusmenuproxy
  939 vermaden      3  20    0    83M    30M select   0:00   0.00% xembedsniproxy
  976 vermaden      3  20    0    76M    28M select   0:00   0.00% kscreen_backend_lau
  881 polkitd       7  20    0  2125M    27M select   0:00   0.00% polkitd
  926 root          7  20    0    67M    19M select   0:00   0.00% bsdisks
  966 vermaden      2  20    0    86M  9520K select   0:00   0.00% pulseaudio
  852 vermaden      1  20    0    21M  9512K RUN      0:00   0.00% sshd
  849 root          1  33    0    21M  9300K select   0:00   0.00% sshd
  879 root         16  20    0    24M  8320K select   0:00   0.00% console-kit-daemon
  799 root          1  22    0    21M  8208K select   0:00   0.00% sshd
  937 root          4  20    0    19M  7404K select   0:00   0.00% upowerd
  817 root          1  20    0    18M  7140K select   0:00   0.00% sendmail
  982 vermaden      4  20    0    19M  6732K select   0:00   0.00% dconf-service
  820 smmsp         1  52    0    18M  6704K pause    0:00   0.00% sendmail
  749 messagebus    1  20    0    14M  4452K select   0:00   0.00% dbus-daemon
  896 vermaden      1  28    0    15M  4252K select   0:00   0.00% dbus-launch
  897 vermaden      1  20    0    14M  4164K select   0:01   0.00% dbus-daemon
  877 vermaden      1  21    0    17M  4124K wait     0:00   0.00% ck-launch-session
 1034 vermaden      1  20    0    14M  3836K nanslp   0:00   0.00% gstat
 1075 vermaden      1  20    0    14M  3392K RUN      0:00   0.00% top
  853 vermaden      1  20    0    13M  3256K wait     0:00   0.00% sh
 1032 vermaden      1  26    0    13M  3232K wait     0:00   0.00% sh
  846 vermaden      1  20    0    13M  3212K wait     0:00   0.00% sh
  838 root          1  25    0    13M  3100K wait     0:00   0.00% login
  873 vermaden      1  20    0    14M  3096K wait     0:00   0.00% xinit
  463 _dhcp         1  52    0    13M  2828K select   0:00   0.00% dhclient
  668 root          1  20    0    13M  2748K select   0:00   0.00% syslogd
  402 root          1  52    0    13M  2708K select   0:00   0.00% dhclient
  399 root          1  52    0    13M  2632K select   0:00   0.00% dhclient
  802 root          1  20    0    13M  2516K nanslp   0:00   0.00% cron
  754 root          1  20    0    13M  2404K select   0:01   0.00% moused
  842 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  845 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  843 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  844 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  841 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  839 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  840 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  464 root          1  20    0    11M  1540K select   0:00   0.00% devd

Openbox

Not really a desktop environment but just for the sake of comparison I wanted to check it. With the default ‘ugly’ settings it consumed about 614 MB or RAM.

Below you will find the top(1) output for Openbox.

% top -b -o res 1000
last pid:   991;  load averages:  0.66,  0.77,  0.43; battery: 99%  up 0+00:04:35    00:52:31
43 processes:  1 running, 41 sleeping, 1 stopped
CPU:  8.8% user,  0.0% nice,  3.8% system,  0.5% interrupt, 86.9% idle
Mem: 126M Active, 196M Inact, 391M Wired, 40K Buf, 7210M Free
ARC: 210M Total, 78M MFU, 120M MRU, 1783K Header, 10M Other
     164M Compressed, 374M Uncompressed, 2.28:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  894 vermaden      3  20    0   307M   217M select   0:02   0.00% Xorg
  942 vermaden      6  20    0   208M   112M select   0:02   0.00% caja
  981 vermaden      3  20    0   100M    60M select   0:02   0.00% geany
  897 vermaden      1  20    0    54M    25M select   0:00   0.00% openbox
  898 vermaden      1  20    0    53M    25M select   0:01   0.00% tint2
  939 vermaden      1  20    0    25M    13M select   0:00   0.00% xterm
  916 vermaden      1  20    0    25M    13M select   0:00   0.00% xterm
  986 vermaden      1  20    0    21M  9500K select   0:00   0.00% sshd
  983 root          1  28    0    21M  9360K select   0:00   0.00% sshd
  953 vermaden      4  20    0    21M  9228K select   0:00   0.00% at-spi2-registryd
  949 vermaden      5  49    0    21M  8736K select   0:00   0.00% at-spi-bus-launcher
  934 vermaden      1  20    0    20M  8608K STOP     0:00   0.00% dzen2
  799 root          1  20    0    21M  8208K select   0:00   0.00% sshd
  817 root          1  20    0    18M  7140K select   0:00   0.00% sendmail
  955 vermaden      4  30    0    19M  6916K select   0:00   0.00% dconf-service
  820 smmsp         1  52    0    18M  6636K pause    0:00   0.00% sendmail
  946 vermaden      1  30    0    15M  4380K select   0:00   0.00% dbus-launch
  947 vermaden      1  43    0    14M  3908K select   0:00   0.00% dbus-daemon
  937 vermaden      1  20    0    14M  3760K nanslp   0:00   0.00% gstat
  950 vermaden      1  20    0    14M  3732K select   0:00   0.00% dbus-daemon
  749 messagebus    1  52    0    14M  3648K select   0:00   0.00% dbus-daemon
  991 vermaden      1  22    0    14M  3428K RUN      0:00   0.00% top
  987 vermaden      1  21    0    13M  3316K wait     0:00   0.00% sh
  918 vermaden      1  20    0    13M  3292K wait     0:00   0.00% sh
  941 vermaden      1  22    0    13M  3280K wait     0:00   0.00% sh
  982 vermaden      1  52    0    13M  3272K ttyin    0:00   0.00% sh
  846 vermaden      1  20    0    13M  3212K wait     0:00   0.00% sh
  838 root          1  26    0    13M  3100K wait     0:00   0.00% login
  893 vermaden      1  20    0    14M  3096K wait     0:00   0.00% xinit
  463 _dhcp         1  52    0    13M  2828K select   0:00   0.00% dhclient
  668 root          1  20    0    13M  2748K select   0:00   0.00% syslogd
  402 root          1   4    0    13M  2708K select   0:00   0.00% dhclient
  399 root          1  52    0    13M  2632K select   0:00   0.00% dhclient
  802 root          1  20    0    13M  2516K nanslp   0:00   0.00% cron
  754 root          1  20    0    13M  2404K select   0:00   0.00% moused
  843 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  842 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  845 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  844 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  840 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  839 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  841 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  464 root          1  20    0    11M  1540K select   0:00   0.00% devd

GNOME

GNOME with the same test procedure used 2622 MB of RAM.

Below you will find the top(1) output for GNOME.

% top -b -o res 1000
last pid:  1114;  load averages:  2.62,  1.76,  0.81; battery: 99%  up 0+00:03:38    12:44:58
91 processes:  2 running, 89 sleeping
CPU: 45.9% user,  0.0% nice,  9.1% system,  0.3% interrupt, 44.7% idle
Mem: 531M Active, 560M Inact, 2152K Laundry, 522M Wired, 56K Buf, 6295M Free
ARC: 319M Total, 151M MFU, 156M MRU, 2354K Header, 9740K Other
     266M Compressed, 640M Uncompressed, 2.41:1 Ratio
Swap: 2048M Total, 2048M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
  904 vermaden      9  23    0  2656M   379M select   0:29   9.96% gnome-shell
  855 vermaden      3  22    0   333M   238M select   0:10   6.98% Xorg
 1040 vermaden     12  20    0   349M   192M select   0:10   0.00% epiphany-search-pro
  962 vermaden      4  20    0   312M   136M select   0:01   0.00% kgpg
 1026 vermaden      8  20    0   215M   125M select   0:01   0.00% gnome-calendar
 1107 vermaden      5  20    0   187M   108M select   0:01   0.00% gnome-control-cente
  958 vermaden      7  20    0   211M    81M select   0:01   0.00% evolution-alarm-not
 1058 vermaden     15  20    0   194M    78M select   0:01   0.00% WebKitNetworkProces
 1071 vermaden      6  20    0   144M    77M select   0:03   0.00% nautilus
 1112 vermaden      5  52    0    85M    59M select   0:13  19.97% gedit
 1065 vermaden      5  20    0   114M    56M select   0:01   0.00% gnome-terminal-serv
  961 vermaden      3  20    0   132M    53M select   0:00   0.00% kalendarac
  917 vermaden      5  20    0   146M    51M select   0:00   0.00% goa-daemon
 1034 vermaden      5  20    0    70M    50M select   0:00   0.00% seahorse
  921 vermaden      7  20    0   109M    48M select   0:00   0.00% evolution-addressbo
  919 vermaden     10  20    0    81M    44M select   0:00   0.00% evolution-calendar-
  912 vermaden      5  20    0    78M    42M select   0:00   0.00% evolution-source-re
  950 vermaden      6  20    0  2134M    39M select   0:00   0.00% gjs-console
  931 vermaden      6  20    0  2134M    39M select   0:00   0.00% gjs-console
  935 vermaden      5  20    0   119M    32M select   0:00   0.00% gsd-media-keys
  939 vermaden      5  20    0    51M    30M select   0:00   0.00% gsd-xsettings
  937 vermaden      5  20    0    49M    30M select   0:00   0.00% gsd-power
  957 vermaden      3  40   19   256G    28M select   0:00   0.00% baloo_file
  907 vermaden      7  20    0    83M    28M select   0:00   0.00% gnome-shell-calenda
  947 vermaden      5  20    0    46M    28M select   0:00   0.00% gsd-keyboard
  994 vermaden      4  20    0    46M    28M select   0:00   0.00% ibus-extension-gtk3
  993 vermaden      4  20    0    46M    27M select   0:00   0.00% ibus-ui-gtk3
  893 polkitd       7  20    0  2125M    27M select   0:00   0.00% polkitd
  965 vermaden      5  20    0    48M    27M select   0:00   0.00% zeitgeist-datahub
  952 vermaden      4  24    0    49M    26M select   0:00   0.00% gsd-printer
  859 vermaden      5  20    0    49M    20M select   0:00   0.00% gnome-session-binar
  874 root          7  20    0    67M    16M select   0:00   0.00% bsdisks
  956 vermaden      4  20    0    27M    14M select   0:00   0.00% ibus-daemon
  942 vermaden      5  20    0    27M    13M select   0:00   0.00% gsd-datetime
  945 vermaden      5  20    0    26M    13M select   0:00   0.00% gsd-sound
  943 vermaden      6  20    0    25M    12M select   0:00   0.00% gsd-smartcard
  940 vermaden      4  20    0    27M    11M select   0:00   0.00% gsd-print-notificat
 1092 vermaden      4  20    0    24M    11M select   0:00   0.00% gvfsd-dnssd
 1082 vermaden      5  20    0    24M    11M select   0:00   0.00% gvfsd-network
  872 vermaden      5  20    0    27M    11M select   0:00   0.00% gvfs-udisks2-volume
 1041 vermaden      4  20    0    24M    11M select   0:00   0.00% gvfsd-trash
 1062 vermaden      4  20    0    24M    11M select   0:00   0.00% gvfsd-burn
  976 vermaden      4  20    0    25M    10M select   0:00   0.00% zeitgeist-daemon
  902 vermaden      5  20    0    23M    10M select   0:00   0.00% gnome-keyring-daemo
  894 vermaden      1  20    0    21M  9488K RUN      0:00   0.00% sshd
  944 vermaden      5  20    0    21M  9356K select   0:00   0.00% gsd-housekeeping
  887 root          1  24    0    21M  9332K select   0:00   0.00% sshd
  933 vermaden      4  20    0    21M  9252K select   0:00   0.00% at-spi2-registryd
  925 vermaden      2  21    0    86M  9216K select   0:00   0.00% pulseaudio
  870 vermaden      4  20    0    24M  8860K select   0:00   0.00% gvfsd
  934 vermaden      5  20    0    20M  8616K select   0:00   0.00% gsd-usb-protection
  891 root         16  20    0    24M  8488K select   0:00   0.00% console-kit-daemon
  811 root          1  20    0    21M  8208K select   0:00   0.00% sshd
  867 vermaden      5  20    0    21M  8128K select   0:00   0.00% at-spi-bus-launcher
  927 root          4  20    0    20M  8040K select   0:00   0.00% accounts-daemon
  941 vermaden      5  20    0    20M  7976K select   0:00   0.00% gsd-sharing
  886 vermaden      5  20    0    21M  7936K select   0:00   0.00% gvfs-gphoto2-volume
  936 vermaden      5  20    0    20M  7936K select   0:00   0.00% gsd-a11y-settings
  923 vermaden      4  20    0    19M  7716K select   0:00   0.00% gvfsd-metadata
  910 root          4  20    0    19M  7620K select   0:00   0.00% upowerd
  881 vermaden      5  20    0    19M  7220K select   0:00   0.00% gvfs-mtp-volume-mon
  938 vermaden      4  20    0    19M  7180K select   0:00   0.00% gsd-screensaver-pro
  825 root          1  20    0    18M  7140K select   0:00   0.00% sendmail
  914 vermaden      4  20    0    19M  7128K select   0:00   0.00% dconf-service
  828 smmsp         1  52    0    18M  6704K pause    0:00   0.00% sendmail
  864 vermaden      1  20    0    14M  5268K select   0:01   0.00% dbus-daemon
  753 messagebus    1  20    0    14M  4280K select   0:00   0.00% dbus-daemon
  863 vermaden      1  21    0    15M  3912K select   0:00   0.00% dbus-launch
  868 vermaden      1  20    0    14M  3812K select   0:00   0.00% dbus-daemon
 1067 vermaden      1  20    0    14M  3704K nanslp   0:00   0.00% gstat
 1114 vermaden      1  20    0    14M  3420K RUN      0:00   0.00% top
  895 vermaden      1  20    0    13M  3252K wait     0:00   0.00% sh
  851 vermaden      1  22    0    13M  3212K wait     0:00   0.00% sh
 1066 vermaden      1  26    0    13M  3208K wait     0:00   0.00% sh
  843 root          1  23    0    13M  3100K wait     0:00   0.00% login
  854 vermaden      1  20    0    14M  3096K wait     0:00   0.00% xinit
  858 vermaden      1  21    0    13M  3016K wait     0:00   0.00% sh
  467 _dhcp         1  52    0    13M  2828K select   0:00   0.00% dhclient
  672 root          1  20    0    13M  2748K select   0:00   0.00% syslogd
  406 root          1   4    0    13M  2708K select   0:00   0.00% dhclient
  403 root          1  44    0    13M  2632K select   0:00   0.00% dhclient
  814 root          1  26    0    13M  2516K nanslp   0:00   0.00% cron
  757 root          1  20    0    13M  2404K select   0:00   0.00% moused
  850 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  847 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  848 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  844 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  849 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  845 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  846 root          1  52    0    13M  2248K ttyin    0:00   0.00% getty
  468 root          1  20    0    11M  1540K select   0:00   0.00% devd

Summary of the RAM usage results are below.

  MB  ENVIRONMENT
----  --------------------
  97  FreeBSD Text Console
 614  Openbox
1361  MATE
1548  XFCE
2622  GNOME
2843  KDE/Plasma

Seems that MATE

CPU Time Usage Results

As I already had the top(1) outputs I also compared the CPU time used for that limited test. I will not post the top(1) results again as they are posted above. The Seconds column below is the sum of the TIME field from the top(1) command output.
Here are the results of used CPU time.

SECONDS  ENVIRONMENT
-------  --------------------
      0  FreeBSD Text Console
      7  Openbox
     26  MATE
     42  XFCE
     72  GNOME
     92  KDE/Plasma

Seems that MATE is twice as light on resources then XFCE. World has changed a lot since GNOME 2 was considered heavy fully fledged desktop environment while XFCE was light and fast … but even that ‘hungry’ XFCE takes only half of the time that KDE/Plasma uses for the same tasks.

Subjective Experience

The Openbox ‘environment’ started fastest and was most responsive to operate.

Both MATE and XFCE started little slower but after they loaded the desktop and taskbars they were snappy and fast to use.

On the other hand the KDE/Plasma took longest to load and each application I started – I needed to wait ‘a little’ with ‘bouncing mouse cursor’ for them to load. Also scrolling the /etc/ssh/moduli file to its end in Kate took REALLY long … even with Logitech M720 mouse which scroll wheel was spinning freely (without clicks). I want to mention that I am not disappointed by KDE/Plasma. Its just slower when used in a limiter 1 CPU and 8 GB RAM environment. Probably the load times and usability is a lot better on a 4 CORE system with 16 GB of RAM on fast NVMe SSD disk while we used rather slow virtual machine.

After adding GNOME to comparison it felt even slower then KDE/Plasma. Probably because GNOME requires hardware 3D acceleration for smooth operation. With its software rendering it felt really sluggish … while consuming less RAM and CPU time then KDE/Plasma.

Summary

Not sure how useful that is but I just was curious and wanted to check it out – and as I did I share what I found.

UPDATE 1 – Added `freecolor(1)` and `htop(1)` RAM Measurements

After suggestions from other places I added freecolor(1) and htop(1) measurements when it comes to RAM usage. Here are the results.

            | htop(1) | conky(1) | freecolor(1)
    FreeBSD |  112 MB |    - -   |  157 MB 
    Openbox |  237 MB |  460 MB  |  382 MB 
       MATE |  508 MB |  778 MB  |  788 MB 
       XFCE |  533 MB |  794 MB  |  829 MB 
helloSystem |  585 MB |    - -   |  830 MB 
      GNOME |  625 MB |  990 MB  | 1000 MB 
 KDE/Plasma |  730 MB | 1659 MB  | 1167 MB

Hope that helps.

EOF

Ghost in the Shell – Part 7 – ZSH Setup

3 Replies

Today I would like to share with you my simple yet useful zsh(1) shell config that I use daily.

You may want to check other articles in the Ghost in the Shell series on the Ghost in the Shell – Global Page where you will find links to all episodes of the series along with table of contents for each episode’s contents.

I have been various UNIX and Linux systems since almost two decades. Through that time I was always looking for the best interactive shell out there. Obviously I have started with the bash(1) on Linux and it generally worked but bash(1) also did not impressed me at all. Just a shell. I have similar experiences with the ksh(1) shell. Today even the plain POSIX /bin/sh shell on FreeBSD has basic completion similar in many ways to what bash(1) or ksh(1) allows. The bash(1) shell gets little better when you install the bash-completion companion but its very limited still.

When I moved to FreeBSD I got to know its default tcsh(1)/csh(1) shell … which is PITA to use and scripting. Its pointless to learn limited CSH shell syntax in 2021. Omit it at any costs. After I settled a little in the FreeBSD wonderland I started to try other shells such as zsh(1) or fish(1) shells. I really liked fish(1) shell preconfigured setup and its defaults because it required literally zero effort to use it at its peak possibilities … but when I tried one of my typical use cases which is some commands | while read I; other command "${I}"; done it came to me that fish(1) is very limited shell and even does not support critical POSIX /bin/sh syntax! What a disappointment to say the last.

I abandoned the fish(1) shell and went to the zsh(1) which by default does not do more then a bash(1) shell and needs well thought configuration to be useful and powerful. After checking some guides and howtos about zsh(1) shell I started to create my own config and this was the interactive environment I was looking for. Of course I had several newbie problems or things that did not worked well for me like for example automatic completion of user home directories or UPPERCASE to lowercase automatic translation but after digging more into the zsh(1) config and man pages I finally settled with sensible and reasonable zsh(1) shell config.

I also tried various ready to use zsh(1) preconfigurations such as PowreLevel10k or Oh-My-Zsh but none of them really satisfied my while being kinda ‘blackbox’ with features that I do not really need. I really like to use things that I understand under the hood so I stayed with my quite simple yet fast loading config.

Why ZSH Shell Anyway?

Besides The Usual Suspects (really great movie by the way) like recursive search with [CTRL]+[R] for forward search and [CTRL]+[SHIFT]+[R] for reverse search – argument completion for most commands like shown below.

% tar -[TAB]
A  -- append to an archive
c  -- create a new archive
f  -- specify archive file or device
t  -- list archive contents
u  -- update archive
v  -- verbose output
x  -- extract files from an archive

The graphical example of that can be shown here. Of course I am not able to show [TAB] key there as I am able to add in the ‘text’ examples.

zsh-gstat

Interactive argument completion like showing the list of processes you can kill(1) by pressing the [TAB] key while being at kill(1) or killall(1) commands.

% kill -9 [TAB]
 9289  4  Ss+  0:00.62 -zsh (zsh)
16994  2  Is   0:00.28 -zsh (zsh)
17860  1  Is+  0:00.17 -zsh (zsh)
23797  3  Is+  0:00.23 -zsh (zsh)
30335  4  S+   0:00.01 -zsh (zsh)
32381  4  R+   0:00.00 ps
44994  0  Is+  0:00.50 -zsh (zsh)
59828  2  I+   0:00.02 /bin/sh /usr/bin/man zsh
65435  2  I+   0:00.05 less

Similar with the pkill(1) command when trying to autocomplete with ‘h‘ letter. The filter adds all running processes that have ‘h‘ letter in them – not only processes that start with the ‘h‘ letter.

zsh-pkill

There are also other more sophisticated completions like completioning the file name but not from the front but by the part of it … or by extension. Take a look at these two examples below. This is out directory listing that we will be using as an example here.

% exa -l
drwxr-xr-x - vermaden 2021-09-18 21:47 and a really PITA dir with spaces
.rw-r--r-- 0 vermaden 2021-09-18 21:54 huge.iso
.rw-r--r-- 0 vermaden 2021-09-18 21:46 really.async.example.txt
.rw-r--r-- 0 vermaden 2021-09-18 21:47 some-plain-file.txt

Three files and one directory with spaces in its name.

To autocomplete any of them with bash(1)you would have to start typing the file or dir name from the beginning. The fish(1) shell is on par with zsh(1) here as it would also support the thing that I want to show you.

First things first – the cd(1) command to change current working directory. Because there is only ONE directory there both zsh(1) and fish(1) shells would properly autocomplete the only once and a really PITA dir with spaces dir for the cd(1) command like shown below.

% exa -l
drwxr-xr-x - vermaden 2021-09-18 21:47 and a really PITA dir with spaces
.rw-r--r-- 0 vermaden 2021-09-18 21:54 huge.iso
.rw-r--r-- 0 vermaden 2021-09-18 21:46 really.async.example.txt
.rw-r--r-- 0 vermaden 2021-09-18 21:47 some-plain-file.txt

% cd [TAB]

// after pressing [TAB] once becomes this

% cd and\ a\ really\ PITA\ dir\ with\ spaces

The bash(1) (and csh(1)/tcsh(1) for the record) would obviously need to start from the first letter of any of those dir or files trying the really stupid completion method.

Now the second part about completion of files extensions or names in the middle of dirs or files. Both zsh(1) and fish(1) shells support that. Examples below.

% exa -l
drwxr-xr-x - vermaden 2021-09-18 21:47 and a really PITA dir with spaces
.rw-r--r-- 0 vermaden 2021-09-18 21:54 huge.iso
.rw-r--r-- 0 vermaden 2021-09-18 21:46 really.async.example.txt
.rw-r--r-- 0 vermaden 2021-09-18 21:47 some-plain-file.txt

% cat txt[TAB]

// NOW zsh(1) will show all files that have 'txt' string in it

% cat le.txt[TAB]

// AFTER SECOND [TAB] HIT IT WILL LIST THEM WITH MENU FEATURE (MORE ON THAT IN A MOMENT)

% cat really.async.example.txt[TAB]
really.async.example.txt  some-plain-file.txt

// NOW FIRST FILE WITH 'txt' IS USED - HIT [TAB] AGAIN TO SWITCH TO NEXT ONE

% cat some-plain-file.txt[TAB]
really.async.example.txt  some-plain-file.txt

// YOU CAN ALSO USE ARROW KEYS TO SELECT BETWEEN THEM - CHECK SCREENSHOT BELOW

Example of menu completion feature below.

Time to stop showing off and start to provide some useful content.

System Config

There are lots of guides and ideologies about how you spread your zsh(1) configuration between system wide config file and user customized ones like these:

/etc/zshenv
/etc/zprofil
/etc/zshrc
/etc/zlogin
/etc/zlogout
~/.zshenv
~/.zprofile
~/.zshrc
~/.zlogin
~/.zlogout

My take? Lets not make some big issue about that. I really like simple sensible setups and I use zsh(1) as interactive shell so ‘system wide’ configuration is not crucial here. To make things as simple as possible I only use two of all of the above. The /usr/local/etc/zshrc for the ‘system wide’ part and ~/.zshrc for my ‘user’ part. Thats it. I have been doing that since more then a decade and it worked for me like a charm but as in every case your millage may vary here.

As there are too many Linuxisms out there assuming that you are using Ubuntu Linux or that bash(1) shell is always available as /bin/sh binary after 16 years of me using FreeBSD UNIX there for sure will several BSDisms but at least they are harmless and documented 🙂

The /usr/local/etc/zshrc (or should I say /etc/zshrc on Linux and other then FreeBSD UNIX systems) is as follows.

# BASICS
  umask 022
  export PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
  export EDITOR=vi
  export PAGER=less

# USE ftp(1) PASSIVE MODE
  export FTP_PASSIVE_MODE=1

# DISABLE less(1) HISTORY
  export LESSHISTSIZE=0

# IMITATE sockstat(1) ON LINUX
  case $( uname ) in
    (Linux) alias sockstat="netstat -lnptu --protocol=inet,unix" ;;
  esac

# ZSH HISTORY
  export HISTSIZE=655360
  export HISTFILE="${HOME}/.zhistory"
  export SAVEHIST=${HISTSIZE}

# ZSH HISTORY SEARCH
  bindkey -M vicmd '/' history-incremental-pattern-search-backward
  bindkey -M vicmd '?' history-incremental-pattern-search-forward

# ZSH HISTORY SEARCH FOR vi(1) INSERT MODE
  bindkey -M viins '^R' history-incremental-pattern-search-backward
  bindkey -M viins '^F' history-incremental-pattern-search-forward

# ZSH HISTORY MAPPINGS
  bindkey '^[[A' up-line-or-search
  bindkey '^[[B' down-line-or-search
  bindkey "^R" history-incremental-search-backward

# ZSH USE SHIFT-TAB FOR REVERSE COMPLETION
  bindkey '^[[Z' reverse-menu-complete

# ZSH LAST ARG FROM EARLIER COMMAND WITH [ALT]-[.]
  bindkey '\e.' insert-last-word

# ZSH BEGIN/END OF LINE
  bindkey "^A" beginning-of-line
  bindkey "^E" end-of-line

# KEY BINDINGS
  case "${TERM}" in
    (cons25*|linux)
      # PLAIN BSD/LINUX CONSOLE
      bindkey '\e[H'    beginning-of-line   # HOME
      bindkey '\e[F'    end-of-line         # END
      bindkey '\e[5~'   delete-char         # DELETE
      bindkey '[D'      emacs-backward-word # ESC+LEFT
      bindkey '[C'      emacs-forward-word  # ESC+RIGHT
      ;;
    (*rxvt*)
      # RXVT DERIVATIVES
      bindkey '\e[3~'   delete-char         # DELETE
      bindkey '\eOc'    forward-word        # CTRL+RIGHT
      bindkey '\eOd'    backward-word       # CTRL+LEFT
      # RXVT WORKAROUND FOR screen(1) UNDER urxvt(1)
      bindkey '\e[7~'   beginning-of-line   # HOME
      bindkey '\e[8~'   end-of-line         # END
      bindkey '^[[1~'   beginning-of-line   # HOME
      bindkey '^[[4~'   end-of-line         # END
      ;;
    (*xterm*)
      # XTERM DERIVATIVES
      bindkey '\e[H'    beginning-of-line   # HOME
      bindkey '\e[F'    end-of-line         # END
      bindkey '\e[3~'   delete-char         # DELETE
      bindkey '\e[1;5C' forward-word        # CTRL+RIGHT
      bindkey '\e[1;5D' backward-word       # CTRL+LEFT
      # XTERM WORKAROUND FOR screen(1) UNDER xterm(1)
      bindkey '\e[1~'   beginning-of-line   # HOME
      bindkey '\e[4~'   end-of-line         # END
      ;;
    (screen)
      # GNU SCREEN
      bindkey '^[[1~'   beginning-of-line   # HOME
      bindkey '^[[4~'   end-of-line         # END
      bindkey '\e[3~'   delete-char         # DELETE
      bindkey '\eOc'    forward-word        # CTRL+RIGHT
      bindkey '\eOd'    backward-word       # CTRL+LEFT
      bindkey '^[[1;5C' forward-word        # CTRL+RIGHT
      bindkey '^[[1;5D' backward-word       # CTRL+LEFT
      ;;
  esac

# ZSH COMPLETION CASE (IN)SENSITIVE
# zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}'

# ZSH DISABLE USER COMPLETION FOR THESE NAMES
  zstyle ':completion:*:*:*:users' ignored-patterns \
    dladm dbus distcache dovecot list ftp games gdm gkrellmd gopher gnats \
    adm amanda apache avahi backup beaglidx bin cacti canna clamav daemon \
    sshd sync sys syslog uucp vcsa smmsp svctag upnp unknown webservd xfs \
    listen mdns fax mailman mailnull mldonkey mysql man messagebus netadm \
    hacluster haldaemon halt hsqldb mail junkbust ldap lp irc xvm libuuid \
    nscd ntp nut nx ident openldap operator pcap pkg5srv postfix postgres \
    netcfg nagios noaccess nobody4 openvpn named netdump nfsnobody nobody \
    proxy privoxy pulse pvm quagga radvd rpc rpcuser shutdown statd squid \
    www-data news nuucp zfssnap rpm '_*'

# ZSH COMPLETION OPSTIONS
  zstyle ':completion:*' completer _expand _complete _correct _approximate _history
  zstyle ':completion:*' matcher-list '' '' 'l:|=* r:|=*' 'l:|=* r:|=*'
  zstyle ':completion:*' list-colors ''
  zstyle ':completion:*' users root
  zstyle ':completion:*' menu select
  zstyle :compinstall filename '~/.zshrc'
  autoload -Uz compinit
  autoload -U colors && colors
  compinit

# ZSH OTHER FEATURES
  unsetopt beep
  setopt no_beep
  setopt nohashdirs
  setopt extended_glob
  setopt auto_cd
  setopt auto_menu
  setopt list_rows_first
  setopt multios
  setopt hist_ignore_all_dups
  setopt append_history
  setopt inc_append_history
  setopt hist_reduce_blanks
  setopt always_to_end
  setopt no_hup
  setopt complete_in_word
  limit coredumpsize 0

# ZSH zshcontrib(1) zmv
  autoload zmv
  alias zmv_to_lower='zmv      "*" "\${(L)f}"'
  alias zmv_to_upper='zmv      "*" "\${(U)f}"'
  alias zmv_to_capital='zmv    "*" "\${(C)f}"'
  alias zmv_to_hypen='zmv      "*" "\$f:gs/ /-/"'
  alias zmv_to_underscore='zmv "*" "\$f:gs/ /_/"'

# COLOR grep(1)
  export GREP_COLOR='1;32'
  export GREP_COLORS='1;32'
  export GREP_OPTIONS='--color=auto'
  alias grep='grep --color'
  alias egrep='egrep --color'

# FreeBSD ifconfig(8) CIDR NOTATION
  export IFCONFIG_FORMAT=inet:cidr

# SET ls(1) COLORS
  export LSCOLORS='exExcxdxcxexhxhxhxbxhx'
  export LS_COLORS='no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32'

# DISABLE XON/XOFF FLOW CONTROL (^S/^Q)
  stty -ixon

# COLOR LIST
# 30 - black     # 34 - blue
# 31 - red       # 35 - magenta
# 32 - green     # 36 - cyan
# 33 - yellow    # 37 - white

# COLOR PROMPT
  cSRV="%F{magenta}"
  case $( whoami ) in
    (root)
      cUSR="%F{red}"
      cPMT="%F{red}"
      ;;
    (*)
      cUSR="%F{green}%B"
      cPMT=""
      ;;
  esac
  cTIM="%F{cyan}%B"
  cPWD="%F{magenta}%B"
  cSTD="%b%f"
  export PS1="$cTIM%T$cSTD $cSRV%m$cSTD $cUSR%n$cSTD $cPWD%~$cSTD $cPMT%#$cSTD "
  export PS2="$cTIM%T$cSTD $cUSR>$cSTD $cPWD"

# SET PROPER ENCODINGS
  case ${TERM} in
    (cons25*) export LC_ALL=en_US.ISO8859-1 ;;
    (*)       export LC_ALL=en_US.UTF-8     ;;
  esac

# ALIASES
  alias rehash='hash -r'
  alias make='env LANG=C LC_ALL=C make'
  alias h='history'
  alias c='clear'
  alias vim='vim -i NONE'
  alias fetch='fetch -Rr --no-verify-peer --no-verify-hostname'
  alias wget='wget -c -t 0'

# LS/GLS/EXA
  case $( uname ) in
    (FreeBSD)
      if /usr/bin/env which exa 1> /dev/null 2> /dev/null
      then
        alias bls='/bin/ls -p -G -D "%Y.%m.%d %H:%M"'
        alias gls='gls -p --color=always --time-style=long-iso --group-directories-first --quoting-style=literal'
        alias ls='exa --time-style=long-iso --group-directories-first'
      elif /usr/bin/env which gls 1> /dev/null 2> /dev/null
      then
        alias bls='/bin/ls -p -G -D "%Y.%m.%d %H:%M"'
        alias ls=' gls -p --color=always --time-style=long-iso --group-directories-first --quoting-style=literal'
      else
        alias ls=' /bin/ls -p -G -D "%Y.%m.%d %H:%M"'
      fi
      ;;
    (OpenBSD)
      export PKG_PATH=http://ftp.openbsd.org/pub/OpenBSD/$( uname -r )/packages/$( uname -m )/
      [ -e /usr/local/bin/colorls ] && alias ls='/usr/local/bin/colorls -G'
      ;;
    (Linux)
      if /usr/bin/env which exa 1> /dev/null 2> /dev/null
      then
        alias gls='ls -p --color=always --time-style=long-iso --group-directories-first --quoting-style=literal'
        alias ls='exa --time-style=long-iso --group-directories-first'
      else
        alias ls='ls -p --color=always --time-style=long-iso --group-directories-first --quoting-style=literal'
      fi
      ;;
  esac
  alias la='ls -A'
  alias ll='ls -l'
  alias exa='exa --time-style=long-iso --group-directories-first'

If for any reason WordPress would mess the above config up here is the plain text version – https://raw.githubusercontent.com/vermaden/scripts/master/zshrc – available from my GitHub scripts repository.

While its comments generally say a lot about that is happening there I will also add several notes here.

I have left disabled the UPPERCASE from/to lowercase transparent translation because while it helped at about 10% of times it really pissed me off with pointless autocomplete suggestions the 90% of the time. If your work/complete schema is different the enable and test it. Maybe it will suit you better then me. Below is the part I am talking about – in enabled form.

# ZSH COMPLETION CASE (IN)SENSITIVE
  zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}'

At the end of the config you will find ‘casting’ for the best ls(1) solution existing in a system. After trying various listing commands such as:

FreeBSD ls(1) command
Linux ls(1) (known as gls(1) under FreeBSD)
New exa(1) command
New lsd(1) command

I have abandoned lsd(1) as besides colors its close to useless to use exa(1) as primary listing command. The second one that I recommend (that may be a surprise to FreeBSD users) would be the Linux ls(1) command from sysutils/coreutils package on FreeBSD. The last ‘resort’ command would be the FreeBSD ls(1) command as documented in the config. Why you should ask? The answer is quite simple – the directory listing. Both exa(1) and gls(1) have options to list directories (and what is more important SYMLINKS to directories) first. The FreeBSD ls(1) not only does not list symlinks to directories first – it also treat any directory as any other object and just list directories and symlinks put somewhere there withing all other files. Its unacceptable for me. Its just a messy pointless output. As much as I like and respect FreeBSD UNIX this is just plain fucking stupid. No matter how much history is in it.

Here is the comparison between them. I also wanted to show you the long listing (with -l option obviously) but its the same ‘not dirs first’ behavior for the FreeBSD ls(1) so not need for that.

zsh.ls

User Config

I will not add the ‘user’ part of my zsh(1) config and add some comments below.

# IMPORT DOAS/SUDO
  if [ -f ~/.zshrc.DOAS.SUDO ]
  then
    source ~/.zshrc.DOAS.SUDO
  else
    echo "NOPE: file ~/.zshrc.DOAS.SUDO absent."
  fi

# BASICS
  export PATH=${PATH}:~/scripts:~/scripts/bin:~/.cargo/bin
  export EDITOR=vi
  export VISUAL=vi
  export BROWSER=firefox
  export MANWIDTH=tty
  export ENV=${HOME}/.shrc
  export IFCONFIG_FORMAT=inet:cidr
  export LC_COLLATE=C

# BASICS DESKTOP
  export DISPLAY=:0
  export MOZ_DISABLE_IMAGE_OPTIMIZE=1
  export _JAVA_OPTIONS='-Dawt.useSystemAAFontSettings=on'
  export NO_AT_BRIDGE=1

# ENABLE ICONS IN exa(1)
  case ${TERM} in
    (rxvt)   : ;;
    (xterm*) : ;;
    (*)      alias exa='exa --icons' ;;
  esac

# ALIASES
  alias Grep=grep
  alias grpe=grep
  alias grepMAC='grep -i -E "[0-9a-f]{2}\:[0-9a-f]{2}\:[0-9a-f]{2}\:[0-9a-f]{2}\:[0-9a-f]{2}\:[0-9a-f]{2}"'
  alias grepIP='grep -E "([0-9]+\.){3}[0-9]+"'
  alias cls='printf "\033[H\033[J"'
  alias e=exa
  alias bat='bat --color=always'
  alias x='xinit ~/.xinitrc -- -dpi 75 -nolisten tcp 1> /dev/null 2> /dev/null'
  alias ffmpeg='ffmpeg -hide_banner'
  alias mupdf='mupdf -r 120'
  alias tac='tail -r'
  alias lsof='lsof -w'
  alias less='less -r --chop-long-lines'
  alias more='less -r --chop-long-lines'
  alias pstree='pstree -g 2'
  alias lupe='lupe -noshape -mag 2 -nohud -geometry 300x200 -noreticle -noiff'
  alias parallel='parallel --no-notice --progress -j 3'
  alias pv='pv -t -r -a -b -W -B 1048576'
  alias caja='caja --browser --no-desktop'
  alias evince=atril
  alias we="curl -4 http://wttr.in/Lodz\?Q\?n 2> /dev/null | sed '\$d' | sed '\$d'"
  alias cclive='cclive -c'
  alias yu='youtube-dl -c -i -f best --skip-unavailable-fragments'
  alias aria2c='aria2c --file-allocation=none'
  alias dig=drill
  alias cssh='cssh -o "-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"'
  alias ssh='ssh -o LogLevel=quiet -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
  alias feh="feh --scale-down \
                 --auto-rotate \
                 --auto-zoom \
                 --fontpath ~/.fonts \
                 --font       ubuntu/8 \
                 --menu-font  ubuntu/8 \
                 --title-font ubuntu/8"
  alias wget='wget -c --no-check-certificate \
                   -U "Opera/12.16 (X11; FreeBSD 13.0 amd64; U; en) Presto/3 Version/12"'
  alias scp='scp -o ControlMaster=yes \
                 -o ControlPath=/tmp/%r@%h:%p \
                 -o UserKnownHostsFile=/dev/null \
                 -o StrictHostKeyChecking=no'

# SHORT HISTORY ALIASES h() H()
  alias h='< ~/.zhistory grep -i'
  alias H='< ~/.zhistory grep'

# SHORT GREP FUNCTIONS
  alias g='grep -i'
  alias G='grep'

# SHORT QUERY FUNCTIONS q()
  q() {
    if [ ${#} -eq 1 ]
    then
      /bin/ls | grep --color -i ${1} 2> /dev/null
    else
      echo "usage: q string"
    fi
  }

# SHORT QUERY FUNCTIONS Q()
  Q() {
    if [ ${#} -eq 1 ]
    then
      /bin/ls | grep --color ${1} 2> /dev/null
    else
      echo "usage: Q string"
    fi
  }

# SHORT QUERY FUNCTIONS qq()
  qq() {
    if [ ${#} -eq 1 ]
    then
      find . \
        | grep -i ${1} 2> /dev/null \
        | cut -c 3-999 \
        | grep --color -i ${1} 2> /dev/null
    else
      echo "usage: qq string"
    fi
  }

# SHORT QUERY FUNCTIONS QQ()
  QQ() {
    if [ ${#} -eq 1 ]
    then
      find . \
        | grep ${1} 2> /dev/null \
        | cut -c 3-999 \
        | grep --color ${1} 2> /dev/null
    else
      echo "usage: QQ string"
    fi
  }

# FUNTIONS / INTELIGENT CD()
  dc() {
    if [ -f "${@}" ]
    then
      cd "${@%/*}"
      return 0
    fi

    if [ -d "${@}" ]
    then
      cd "${@}"
      return 0
    fi

    echo "${0}: no such file or directory: ${@}"
    return 1
  }

# FUNTIONS / PORTS / ports-check()
  ports-check() {
    CUT='Major OS version upgrade detected.'
  # ${CMD} nice -n 20 portsnap auto
    ${CMD} nice -n 20 gitup ports
    echo
    ${CMD} nice -n 20 portmaster -L --index-only \
      | grep -v "${CUT}" \
      | awk '/ [Nn]ew / { print substr($0,9,9999) }'
    echo
    VULNS=$( ${CMD} pkg audit -F 2>&1 | grep ' vulnerable' | sort -u | sed 's/\ is\ vulnerable://g' )
    echo Vulnerabilities:
    if [ "${VULNS}" = "" ]
    then
      echo None.
    else
      echo "${VULNS}"
    fi
    echo
    pkg updating \
      -d $( date -j -f "%s" "$( pkg query -a %t | grep -v "${CUT}" | sort | tail -1 )" "+%Y%m%d" )
  }

# FUNTIONS / PORTS / ports-rebuild()
  ports-rebuild() {
    # OPTIONS
    local PORTS='multimedia/ffmpeg'
  # local PORTS='multimedia/ffmpeg audio/lame sysutils/exfat-utils sysutils/fusefs-exfat'

    for PORT in ${PORTS}
    do
      ${CMD} pkg unlock -y ${PORT} 1> /dev/null 2> /dev/null
      ${CMD} idprio 10 env BATCH=yes DISABLE_VULNERABILITIES=yes make -C /usr/ports/${PORT} build deinstall install clean &
      MAKE=${!}
      ${CMD} rctl -a process:${MAKE}:pcpu:deny=40
      ${CMD} wait ${MAKE}
    # ${CMD} pkg lock -y ${PORT}
    done
  }

# FUNTIONS / PORTS / ports-build()
  ports-build() {
    case ${#} in
      (0) ${CMD} nice -n 20 portmaster -y --no-confirm -m 'BATCH=yes' -d -a ;;
      (*) ${CMD} nice -n 20 portmaster -y --no-confirm -m 'BATCH=yes' -d $@ ;;
    esac
    ${CMD} nice -n 20 find /var/db/pkg -type d -depth 1 -exec rm -rf {} ';' 2> /dev/null
  }

# FUNTIONS / PKG / pkg-defunct()
  pkg-defunct() {
    pkg version -Rl\? | cut -wf1
  }

# FUNTIONS / PORTS / pkg-version()
  pkg-version() {
    pkg version -I -l '<' | awk '{print $1}'
  }

# FUNTIONS / PORTS / pkg-size()
  pkg-size() {
    pkg info -as | sort -k 2 -h | tail -100
  }

# FUNTIONS / BMI
  bmi() { # 1=HEIGHT 2=WEIGHT
    if [ ${#} -ne 2 ]
    then
      echo "usage: $( basename ${0} ) HEIGHT WEIGHT"
      echo
      echo "table:"
      echo "  UNDER WEIGHT   LESS - 18.4"
      echo "  NORMAL WEIGHT  18.5 - 24.9"
      echo "  OVER WEIGHT    25.0 - 29.9"
      echo "  OBESITY        30.0 - MORE"
      echo
      return 1
    fi
    local BMI=$( echo "${2} / ( ${1} * ${1} ) * 10000" | bc -l )
    printf "%.1f\n" "${BMI}"
  }

# FUNTIONS / BFP
  bfp() {
    if [ ${#} -ne 4 ]
    then
      echo "usage: $( basename ${0} ) HEIGHT WEIGHT AGE SEX"
      echo
      echo "SEX: f - female"
      echo "     m - male"
      return 1
    fi
    case ${4} in
      (m) SEX=1 ;;
      (f) SEX=0 ;;
    esac
    local BMI=$( echo "${2} / ( ${1} * ${1} ) * 10000" | bc -l )
    local BFP=$( echo "( 1.2 * ${BMI} ) + ( 0.23 * ${3} ) - ( 10.8 * ${SEX} ) - 5.4" | bc -l )
    printf "%.1f%%\n" "${BFP}"
  }

# FUNTIONS / BMR
  bmr() {
    if [ ${#} -ne 3 ]
    then
      echo "usage: $( basename ${0} ) WIEGHT HEIGHT AGE"
      echo
      return 1
    fi
    local RESULT=$( echo "( 10 * ${1} ) + ( 6.25 * ${2} ) - ( 5 * ${3} ) + 5" | bc -l )
    if echo ${RESULT} | grep -q '^\.'
    then
      echo -n 0
    fi
    echo ${RESULT} | awk -F '.' '{print $1}'
  }

# FUNTIONS / MATH
  math() {
    local SCALE=2
    local INPUT=$( echo "${@}" | tr 'x' '*' | tr ',' '.' )
    local RESULT=$( echo "scale=${SCALE}; ${INPUT}" | bc -l )
    if echo ${RESULT} | grep -q '^\.'
    then
      echo -n 0
    fi
    echo ${RESULT}
  }

# FUNTIONS / MAH2WH
  conv_mah_2_wh() {
    if [ ${#} -ne 2 ]
    then
      echo "usage: $( basename ${0} ) mAh V"
      echo
      return 1
    fi
    local MAH2WH=$( echo "${1} * ${2} / 1000" | bc -l )
    printf "%.1f Wh\n" "${MAH2WH}"
  }

# FUNTIONS / WH2MAH
  conv_wh_2_mah() {
    if [ ${#} -ne 2 ]
    then
      echo "usage: $( basename ${0} ) Wh V"
      echo
      return 1
    fi
    local WH2MAH=$( echo "${1} / ${2} * 1000" | bc -l )
    printf "%.1f mAh\n" "${WH2MAH}"
  }

# FUNTIONS / CM2IN
  conv_cm_2_in() {
    if [ ${#} -ne 1 ]
    then
      echo "usage: $( basename ${0} ) INCH"
      echo
      return 1
    fi
    local CM=$( echo "${1} / 2.54" | bc -l )
    printf "%.1f cm EQUALS %.1f inch(es)\n" "${1}" "${CM}"
  }

# FUNTIONS / IN2CM
  conv_in_2_cm() {
    if [ ${#} -ne 1 ]
    then
      echo "usage: $( basename ${0} ) INCH"
      echo
      return 1
    fi
    local INCH=$( echo "${1} * 2.54" | bc -l )
    printf "%.1f inch(es) EQUALS %.1f cm\n" "${1}" "${INCH}"
  }

# FUNTIONS / REMOVE SSH known_hosts KEY
  ssh_known_hosts_key_remove() {
    if [[ -z "${1}" ]]
    then
      echo "usage: ${0} [host]"
      echo "  Removes specified host from ~/.ssh/known_hosts file."
    else
      sed -i '' -e "/${1}/d" ${HOME}/.ssh/known_hosts
    fi
  }

# FUNTIONS / CAL
  cal() {
    if which gcal 1> /dev/null 2> /dev/null
    then
      local TEST="${@}"
      if [ "${TEST}" = "-3" ]
      then
        gcal -s 1 .  | sed 1,2d | sed 3d
      else
        gcal -s 1 ${@}
      fi
    else
      cal ${@}
    fi
  }

# FUNTIONS / DAY
  day() {
    if [ ${#} -eq 0 ]
    then
      echo "usage: ${0##*/} DAY-OF-MONTH"
      return 1
    fi
    cal $( date +%Y ) \
      | env GREP_COLOR="07;32" grep --color=always -EC 6 " $1 |^$1 | $1\$" \
      | env GREP_COLOR="07;33" grep --color=always -B2 -A6 -E 'Mo|Tu|We|Th|Fr|Sa|Su' \
      | grep -v -- --;
  }

# FUNTIONS / SSH-COPY-ID
  if ! which ssh-copy-id 1> /dev/null 2> /dev/null
  then
    ssh-copy-id() {
      echo 'INFO: ssh-copy-id(1) is not available'
      echo 'HINT: cat ~/.ssh/id_rsa.pub | ssh USER@HOST "cat >> ~/.ssh/authorized_keys"'
    }
  fi

If for any reason WordPress would mess the above config up here is the plain text version – https://raw.githubusercontent.com/vermaden/scripts/master/DOT.zshrc – available from my GitHub scripts repository.

I will not try to describe more useful parts of it. There are tons of aliases there from which these are more interesting ones.

Quickly grep(1) for IP or MAC address with grepIP or grepMAC functions respectively.

While most folks out there recommend the [CTRL]+[L] shortcut I really rarely use it. I know it and I generally advocate for those old UNIX shortcuts but for some reason the right [CTRL] key on my keyboard can not exist. I just do not use it at all. Its like in new condition on any keyboard I use. I should swap [CTRL] keys every quarter to make them look similarly used 🙂

Because of that I often use ‘c‘ shortcut to clear the screen. As I was forced to use Windows in my earlier employer I also had The Microsoft equivalent for clearing the terminal – the cls command – thus you will also find an alias for that in my config – so called muscle memory is still strong 🙂

Other aliases just have some arguments that are useful to add in 95% of cases.

Now some comment on the functions. There are for sure the Short Query Functions that I described in my Ghost in the Shell series. There is also additional dc alias to take me into directory where a file is. For example I have full path file under my X11 PRIMARY BUFFER. For example its /home/vermaden/gfx/wallpapers/amiga-500-grey.png value. I can now type cd and paste that buffer and then remove the amiga-500-grey.png characters with [BACKSPACE] key or type dc and then paste /home/vermaden/gfx/wallpapers/amiga-500-grey.png value and hit [ENTER] key. That alias(1) will now take me to the /home/vermaden/gfx/wallpapers/ dir.

There are several FreeBSD related commands also. Both pkg(8) or FreeBSD Ports related.

There are several that are health related such as BMI/BFP/BMR calculations that I sometimes use.

I really like the (and often use) the math function as it has the best of both worlds – the expr(1) and bc(1) commands.

There are also several functions related to conversions like converting the battery capacities between the Wh and mAh values or inches to centimeters conversions.

Similarly to the FreeBSD ls(1) command I also prefer to use the Linux (or should I say GNU) version of cal(1) command (known as gcal(1) in FreeBSD).

I also sometimes use the day function to highlight the exact day in the context of full year. Sometimes (quite rarely but still) its useful to know each occurrence of the 19 day of each month in current year. Below you will find screenshot with example.

zsh-day

You probably noticed the ~/.zshrc.DOAS.SUDO file at the beginning. Its about the detection of both sudo(8) and doas(1) supervisor commands. I prefer the more secure and simpler doas(1) command so when both are detected in the system then the doas(1) will be chosen as the right one.

Here are the ~/.zshrc.DOAS.SUDO contents.

% cat ~/.zshrc.DOAS.SUDO
SUDO_WHICH=0
SUDO=0
DOAS_WHICH=0
DOAS=1
ROOT=0

# CHECK doas(8) WITH which(1)
if which doas 1> /dev/null 2> /dev/null
then
  DOAS_WHICH=1
else
  DOAS_WHICH=0
fi

# CHECK sudo(8) WITH which(1)
if which sudo 1> /dev/null 2> /dev/null
then
  SUDO_WHICH=1
else
  SUDO_WHICH=0
fi

# CHECK USER WITH whoami(1)
if [ "$( whoami )" = "root" ]
then
  ROOT=1
fi

# CHOOSE ONE FROM doas(8) AND sudo(8)
if [ ${DOAS_WHICH} -eq 1 -o ${SUDO_WHICH} -eq 1 ]
then
  if [   ${DOAS} -eq 0 -a ${SUDO} -eq 1 -a ${SUDO_WHICH} -eq 1 ]
  then
    CMD=sudo
  elif [ ${DOAS} -eq 1 -a ${SUDO} -eq 0 -a ${DOAS_WHICH} -eq 1 ]
  then
    CMD=doas
  elif [ ${DOAS} -eq 1 -a ${SUDO} -eq 1 -a ${DOAS_WHICH} -eq 1 ]
  then
    CMD=doas
  fi
elif [ ${ROOT} -eq 1 ]
then
  CMD=''
else
  echo "NOPE: This script needs 'doas' or 'sudo' to work properly."
  exit 1
fi

unset SUDO_WHICH
unset DOAS_WHICH
unset ROOT

If for any reason WordPress would mess the above config up here is the plain text version – https://raw.githubusercontent.com/vermaden/scripts/master/DOT.zshrc.DOAS.SUDO – available from my GitHub scripts repository.

Summary

As the zsh(1) shell is very configurable there are probably at least dozen guides that make it better then me and in more depth but I just wanted to share all these with you as many of you asked what I actually use as my daily shell ‘driver’ setup.

Maybe you will be able to show me some other interesting zsh(1) tips s that would make it even more productive setup 🙂

UPDATE 1 – fish(1) Coloring and Completion in zsh(1)

One of the things I likes about the fish(1) shell was its ‘predictive’ showing what command can be executed completing from command history … I now also have that in the zsh(1) shell with two additional packages from FreeBSD.

First you need to install shells/zsh-autosuggestions and shells/zsh-syntax-highlighting packages.

# pkg install -y \
    shells/zsh-autosuggestions \
    shells/zsh-syntax-highlighting

Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
zsh-autosuggestions: 0.7.0
zsh-syntax-highlighting: 0.7.1,1

Number of packages to be installed: 2

40 KiB to be downloaded.

[1/2] Fetching zsh-autosuggestions-0.7.0.pkg: 100% 9 KiB 8.7kB/s 00:01
[2/2] Fetching zsh-syntax-highlighting-0.7.1,1.pkg: 100% 31 KiB 31.9kB/s 00:01
Checking integrity... done (0 conflicting)
[1/2] Installing zsh-autosuggestions-0.7.0...
[1/2] Extracting zsh-autosuggestions-0.7.0: 100%
[2/2] Installing zsh-syntax-highlighting-0.7.1,1...
[2/2] Extracting zsh-syntax-highlighting-0.7.1,1: 100%

=====
Message from zsh-autosuggestions-0.7.0:

--
Add the line below to your .zshrc to enable auto suggestions.

source /usr/local/share/zsh-autosuggestions/zsh-autosuggestions.zsh

=====
Message from zsh-syntax-highlighting-0.7.1,1:

--
Add the line below to *the end of* your .zshrc to enable highlighting.

source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

The second part of the job is to add two additional scripts to your ~/.zshrc config file as shown below.

% tail -13 ~/.zshrc

# ADDITIONAL COMPLETIONS zsh-autosuggestions
if [ -e /usr/local/share/zsh-autosuggestions/zsh-autosuggestions.zsh ]
then
  source /usr/local/share/zsh-autosuggestions/zsh-autosuggestions.zsh
fi

# ADDITIONAL COMPLETIONS zsh-syntax-highlighting
if [ -e /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh ]
then
  source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
fi

Now your zsh(1) shell will color the commands and also underline the existing files/directories and also showing grey additional completion of last time usage from history.

zsh-fish-completions

Not sure if now zsh(1) makes ALL the bells and whistles that fish(1) shell does but its definitely very small difference now if you are a big fish(1) fan 🙂

EOF

Ghost in the Shell – Part 6 – Learn Shell Scripting

6 Replies

The Ghost in the Shell series were about efficient working in the shell environment but one of the feats of any sysadmin profession is the shell scripting. It is often needed to ‘glue’ various solutions and technologies to work as ‘Business’ requires or to fill the gap where any solution is not available – or at least not for free. It also serves a growing role in the automation of various tasks. Today I will try to show you the basics of writing POSIX /bin/sh compatible shell scripts.

Basics

In your own ‘yard’ you can use any shell language you want – there are many good interactive shells like zsh(1)/bash(1)/fish(1)/ksh(1) to name a few. Just keep in mind to stay away from csh(1)/tcsh(1) shells as they are mediocre at most in interactive mode and terrible for scripting. Its really pity that csh(1)/tcsh(1) shells are still used as the default FreeBSD shells today knowing that zsh(1) is available under MIT license and could be painlessly integrated into the FreeBSD Base System – but who I am to fix all the world’s problems … I just install zsh(1) from packages and live on.

By writing POSIX /bin/sh scripts you are making sure that they will run not only on bash(1) in Linux but also on all BSD systems and all other UNIX systems out there. Even the really old dinosaurs like HP-UX or AIX.

I always struggled to find good example for learning the shell scripting but recently I got one idea and we will follow it today.

For our ‘target’ I have chosen the kldstat(8) command from FreeBSD. Its output is far from perfect (from my perspective) with showing the Size column in hexadecimal values – while sysadmin expects values in (mega/giga/tera)bytes. Our task will be to parse that kldstat(8) output into something more human readable.

Lets check that kldstat(8) output then.

% kldstat | head
Id Refs Address                Size Name
 1  133 0xffffffff80200000  1f11f28 kernel
 2    1 0xffffffff82112000   67feb0 zfs.ko
 3    1 0xffffffff82792000    1abe8 geom_eli.ko
 4    3 0xffffffff82a3c000    56ec0 vboxdrv.ko
 5    2 0xffffffff82a93000     4240 vboxnetflt.ko
 6    3 0xffffffff82a98000     aac8 netgraph.ko
 7    1 0xffffffff82aa3000     31c8 ng_ether.ko
 8    1 0xffffffff82aa7000     55e0 vboxnetadp.ko
 9    1 0xffffffff82aad000   158458 i915kms.ko

Now what does 1f11f28 tell me about kernel for the Size column. Not much.

For a start I would like to print just the Size and Name columns in our script – we will call it kld.sh for the lack of better name and I will add version ‘tag’ to its name for each of our steps like kld.0.1.sh for first and ./kld.0.2.sh for the second one and so on.

There are many ways to parse that kldstat(8) output in our script but I will discuss two approaches here.

First is to get the /bin/sh output into variable and then parse it in a loop.

Second one to parse it in a loop in pipe after the command directly. I will use the second one here because the first one – with keeping then /bin/sh output in a variable – my be useful if we want to parse it more then once and as we will parse it only once then its pointless to ‘waste’ memory for that variable. Below you will find the first draft or kld.sh.

0.1

Our first 0.1 version has only the interpreter set at the beginning (the #!/bin/sh shebang) and the simple while read loop to get output of the kldstat(8) command and print it on the screen with shell builtin echo(1) command.

% cat ./kld.0.1.sh
#!/bin/sh

kldstat \
  | while read LINE
    do
      echo "${LINE}"
    done

Here is our script output – its generally identical as the kldstat(8) command.

% ./kld.0.1.sh | head
Id Refs Address                Size Name
1  133 0xffffffff80200000  1f11f28 kernel
2    1 0xffffffff82112000   67feb0 zfs.ko
3    1 0xffffffff82792000    1abe8 geom_eli.ko
4    3 0xffffffff82a3c000    56ec0 vboxdrv.ko
5    2 0xffffffff82a93000     4240 vboxnetflt.ko
6    3 0xffffffff82a98000     aac8 netgraph.ko
7    1 0xffffffff82aa3000     31c8 ng_ether.ko
8    1 0xffffffff82aa7000     55e0 vboxnetadp.ko
9    1 0xffffffff82aad000   158458 i915kms.ko

0.2

As we know that kldstat(8) has fixed number of columns we can read its more intelligently with variables names as its columns and print only Size and Name columns as we wanted it in the first place. We should also omit the first line of kldstat(8) output as we will be printing our own header for just Size and Name columns. We will achieve that with sed(1) command.

Here is out script after our improvements.

% cat kld.0.2.sh
#!/bin/sh

echo "SIZE NAME"
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      echo "${SIZE} ${NAME}"
    done

Here is its output at current early stage.

% ./kld.0.2.sh | head
SIZE NAME
1f11f28 kernel
67feb0 zfs.ko
1abe8 geom_eli.ko
56ec0 vboxdrv.ko
4240 vboxnetflt.ko
aac8 netgraph.ko
31c8 ng_ether.ko
55e0 vboxnetadp.ko
158458 i915kms.ko

As you can see the columns are not aligned so we can use column(1) command to make it look more like original command.

% ./kld.0.2.sh | column -t | head
SIZE     NAME
1f11f28  kernel
67feb0   zfs.ko
1abe8    geom_eli.ko
56ec0    vboxdrv.ko
4240     vboxnetflt.ko
aac8     netgraph.ko
31c8     ng_ether.ko
55e0     vboxnetadp.ko
158458   i915kms.ko

But typing that each time we execute our script can be PITA so we will now use printf(1) instead of echo(1) to print our output. We will also alight the first Size column to the right to make the command output even more human readable. We will sacrifice 8 places of width for the Size column (%8s) and the rest with aligned to left (%-s) for Name column.

0.3

Here is our improved script.

% cat kld.0.3.sh
#!/bin/sh

printf "%8s %-s\n" SIZE NAME
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      printf "%8s %-s\n" ${SIZE} ${NAME}
    done

Our output now looks like that one below.

% ./kld.0.3.sh | head
      SIZE NAME
   1f11f28 kernel
    67feb0 zfs.ko
     1abe8 geom_eli.ko
     56ec0 vboxdrv.ko
      4240 vboxnetflt.ko
      aac8 netgraph.ko
      31c8 ng_ether.ko
      55e0 vboxnetadp.ko
    158458 i915kms.ko

Better. Now we will improve two things. First we will start keeping our output format ("%8s %-s\n") in a separate variable and we will finally convert that hexadecimal values into decimal ones – to bytes – there are many ways to do that but I am leaning to use the printf(1) builtin because both of speed and it being available in the shell (builtin).

0.4

Here is the script.

% cat kld.0.4.sh
#!/bin/sh

FORMAT="%8s %-s\n"
printf "${FORMAT}" SIZE NAME
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      SIZE=$( printf "%d" 0x${SIZE} )
      printf "${FORMAT}" ${SIZE} ${NAME}
    done

And its output with bytes instead of hexadecimal values.

% ./kld.0.4.sh | head
      SIZE NAME
  32579368 kernel
   6815408 zfs.ko
    109544 geom_eli.ko
    356032 vboxdrv.ko
     16960 vboxnetflt.ko
     43720 netgraph.ko
     12744 ng_ether.ko
     21984 vboxnetadp.ko
   1410136 i915kms.ko

Now we have output in bytes and its nicely formatted. Its even easily sortable by the sort(1) command so its leaning nicely with UNIX principles.

% ./kld.0.4.sh | sort -n | head
      SIZE NAME
      8432 coretemp.ko
      8504 cd9660_iconv.ko
      8504 msdosfs_iconv.ko
      8504 udf_iconv.ko
      8576 smbus.ko
      8736 cpuctl.ko
      8800 pty.ko
      9000 lindebugfs.ko
      9024 uhid.ko

The next step would be to print that information in megabytes instead of just plain bytes. To convert bytes into kilobytes we need to divide our bytes value by 1024. To get the megabytes we need to do it twice. We will use the $(( ... )) syntax to use the shell builtin for simple math calculations instead of dropping that task to a subshell with $( ... ) syntax and external commands.

0.5

This is our ‘show in megabytes only’ script looks like.

% cat kld.0.5.sh
#!/bin/sh

FORMAT="%8s %-s\n"
printf "${FORMAT}" SIZE NAME
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      SIZE=$( printf "%d" 0x${SIZE} )
      SIZE=$(( ${SIZE} / 1024 / 1024 ))
      printf "${FORMAT}" ${SIZE} ${NAME}
    done

And here is its output.

% ./kld.0.5.sh | head
      SIZE NAME
        31 kernel
         6 zfs.ko
         0 geom_eli.ko
         0 vboxdrv.ko
         0 vboxnetflt.ko
         0 netgraph.ko
         0 ng_ether.ko
         0 vboxnetadp.ko
         1 i915kms.ko

That did not wend too well, didn’t it? Because many module use less then 1 megabytes of memory after being rounded to natural numbers its 0 megabytes value for many modules. We will try to use bc(1) calculator instead with up to tenths precision.

0.6

Here is out script after using bc(1) instead of using the $(( ... )) syntax with dividing.

% cat kld.0.6.sh
#!/bin/sh

FORMAT="%8s %-s\n"
printf "${FORMAT}" SIZE NAME
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      SIZE=$( printf "%d" 0x${SIZE} )
      SIZE=$( echo "scale=1; ${SIZE} / 1024 / 1024" | bc -l )
      printf "${FORMAT}" ${SIZE} ${NAME}
    done

And here is its output.

% ./kld.0.6.sh | head
      SIZE NAME
      31.0 kernel
       6.4 zfs.ko
        .1 geom_eli.ko
        .3 vboxdrv.ko
         0 vboxnetflt.ko
         0 netgraph.ko
         0 ng_ether.ko
         0 vboxnetadp.ko
       1.3 i915kms.ko

Far from ideal. The bc(1) output omits the leading zero if value is less then one. Seems that we can fix that with different printf(1) formatting. Lets try that. We will change from %8s (string) into %8.1f (float). That will also force us to use different formats for header and values so will stop using single FORMAT variable and we will use separate ones.

0.7

This is our current script state.

% cat kld.0.7.sh
#!/bin/sh

HEAD_FORMAT="%8s %-s\n"
LOOP_FORMAT="%8.1f %-s\n"
printf "${HEAD_FORMAT}" SIZE NAME
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      SIZE=$( printf "%d" 0x${SIZE} )
      SIZE=$( echo "scale=1; ${SIZE} / 1024 / 1024" | bc -l )
      printf "${LOOP_FORMAT}" ${SIZE} ${NAME}
    done

And its output.

% ./kld.0.7.sh | head
      SIZE NAME
      31.0 kernel
       6.4 zfs.ko
       0.1 geom_eli.ko
       0.3 vboxdrv.ko
       0.0 vboxnetflt.ko
       0.0 netgraph.ko
       0.0 ng_ether.ko
       0.0 vboxnetadp.ko
       1.3 i915kms.ko

Works as advertised. We can now think of something different. How about we will also add an argument to include the kernel and modules file sizes as well? Not very useful I think but for the the purpose of shell scripting learning process we will do it anyway. The first caveat here is that kernel modules are on two locations on FreeBSD. The Base System modules are kept at /boot/kernel location and the modules that were installed by pkg(8) packages (or from FreeBSD Ports) are located at /boot/modules place. To get their size we will use the stat(1) command. Similarly like with memory usage – we would like to have the output of kernel and its modules size in megabytes.

There are of course several ways to achieve that. Lets start with the longest most educational example below. I will just paste the fragment that gets that kernel or module size for the FILE column.

if [ -f /boot/modules/${NAME} ]
then
  FILE=$( stat -f %z /boot/modules/${NAME} )
fi

if [ -f /boot/kernel/${NAME} -a -z ${NAME} ]
then
  FILE=$( stat -f %z /boot/kernel/${NAME} )
fi

if [ "${FILE}" = "" ]
then
  FILE=-
fi

FILE=$( echo "scale=1; ${FILE} / 1024 / 1024" | bc -l )

One note about the [ "${FILE}" = "" ] syntax – in all old POSIX shells out there that I used /bin/sh always worked well with that syntax when FILE variable was empty or non existing. In a extreme example this one – [ "" = "" ] – works as desired. In case if you find yourself in a situation when this does not work in some POSIX /bin/sh implementation then use the most secure variant with additional same word added to both sides like that – [ "${FILE}test" = "test" ] – this way even the most badly written POSIX /bin/sh implementation will work 🙂

It first checks the /boot/modules location for the module because I know a period of FreeBSD history in which the i915kms.ko module existed in both of these places and if you had them both then there is 99% percent chance that you are using the one installed by packages – that is why we try the third party modules first – then the ones from the Base System place. We also make sure that if for some reason the file will not be found the stat(1) command would not yield about its missing with 2> /dev/null at the end of command.

If we fail to find it under the third party modules then we will try the Base System location – but only when we did not find anything in the third party place – hence the additional test with -z ${NAME}.

For the record the syntax for these tests is:

for single test its like that: [ TEST ]
to test for both parameters (AND operator) its like that: [ TEST1 -a TEST ]
for only one of tests to pass (OR operator) its like that: [ TEST1 -o TEST ]

If we fail to find the file size then we set that to ‘–‘ value.

At the end we divide by 1024 two times so we get megabytes from bytes.

This can be shortened to to take less place (and writing) into something like that.

[ -f /boot/modules/${NAME} ]              && FILE=$( stat -f %z /boot/modules/${NAME} 2> /dev/null )
[ -f /boot/kernel/${NAME} -a -z ${NAME} ] && FILE=$( stat -f %z /boot/kernel/${NAME}  2> /dev/null )
[ ${FILE} = "" ]                          && FILE=-
FILE=$( echo "scale=1; ${FILE} / 1024 / 1024" | bc -l )

The end result is the same but it requires less space and writing. I also added some spaces for ‘logical formatting’ to make it more readable.

There is also more extreme way to shorten this up while keeping the same logic – here it is.

FILE=$( stat -f %z /boot/kernel/${NAME}  2> /dev/null \
     || stat -f %z /boot/modules/${NAME} 2> /dev/null \
     || FILE=- )
FILE=$( echo "scale=1; ${FILE} / 1024 / 1024" | bc -l )

We use then || OR operator in the subshell to make that shorter and still keep it readable. This is the version that we will use in our script.

0.8

Lets see now how it looks after modifications.

% cat kld.0.8.sh
#!/bin/sh

HEAD_FORMAT="%8s %8s %-s\n"
LOOP_FORMAT="%8.1f %8.1f %-s\n"
printf "${HEAD_FORMAT}" SIZE FILE NAME
kldstat \
  | sed 1d \
  | while read ID REFS ADDRESS SIZE NAME
    do
      FILE=$( stat -f %z /boot/kernel/${NAME}  2> /dev/null \
           || stat -f %z /boot/modules/${NAME} 2> /dev/null \
           || FILE=- )
      FILE=$( echo "scale=1; ${FILE} / 1024 / 1024" | bc -l )
      SIZE=$( printf "%d" 0x${SIZE} )
      SIZE=$( echo "scale=1; ${SIZE} / 1024 / 1024" | bc -l )
      printf "${LOOP_FORMAT}" ${SIZE} ${FILE} ${NAME}
    done

And here is its output.

% ./kld.0.8.sh | head
    SIZE     FILE NAME
    31.0     27.7 kernel
     6.4      5.0 zfs.ko
     0.1      0.1 geom_eli.ko
     0.3      0.4 vboxdrv.ko
     0.0      0.0 vboxnetflt.ko
     0.0      0.1 netgraph.ko
     0.0      0.0 ng_ether.ko
     0.0      0.0 vboxnetadp.ko
     1.3      2.2 i915kms.ko

Its interesting to see that used memory and file size are different.

Another step would be printing also the summary of the used RAM for each column. This is where things get more interesting. The while loop is created in a pipe which means its in a subshell. This has some serious implications. Normally we would add two variables like SIZE_TOTAL and FILE_TOTAL to add each module size there and then after the loop ends just print the summary. Because the while loop is spawned as subshell these variables will vanish as soon as the loop will end its life and these variables would not exist (they existed only in that while subshell).

But fear not – there is very clever way with file descriptor to have these variables exist with their values after the while loop ends. Below you will find the shortened prototypes of our currently used ‘pipe’ way and the ‘descriptor’ way.

This is the way you already know.

kldstat \
  | sed 1d \
  | while read LINE
    do
      echo "${LINE}"
      TOTAL="Now You Don't."
    done

echo ${TOTAL}

When you will execute that you will NOT see the "Now You Don't." string.

Now this is the way to overcome that subshell limitation.

while read LINE
do
  echo "${LINE}"
  TOTAL="Now You See Me."
done << BSD
  $( kldstat | sed 1d )
BSD

echo ${TOTAL}

As you try it you will see the "Now You See Me." sign at the end.

This way we will provide summary for each column.

0.9

This is our code after our effort to add summary for the columns. You may noticed that we added the FILE_TOTAL and SIZE_TOTAL before the FILE and SIZE values are converted to megabytes. That ensures we are as accurate as possible. If we would just sum up the SIZE and FILE after they were converted to megabytes we would lost several bytes in the process.

% cat kld.0.9.sh
#!/bin/sh

HEAD_FORMAT="%8s %8s %-s\n"
LOOP_FORMAT="%8.1f %8.1f %-s\n"
printf "${HEAD_FORMAT}" SIZE FILE NAME
while read ID REFS ADDRESS SIZE NAME
do
  FILE=$( stat -f %z /boot/kernel/${NAME}  2> /dev/null \
       || stat -f %z /boot/modules/${NAME} 2> /dev/null \
       || FILE=- )
  FILE_TOTAL=$(( ${FILE_TOTAL} + ${FILE} ))
  FILE=$( echo "scale=1; ${FILE} / 1024 / 1024" | bc -l )
  SIZE=$( printf "%d" 0x${SIZE} )
  SIZE_TOTAL=$(( ${SIZE_TOTAL} + ${SIZE} ))
  SIZE=$( echo "scale=1; ${SIZE} / 1024 / 1024" | bc -l )
  printf "${LOOP_FORMAT}" ${SIZE} ${FILE} ${NAME}
done << BSD
  $( kldstat | sed 1d )
BSD
FILE_TOTAL=$( echo "scale=1; ${FILE_TOTAL} / 1024 / 1024" | bc -l )
SIZE_TOTAL=$( echo "scale=1; ${SIZE_TOTAL} / 1024 / 1024" | bc -l )
printf "${LOOP_FORMAT}" ${SIZE_TOTAL} ${FILE_TOTAL} TOTAL

This is how its execution looks like.

% ./kld.0.9.sh | (head -5; echo '(...)'; tail -5)
    SIZE     FILE NAME
    31.0     27.7 kernel
     6.4      5.0 zfs.ko
     0.1      0.1 geom_eli.ko
     0.3      0.4 vboxdrv.ko
(...)
     0.0      0.0 linsysfs.ko
     0.0      0.0 fdescfs.ko
     0.0      0.0 nullfs.ko
     0.0      0.0 acpi_ibm.ko
    40.9     39.5 TOTAL

As you can see I also used shell feature to pipe output into many commands at once – this allows us to show information that is most important to use – beginning and ending – for the summary.

We even can do nested piping as shown on the screenshot below.

lolcat

I deliberately used head(1) for entire guide because I have total of 42 kernel modules loaded. I did not wanted these outputs to overshadow our objective here. Here at the end I will show you complete output for the sake of it.

% kldstat | wc -l
      42

% ./kld.0.9.sh
    SIZE     FILE NAME
    31.0     27.7 kernel
     6.4      5.0 zfs.ko
     0.1      0.1 geom_eli.ko
     0.3      0.4 vboxdrv.ko
     0.0      0.0 vboxnetflt.ko
     0.0      0.1 netgraph.ko
     0.0      0.0 ng_ether.ko
     0.0      0.0 vboxnetadp.ko
     1.3      2.2 i915kms.ko
     0.4      0.8 drm.ko
     0.0      0.0 linuxkpi_gplv2.ko
     0.0      0.0 lindebugfs.ko
     0.0      0.1 fusefs.ko
     0.0      0.0 coretemp.ko
     0.0      0.0 sem.ko
     0.0      0.0 cpuctl.ko
     0.0      0.0 ichsmb.ko
     0.0      0.0 smbus.ko
     0.0      0.0 cuse.ko
     0.0      0.0 libiconv.ko
     0.0      0.0 cd9660_iconv.ko
     0.0      0.0 msdosfs_iconv.ko
     0.0      0.0 udf_iconv.ko
     0.0      0.0 udf.ko
     0.0      0.0 acpi_wmi.ko
     0.0      0.0 uhid.ko
     0.0      0.0 usbhid.ko
     0.0      0.0 hidbus.ko
     0.0      0.0 wmt.ko
     0.0      0.0 ums.ko
     0.1      0.2 ng_btsocket.ko
     0.0      0.0 ng_bluetooth.ko
     0.2      0.6 linux.ko
     0.0      0.1 linux_common.ko
     0.1      0.5 linux64.ko
     0.0      0.0 pty.ko
     0.0      0.0 linprocfs.ko
     0.0      0.0 linsysfs.ko
     0.0      0.0 fdescfs.ko
     0.0      0.0 nullfs.ko
     0.0      0.0 acpi_ibm.ko
    40.9     39.5 TOTAL

% kldstat
Id Refs Address                Size Name
 1  133 0xffffffff80200000  1f11f28 kernel
 2    1 0xffffffff82112000   67feb0 zfs.ko
 3    1 0xffffffff82792000    1abe8 geom_eli.ko
 4    3 0xffffffff82a3c000    56ec0 vboxdrv.ko
 5    2 0xffffffff82a93000     4240 vboxnetflt.ko
 6    3 0xffffffff82a98000     aac8 netgraph.ko
 7    1 0xffffffff82aa3000     31c8 ng_ether.ko
 8    1 0xffffffff82aa7000     55e0 vboxnetadp.ko
 9    1 0xffffffff82aad000   158458 i915kms.ko
10    1 0xffffffff82c06000    7f548 drm.ko
11    2 0xffffffff82c86000     cbc8 linuxkpi_gplv2.ko
12    2 0xffffffff82c93000     2328 lindebugfs.ko
13    1 0xffffffff82c96000    11f10 fusefs.ko
14    1 0xffffffff82ca8000     20f0 coretemp.ko
15    1 0xffffffff82cab000     39e8 sem.ko
16    1 0xffffffff82caf000     2220 cpuctl.ko
17    1 0xffffffff82cb2000     3250 ichsmb.ko
18    1 0xffffffff82cb6000     2180 smbus.ko
19    1 0xffffffff82cb9000     6730 cuse.ko
20    4 0xffffffff82cc0000     4798 libiconv.ko
21    1 0xffffffff82cc5000     2138 cd9660_iconv.ko
22    1 0xffffffff82cc8000     2138 msdosfs_iconv.ko
23    1 0xffffffff82ccb000     2138 udf_iconv.ko
24    1 0xffffffff82cce000     5a00 udf.ko
25    1 0xffffffff82cd4000     3378 acpi_wmi.ko
26    1 0xffffffff82cd8000     2340 uhid.ko
27    1 0xffffffff82cdb000     3380 usbhid.ko
28    1 0xffffffff82cdf000     31f8 hidbus.ko
29    1 0xffffffff82ce3000     3320 wmt.ko
30    1 0xffffffff82ce7000     4350 ums.ko
31    1 0xffffffff82cec000    1ce48 ng_btsocket.ko
32    1 0xffffffff82d09000     25a8 ng_bluetooth.ko
33    1 0xffffffff82d0c000    388f8 linux.ko
34    4 0xffffffff82d45000     db70 linux_common.ko
35    1 0xffffffff82d53000    30ac8 linux64.ko
36    1 0xffffffff82d84000     2260 pty.ko
37    1 0xffffffff82d87000     639c linprocfs.ko
38    1 0xffffffff82d8e000     3284 linsysfs.ko
39    1 0xffffffff82d92000     3530 fdescfs.ko
40    1 0xffffffff82d96000     4700 nullfs.ko
41    1 0xffffffff82d9b000     41d8 acpi_ibm.ko

Summary

This concludes this Ghost in the Shell episode.

Feel free to share your scripting habits and spells 🙂

EOF

FreeBSD Desktop – Part 25 – Configuration – Random Terminal Theme

12 Replies

Some time ago when I was mostly writing about Openbox setup I also showed how to setup xterm(1) so it will start with new random theme with each start. Since then I reworked that feature a little and also added random theme and background selection for urxvt(1) terminal. This post will guide you through the needed steps to make that setup working.

terminal

You may want to check other articles in the FreeBSD Desktop series on the FreeBSD Desktop – Global Page where you will find links to all episodes of the series along with table of contents for each episode’s contents.

I already once wrote about random xterm(1) themes in the FreeBSD Desktop – Part 12 – Configuration – Openbox part – but I really wanted to expand that topic and also include other terminals.

After trying many terminal emulators – some more bulky like Konsole/GNOME Terminal/XFCE Terminal/MATE Terminal – some more lightweight like st(1)/rox-term(1)/eterm(1)/… – I always went back to the most old-school and basic one – good old xterm(1) terminal. Mostly because of its good compatibility with all UNIX systems – especially the older ones like IBM AIX or HP-UX. I do not remember last time when I had to manage these archaic systems but the respect for xterm(1) remains.

xterm

The other one that always got my attention was urxvt(1) terminal. Unfortunately it has some issues with fonts rendering – making larger spaces between the lines and making fonts bigger for example – but as I also like bitmap fonts like CLEAN or FIXED – so I use bitmap fonts for urxvt(1).

Recently also sakura(1) got my attention – but it’s theming possibilities are even more limited then xterm(1) with themes/colors hardcoded directly into the source code. That is why I will omit it in this article – but I mention it since its also nice terminal.

This is the Table of Contents for this article.

xterm(1)
- Proper Font Selection
- Selecting Text for Copy/Paste
- Selection Buffers and Keyboard Shortcuts
- Increase/Decrease Font Size on the Fly
- Copy/Paste with [CTRL]+[SHIFT]+[C/V]
- Interactive Menus
- Random xterm(1) Theme
urxvt(1)
- Tabbed Interface
- Daemon and Client Mode
- Random urxvt(1) Theme and Background
sakura(1)
- Different sakura(1) Themes
RAM Usage Comparison
CPU Time Usage Comparison

`xterm(1)`

First lest start with some reasonable configuration in the ~/.Xdefaults file. Alternatively some people use ~/.Xresources file. You can use any of them. Just pick one and stick to it.

! XTERM
! -----------------------------------------------------------------------------
  xterm*allowBoldFonts:     true
  xterm*allowWindowOps:     true
  xterm*boldMode:           false
  xterm*charClass:          33:48,35:48,37:48,43:48,45-47:48,64:48,95:48,126:48,35:48,58:48,63:48,61:48,44:48,38:48,59:48
  xterm*cursorBlink:        false
  xterm*cutNewline:         true
  xterm*faceName:           consolas
  xterm*faceSize:           11
  xterm*fastScroll:         true
  xterm*fullscreen:         false
  xterm*iconHint:           /home/vermaden/.icons/vermaden/xterm.xpm
  xterm*internalBorder:     1
  xterm*jumpScroll:         true
  xterm*keepSelection:      true
  xterm*loginShell:         true
  xterm*metaSendsEscape:    true
  xterm*multiScroll:        true
  xterm*omitTranslation:    fullscreen
  xterm*on4Clicks:          group
  xterm*on5Clicks:          page
  xterm*saveLines:          1024000
  xterm*scaleHeight:        1.0
  xterm*scrollKey:          true
  xterm*scrollTtyOutput:    false
  xterm*selectToClipboard:  true
  xterm*SimpleMenu*font:    -*-clean-*-*-*-*-*-*-*-*-*-*-iso8859-2
  xterm*termName:           xterm-256color
  xterm*title:              xterm
  xterm*veryBoldColors:     14
  xterm*VT100*geometry:     150x40
  xterm*VT100*translations: #override                                             \n\
                            <btn1up>: select-end(PRIMARY, CLIPBOARD, CUT_BUFFER0) \n\
                            Ctrl <key> minus: smaller-vt-font()                   \n\
                            Ctrl <key> plus: larger-vt-font()                     \n\
                            Ctrl Shift <key> C: copy-selection(CLIPBOARD)         \n\
                            Ctrl Shift <key> V: insert-selection(CLIPBOARD)

I will not discuss all possible settings as they are well described in the xterm(1) man page but I will comment some more useful and interesting ones.

Proper Font Selection

This advice is not limited to xterm(1) but its worth to mention it. Many times after adding fonts to my system – and renaming them to my ‘standard’ which looks like that one below I was asking myself how to properly specify the variant I need.

% ls -1 ~/.fonts/ubuntu-mono*
/home/vermaden/.fonts/ubuntu-mono-bold-italic.ttf
/home/vermaden/.fonts/ubuntu-mono-bold.ttf
/home/vermaden/.fonts/ubuntu-mono-italic.ttf
/home/vermaden/.fonts/ubuntu-mono.ttf

The answer to that question comes with fc-match(1) from fontconfig package. Check my ‘queries’ below.

% fc-match consolas:bold
consolas-bold.ttf: "Consolas" "Bold"

% fc-match consolas     
consolas.ttf: "Consolas" "Regular"

% fc-match consolas:bold:italic
consolas-bold-italic.ttf: "Consolas" "Bold Italic"

Selecting Text for Copy/Paste

The xterm*charClass defines which sets of characters should be treated the same when doing cut and paste. Especially with double-clicking the text. The setting above I use is based on 15 years of experience and seems to work best. You are of course encouraged to investigate the CHARACTER CLASSES section of the xterm(1) man page to read more on this topic.

The xterm*on4Clicks and xterm*on5Clicks are not used by default while xterm*on2Clicks are predefined as word and xterm*on3Clicks as line values. This is why I added them so you can select entire group with xterm*on4Clicks and entire page with xterm*on5Clicks option. Alternatively you can also use some fancy regex for some of these ‘CLICKS’ but I never thought about a REGEX that would be useful here – maybe you will come with something sensible.

Here are these ‘CLICKS’ in action.

First the xterm*on2Clicks with word selection. This is when the xterm*charClass is taken into account – what is word and that is not 🙂

xterm.2.clicks

Then xterm*on3Clicks with line selection.

xterm.3.clicks

Now xterm*on4Clicks with group selection.

xterm.4.clicks

Finally the xterm*on5Clicks entire page selection.

xterm.5.clicks

Selection Buffers and Keyboard Shortcuts

The last interesting option is xterm.VT100.translations which is used for keyboard shortcuts.

The first one select-end(PRIMARY, CLIPBOARD, CUT_BUFFER0) is better described in the https://davidsimmons.com/soft/xtermhacks/ page from 2005. I will try to short the meritum here. The X11 applications have two different selection buffers:

CLIPBOARD – selection buffer used for cut/paste functions – you select/highlight text and then select Copy from context menu or use [CTRL]+[C] shortcut. Then you use Paste or [CTRL]+[C] shortcut.

PRIMARY – this one receives data when user selects/highlights text with mouse. None other operations such as Copy or Paste are needed. You end selecting the text and its already in PRIMARY buffer. You then paste it with MIDDLE mouse button.

Using the option above selecting/highlighting the text in xterm(1) copies the text into both selection buffers simultaneously. You can now either Paste it info Firefox or hit MIDDLE mouse button to paste it in other xterm(1) terminal. Best of both worlds.

If that setting does not suit you then use the xterm*selectToClipboard instead. When set to true it copies selected text to CLIPBOARD buffer and when set to false it copies selection to the PRIMARY one.

Increase/Decrease Font Size on the Fly

I always missed the shortcuts to decrease or increase font size on the fly in xterm(1) and for many years I believed that its just not possible and then I found some blog post (do not remember which one now of course) in which I found these settings and started to use them.

They are smaller-vt-font() and larger-vt-font() for decrease and increase respectively with [CTRL]+[-] and [CTRL]+[+] shortcuts – keep in mind that [SHIFT] is not used here.

Copy/Paste with [CTRL]+[SHIFT]+[C/V]

If by some reason you prefer to copy and paste by using [CTRL]+[SHIFT]+[C] and [CTRL]+[SHIFT]+[V] shortcuts then copy-selection(CLIPBOARD) and insert-selection(CLIPBOARD) will do the needed job here. As you probably guessed you can use PRIMARY instead of CLIPBOARD here if that is what you desire.

Interactive Menus

The xterm(1) comes with three different interactive menus. I will now show all three of them here with screenshots.

Menu displayed with clicking [CTRL]+[LEFT-MOUSE-BUTTON] in the terminal area.

xterm.menu.mouse.LEFT

Menu displayed with clicking [CTRL]+[MIDDLE-MOUSE-BUTTON] in the terminal area.

xterm.menu.mouse.MIDDLE

Menu displayed with clicking [CTRL]+[RIGHT-MOUSE-BUTTON] in the terminal area.

xterm.menu.mouse.RIGHT

Random `xterm(1)` Theme

To have random xterm(1) theme on every startup you need four things:

~/.Xdefaults default config file
~/scripts/xterm.sh script.
~/.config/Xdefaults directory with xterm(1) themes.
~/.icons/vermaden/xterm.xpm for xterm(1) icon.

I gathered all these themes all over the Internet, only the VERMADEN and VERMADEN-OLD themes are created by me.

Little preview of some of the included xterm(1) themes.

xterm.random

From now on to have random xterm(1) theme at each start always start it with ~/scripts/xterm.sh script. The script itself is not very complicated. It just draws random theme from the ~/.config/Xdefaults/themes dir – then loads the ~/.Xdefaults config – then merges the colors from chosen random theme – and finally starts new xterm(1) instance.

xterm.sh

`urxvt(1)`

I use urxvt(1) less often but still sometimes I want to use bitmap fonts instead.

urxvt.single

For a start here is the urxvt(1) configuration in the ~/.Xdefaults file.

! URXVT
! -----------------------------------------------------------------------------
  urxvt.letterSpace:    0.0
! urxvt.font:           xft:monaco:pixelsize=9,style=regular,minspace=True
  urxvt.font:           -*-clean-*-*-*-*-*-*-*-*-*-*-iso8859-2
  urxvt.boldFont:       -*-clean-*-*-*-*-*-*-*-*-*-*-iso8859-2
  urxvt.iconFile:       /home/vermaden/.icons/vermaden/xterm.xpm
  urxvt.geometry:       150x40
  urxvt.cutchars:       ,;
  urxvt.scrollBar:      false
  urxvt.imLocale:       en_US.UTF-8
  urxvt.loginShell:     true
  urxvt.saveLines:      1024000
  urxvt.inheritPixmap:  false
  urxvt.shading:        20
  urxvt.xftAntialias:   true
  urxvt.jumpScroll:     true
  urxvt.tintColor:      black
  urxvt.internalBorder: 2
  urxvt.cursorBlink:    false
  urxvt.cursorColor:    #dd9900
  urxvt.cursorColor2:   #000000
  urxvt.colorBD:        #dddddd
  urxvt.colorIT:        #bbbbbb
  urxvt.colorUL:        #999999
  urxvt.underlineColor: #999999

Tabbed Interface

To get tabs in urxvt(1) add the following option to the ~/.Xdefaults configuration file.

  urxvt.perl-ext-common:   default,tabbed

With this option you will open new tab with [SHIFT]+[DOWN] shortcut.

To switch between the tabs left and right use [CTRL]+[SHIFT]+[LEFT] and [CTRL]+[SHIFT]+[RIGHT] shortcut respectively.

You can also use [CTRL]+[LEFT] and [CTRL]+[RIGHT] to move current tab left and right.

The timeless [CTRL]+[D] – which of course is not a strictly urxvt(1) shortcut but a general shortcut for closing all terminals.

urxvt.tabbed

Daemon and Client Mode

The urxvt(1) can be run in special daemon mode where you start one urxvtd(1) server and many urxvtc(1) clients.

% urxvtd
rxvt-unicode daemon listening on /home/vermaden/.urxvt/urxvtd-w520.local.

Now you will start each new urxvt(1) terminal with urxvtc(1) command.

The drawback of that approach is that when urxvtd(1) dies or crashes then also all your urxvtc(1) client terminals disappear 🙂

Random `urxvt(1)` Theme and Background

To have random urxvt(1) theme and background on every startup you need four things:

~/.Xdefaults default config file
~/scripts/urxvt.sh script.
~/.config/Xdefaults directory with urxvt(1) themes and backgrounds.
~/.icons/vermaden/xterm.xpm for urxvt(1) icon.

Little preview of some of the included urxvt(1) themes and backgrounds.

urxvt.random

From now on to have random urxvt(1) theme at each start always start it with ~/scripts/urxvt.sh script. The script for urxvt(1) is little more advanced. First it draws random theme from the ~/.config/Xdefaults/themes dir – then checks if its DARK or LIGHT theme – then draws either random LIGHT or DARK background from the ~/.config/Xdefaults/urxvt dir – finally loads the ~/.Xdefaults config and then merges the colors from chosen LIGHT or DARK theme. Of course then it finally starts new urxvt(1) instance.

urxvt.sh

`sakura(1)`

The more modern and GTK based sakura(1) also supports tabs. To open new tab use [CTRL]+[SHIFT]+[T] shortcut. To switch between the tabs use [CTRL]+[ALT]+[LEFT] and [CTRL]+[ALT]+[RIGHT] shortcuts. You can also move tab between left and right with [CTRL]+[SHIFT]+[LEFT] and [CTRL]+[SHIFT]+[RIGHT].

sakura.single

One things that sakura(1) impresses me is that you can scale down its window and then scale that window up and the contents that did not fit in the window after downscalling are back again in the terminal. Doing the same operation in xterm(1) or urxvt(1) terminals will result in these characters being lost. The output is also dynamically ‘fit’ into the new larger window while maintaining the new lines etc. Besides that nice feature it is small and fast and uses relatively small amount of RAM.

sakura.tabbed

Different `sakura(1)` Themes

If you would also like to start sakura(1) with different theme everytime the options are quite limited here. The palettes and color sets are hardcoded into the sakura(1) source code.

I will not show you how to modify them using the FreeBSD Ports system.

The sakura(1) port is located at /usr/ports/x11/sakura directory. For the record – I use WRKDIRPREFIX option in the /etc/make.conf file. This means that when I type make extract in the /usr/ports/x11/sakura dir the work directory will not be created at /usr/ports/x11/sakura/work directory but at /usr/ports/obj/usr/ports/x11/sakura/work instead. That way I can clean my Ports tree fast by removing the /usr/ports/obj directory.

We will now extract and patch the sakura(1) port on FreeBSD.

% grep WRKDIRPREFIX /etc/make.conf
WRKDIRPREFIX=${PORTSDIR}/obj

# cd /usr/ports/x11/sakura

# make patch

# cd /usr/ports/obj/$( pwd )/work/*/src || cd work/*/src

# pwd
/usr/ports/obj/usr/ports/x11/sakura/work/sakura-3.7.1/src

# grep -m 1 DEFAULT_PALETTE sakura.c
#define DEFAULT_PALETTE "tango"

# grep -o -E '[a-z]+_palette\[PALETTE_SIZE\]' sakura.c
gruvbox_palette[PALETTE_SIZE]
tango_palette[PALETTE_SIZE]
linux_palette[PALETTE_SIZE]
dark_palette[PALETTE_SIZE]
xterm_palette[PALETTE_SIZE]
rxvt_palette[PALETTE_SIZE]

As you can see the default sakura(1) palette is Tango. Fortunately you can use palette=solarized_dark option in the ~/.config/sakura/sakura.conf config file to change it into Solarized Dark for example.

Besides hardcoded palettes sakura(1) also has several Color Sets.

If you would like to make random theme (from the hardcoded ones) for each start you would have to use this syntax with prepared dedicated config files for each palette.

% sakura --config-file ~/.config/sakura/sakura.solarized_dark.conf --colorset 1 
% sakura --config-file ~/.config/sakura/sakura.tango.conf          --colorset 3

I do not use sakura(1) that much so I was too lazy to write random startup theme script also for it 🙂

Changing sakura(1) palette or color set manually is shown below.

sakura.colors.menu

sakura.colors.window

sakura.palette

RAM Usage Comparison

Just started xterm(1) terminal takes about 16 MB or RAM as you have seen on the urxvt(1) screenshots. The urxvt(1) started without tabs uses more then 2 TIMES of xterm(1) terminal RAM usage. The urxvt(1) started in tabbed mode uses more then 3 TIMES of xterm(1) terminal RAM usage. The sakura(1) also uses more then 3 TIMES of xterm(1) terminal RAM usage.

Table below shows RAM usage comparison. I have added more feature packed mate-terminal(1) to the list and also added the st(1) minimalistic terminal from Suckless project for compassion.

RAM/MB  TERMINAL  
    64  mate-terminal
    53  sakura
    52  urxvt (tabbed)
    38  urxvt
    16  xterm
    12  st

CPU Time Usage Comparison

I also made simple benchmark of the CPU used. The ‘benchmark’ was to check how much time each terminal would take to print output of dmesg | lolcat -b -r command. Here xterm(1) does not shine that much.

   TIME  TERMINAL
0:00.93  xterm
0:00.82  mate-terminal
0:00.52  sakura
0:00.43  urxvt
0:00.23  st

The above ‘benchmark’ was was quite ‘hard’ because of all the colors generated by lolcat(1) command. Lets try something more practical now. We will measure CPU time used to display out of the find find /usr/local/share/doc command.

   TIME  TERMINAL
0:01.34  xterm
0:01.18  mate-terminal
0:00.85  sakura
0:00.32  urxvt
0:00.28  st

Seems that lolcat(1) was not that ‘hard’. The st(1) minimalistic terminal really seems to suck less here 🙂

EOF

Ghost in the Shell – Part 5

4 Replies

The Ghost in the Shell series were quite neglected while I was busy writing about other things. Its about time to continue the series. I hope you are not mad at me because of it. Here are another few things that I think some of you may find useful.

Less More Useful

From all less(1) command line options I find these very handy.

Often when you pass some command output to less(1) you loose color. To keep color in the less(1) output use --raw-control-chars (or -r for short equivalent) option.

The other useful less(1) option I find useful is --chop-long-lines (or -S for short equivalent) which prevents line wrapping. You can of course scroll horizontally to see what does not fit on the screen.

While less(1) is a command line program it also has a very nice --mouse option – with this option you can scroll its output with your mouse wheel. How cool is that? You can even specify how many lines you want to scroll with --wheel-lines=n options where ‘n is as you probably guessed the number of scrolled lines.

It may be also useful to make less(1) quit if you want to display file that its contents fits in the current console screen – use --quit-if-one-screen for that.

Often people when they want to just view some config files they use vi(1) (or their other favorite ${EDITOR} that they use) – even if they do not intend to edit the file. Its better to open such file in less(1) and if you find out that you would want to edit the file hit the ‘v char while being at less(1) – it will open that file in your ${EDITOR} for editing.

You can also display line number in less(1) with --LINE-NUMBERS option (or use -N for shorter equivalent).

Detox These Filenames

Often when copying files from various sources the filenames may become corrupt in the process – mostly because of differences in encodings. To fix that very fast one may use detox(1) command. On FreeBSD systems its available as sysutils/detox package. Because the FILE you will be renaming almost for sure contains some special characters or spaces then its best to add quotation marks as shown below.

% detox "FILE"

Of course detox(1) renames one file at a time so to rename all files in the current directory we will use simple loop.

% for FILE in *; do detox "${FILE}"; done

If you want to also include subdirectories the do the following.

% find . -type f -exec detox {} ';'

If you do not want to limit yourself to files only (fix directories names also) then – as Mandalorian would say – this is the way.

% find . -exec detox {} ';'

Man Up the Info Pages

In the learning process of mastering UNIX systems one has to get used to reading man(1) pages and often getting back to re-reading them when needed. Like with many other things the GNU folks wanted to do things in their own way – seems they did not liked the man(1) pages that much as they created info(1) pages as an alternative. I dunno about you but IMHO info(1) pages does not feel like the UNIX way … maybe it’s because GNU is a recursive acronym for GNU IS NOT UNIX 🙂

However there is an elegant way to convert any info(1) page into man(1) page by piping the info(1) page output into less(1) command – or other ${PAGER} that you use.

% info ls | less

Real UNIX Sorting

After you setup your UNIX environment the LC_ALL environment variable is mostly set to some UTF variant – like en_us.utf-8 for example. That has implications as names of files and directories are now sorted case insensitively. To get back to original case sensitive UNIX sorting you can use the LC_ALL variable set ‘C‘. You can use that on the fly or make it permanent by adding it to your shell configuration. For example with ls(1) command shown below.

% ls -1
FreeBSD.org
kernel.org
Linux.com
NetBSD.org
openbsd.org
X11.org
xorg.conf

% env LC_ALL=C ls -1
FreeBSD.org
Linux.com
NetBSD.org
X11.org
kernel.org
openbsd.org
xorg.conf

Faster Better Uptime

When you want check for how long system was running we usually use uptime(1) command.

% uptime
8:15PM  up 5 days,  4:42, 4 users, load averages: 0.71, 0.76, 0.82

But you can type just one letter instead of six and get even more info – the w(1) command. It also includes information about other active sessions to this system – which comes handy because you want to know if someone else can try to fix or configure the same things as you intend to.

% w
8:15PM  up 5 days,  4:42, 4 users, load averages: 0.77, 0.78, 0.83
USER       TTY      FROM    LOGIN@  IDLE WHAT
vermaden   pts/0    :0     Thu10PM  3:09 -zsh (zsh)
szasstam   pts/1    :0     Sun08PM 1day  -zsh (zsh)
edwin      pts/2    :0      7:04PM     - -zsh (zsh)
larloch    pts/3    :0      7:56PM     - w

Filter Huge Files

When you start grep(1) to filter really big file – like several gigabytes in size for example – the grep(1) command uses locale from LC_ALL and LANG variables – which as you probably guess right know from the context of this sentence – slows things down.

You can modify both LC_ALL and LANG on the fly to ‘C‘ value to make that grep(1) really fast – and when I mean fast I mean sometimes you will gain several orders of magnitude.

% env LC_ALL=C LANG=C grep string HUGEFILE

That is all for this episode. Hope you liked it.

EOF

FreeBSD Desktop – Part 18 – Configuration – Global Dashboard

14 Replies

Many times I have found myself watching the various ‘debug’ commands like top/ps/mount/df or various log files like /var/log/messages or /var/log/automount.log when I thought something went wrong … or just takes little too long. I needed to open several terminal xterm(1) sessions (which is quite fast as I open them with [WIN]+[SPACE] and then [ENTER] but still …) and check what went wrong.

These actions tired my so I created a thing called Global Dashboard with all information I would ever need for such debugging.

From all the commands that FreeBSD contains I have chosen these 12 ones:

mount -p
/var/log/automount.log
/var/log/messages
vmstat -i
usbconfig
ps axwww -o %cpu,rss,command
sockstat -l -4
top -m io -o total
gstat -p
df -g
pciconf -l
ifconfig

Make sure you have doas(1) installed and configured. The most basic way to do it is below. You will have to be in wheel group to make it work properly.

# pkg install doas
# echo 'permit nopass :wheel as root' > /usr/local/etc/doas.conf
# chmod 400 /usr/local/etc/doas.conf

Let me show you how it looks.

Here is the typical empty desktop with Global Dashboard disabled.

… and here is the Global Dashboard enabled.

For the sake of comfort I will use [Scroll Lock] key with xbindkeys to toggle between this ‘debug’ session on and off as I already use [Pause Break] key to Pause Any Application described in the Part 16 – Configuration – Pause Any Application episode of FreeBSD Desktop series.

Conky

We will have to use older (1.9) version of Conky as the current one (1.10/1.11) are broken for anything serious.

We will use portdowngrade tool for that job.

First, lets install needed packages.

# pkg install portdowngrade conky xbindkeys

Assuming that you have up to date FreeBSD Ports tree in the /usr/ports directory – we see that current Conky version in the Ports is 1.11.

% cd /usr/ports/sysutils/conky
% cat distinfo 
TIMESTAMP = 1550919299
SHA256 (brndnmtthws-conky-v1.11.3_GH0.tar.gz) = 0140e749537d4d05bf33fbac436e54756faa26021e16f2bca418e9eeea724eb4
SIZE (brndnmtthws-conky-v1.11.3_GH0.tar.gz) = 2390099

We will now downgrade the Conky port to usable 1.9 version with portdowngrade utility. I already tried various Conky Port versions and the one that you are looking for is r419144 revision.

# cd /usr/ports/sysutils
# mv conky conky-1.11
# portdowngrade sysutils/conky | grep -C 17 r419144
------------------------------------------------------------------------
r422880 | madpilot | 2016-09-28 18:55:38 +0200 (Wed, 28 Sep 2016) | 13 lines

- Update conky and conky-awesome to 1.10.4
- Take maintainership [1]
- Options adapted to new version
- Removed LUA option since it's a mandatoory requirement now
- Use project own install target
- Fix installation of lua helper libraries
- Project moved to github
- in conky-awesome, properly use OPTIONS_EXCLUDE

PR:           212629
Submitted by: me
Approved by:  ntarmos@ceid.upatras.gr (former maintainer) [1]

------------------------------------------------------------------------
r419144 | pawel | 2016-07-26 20:57:23 +0200 (Tue, 26 Jul 2016) | 2 lines

Fix typo

------------------------------------------------------------------------
r419142 | pawel | 2016-07-26 20:40:20 +0200 (Tue, 26 Jul 2016) | 8 lines

- Add explicit IMPLIES between dependencies and simplify option handling [1]
- Convert to USES=localbase
- Switch some options helpers from LIB_DEPENDS to USE=xorg and USE=gnome

PR:           210414 [1] (based on)
Submitted by: elferdo@gmail.com
Approved by:  maintainer timeout

------------------------------------------------------------------------
r418767 | mat | 2016-07-19 13:04:13 +0200 (Tue, 19 Jul 2016) | 11 lines

We will now fetch the Conky port from r419144 revision – working 1.9 version.

# portdowngrade sysutils/conky r419144
A    conky/files
A    conky/Makefile
A    conky/files/patch-configure
A    conky/files/patch-lua-cairo.pkg
A    conky/files/patch-src-conky.c
A    conky/files/patch-src-freebsd.c
A    conky/files/patch-src-freebsd.h
A    conky/files/patch-src-fs.c
A    conky/pkg-descr
A    conky/distinfo
Checked out revision 419144.
You should be done-- now cd into conky and you can run
# make deinstall install clean

Please note that portdowngrade no longer modifies the ports tree; the
checked out port is at
/usr/ports/sysutils/conky

Done. Let’s verify that its the version we need.

% pwd
/usr/ports/sysutils
% cat conky-1.11/distinfo 
TIMESTAMP = 1550919299
SHA256 (brndnmtthws-conky-v1.11.3_GH0.tar.gz) = 0140e749537d4d05bf33fbac436e54756faa26021e16f2bca418e9eeea724eb4
SIZE (brndnmtthws-conky-v1.11.3_GH0.tar.gz) = 2390099

% cat conky/distinfo 
SHA256 (conky-1.9.0.tar.bz2) = baf1b550f135fbfb53e5e286a33aadc03a667d63bf6c4d52ba7637366295bb6f
SIZE (conky-1.9.0.tar.bz2) = 626555

Yup. We will now build a Conky 1.9 package (may be handy later).

# pwd
/usr/ports/sysutils
# cd conky
# pwd
/usr/ports/sysutils/conky
# make package
===>   conky-1.9.0_6 depends on file: /usr/local/sbin/pkg - found
=> conky-1.9.0.tar.bz2 doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://downloads.sourceforge.net/project/conky/conky/1.9.0/conky-1.9.0.tar.bz2
conky-1.9.0.tar.bz2                           100% of  611 kB  216 kBps 00m03s
===> Fetching all distfiles required by conky-1.9.0_6 for building
===>  Extracting for conky-1.9.0_6
=> SHA256 Checksum OK for conky-1.9.0.tar.bz2.
===>  Patching for conky-1.9.0_6
===>  Applying FreeBSD patches for conky-1.9.0_6
===>   conky-1.9.0_6 depends on executable: gmake - found
===>   conky-1.9.0_6 depends on package: libiconv>=1.14_11 - found
===>   conky-1.9.0_6 depends on package: pkgconf>=1.3.0_1 - found
===>   conky-1.9.0_6 depends on file: /usr/local/libdata/pkgconfig/x11.pc - found
===>   conky-1.9.0_6 depends on file: /usr/local/libdata/pkgconfig/xext.pc - found
===>   conky-1.9.0_6 depends on file: /usr/local/libdata/pkgconfig/xdamage.pc - found
===>   conky-1.9.0_6 depends on file: /usr/local/libdata/pkgconfig/xfixes.pc - found
===>   conky-1.9.0_6 depends on file: /usr/local/libdata/pkgconfig/xft.pc - found
===>  Configuring for conky-1.9.0_6
===>   FreeBSD 10 autotools fix applied to /usr/ports/obj/usr/ports/sysutils/conky/work/conky-1.9.0/config.rpath
(...)
====> Compressing man pages (compress-man)
===>  Building package for conky-1.9.0_6
===>  Cleaning for conky-1.9.0_6

… but where is our package, its not in the /usr/ports/sysutils/conky directory. Its not in the /usr/ports/distfiles dir either.

As I use WRKDIRPREFIX=${PORTSDIR}/obj option in the /etc/make.conf file it should be somewhere in the /usr/ports/obj then.

% grep WRKDIRPREFIX /etc/make.conf 
WRKDIRPREFIX=${PORTSDIR}/obj

Let’s find(1) it.

% find /usr/ports/obj -name conky\*txz
/usr/ports/obj/usr/ports/sysutils/conky/work/pkg/conky-1.9.0_6.txz

There. I will move it to /root directory to keep it.

# mv /usr/ports/obj/usr/ports/sysutils/conky/work/pkg/conky-1.9.0_6.txz /root

We will not clean up after the port/package building.

# make -C /usr/ports/sysutils/conky clean distclean
===>  Cleaning for conky-1.9.0_6
#

We will now delete installed Conky 1.11 version and install our working 1.9 version.

# pkg delete conky
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
        conky-1.11.3

Number of packages to be removed: 1

Proceed with deinstalling packages? [y/N]: y
[1/1] Deinstalling conky-1.11.3...
[1/1] Deleting files for conky-1.11.3: 100%

# pkg add /root/conky-1.9.0_6.txz
Installing conky-1.9.0_6...
Extracting conky-1.9.0_6: 100%

Last check for the Conky version.

% conky --version
Conky 1.9.0 compiled Tue Mar 19 12:55:55 CET 2019 for FreeBSD 11.2-RELEASE-p9 (amd64)

Compiled in features:

System config file: /usr/local/etc/conky/conky.conf
Package library path: /usr/local/lib/conky

 X11:
  * Xdamage extension
  * XDBE (double buffer extension)
  * Xft
  * ARGB visual

 Music detection:

 General:
  * math
  * config-output

Great. We have needed Conky version.

By the way – did you thought how much work will it take to make the same on Debian or CentOS without the FreeBSD Ports infrastructure? 🙂

Xbindkeys

The only needed configuration in the ~/.xbindkeysrc is this one below – it may be different for your keyboard so make sure to ‘catch’ needed key event.

% cat ~/.xbindkeysrc
# SCROLL LOCK | Scroll Lock
"~/scripts/desktop-debug.sh"
  m:0x0 + c:78

If you need more information about how Xbindkeys work then read the FreeBSD Desktop – Part 9 – Key Components – Keyboard/Mouse Shortcuts episode.

Scripts and Configs

This is the ~/scripts/desktop-debug.sh script.

#! /bin/sh

pgrep -q conky

case ${?} in
  (0) killall -9 conky ;;
  (1) ~/scripts/__openbox_restart_conky.sh ;;
esac

… and the ~/scripts/__openbox_restart_conky.sh script.

#! /bin/sh

VERSION=1.9
PROFILE=T420s

killall -9 conky

nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.1 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.2 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.3 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.4 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.5 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.6 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.7 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.8 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.9 &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.a &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.b &
nice -n 20 conky -c ~/.conkyrc.${VERSION}.${PROFILE}.LOG.c &

I use have several laptops so I need to distinguish which config files are used on which laptop, that is why I use PROFILE field – which is set to ThinkPad T420s in that example.

Here are the commands defined in these ~/.conkyrc.1.9.T420s.LOG.* files.

% grep exec ~/.conkyrc.1.9.T420s.LOG.*
.conkyrc.1.9.T420s.LOG.1:${color #eeeeee}${exec mount -p | awk '{print $1, $2, $3}' | column -t}
.conkyrc.1.9.T420s.LOG.2:${color #eeeeee}${exec tail -n 16 /var/log/automount.log}
.conkyrc.1.9.T420s.LOG.3:${color #eeeeee}${exec grep -v -E 'pulseaudio|message repeated|null_update_chw|route failed:|send_packet: |gen6_gt_|feeder_|cdce0: (Su|Re)' /var/log/messages | tail -16}
.conkyrc.1.9.T420s.LOG.4:${color #eeeeee}${exec vmstat -i}
.conkyrc.1.9.T420s.LOG.5:${color #eeeeee}${exec doas usbconfig}
.conkyrc.1.9.T420s.LOG.6:${color #eeeeee}${exec ps axwww -o %cpu,rss,command | head -1; ps axwww -o %cpu,rss,command | grep -v conky | grep -v '%CPU' | sort -n -r | head -15 }
.conkyrc.1.9.T420s.LOG.7:${color #eeeeee}${exec sockstat -l -4 | cut -c 1-50}
.conkyrc.1.9.T420s.LOG.8:${color #eeeeee}${exec top -m io -o total -b -s 1 -d 2 | grep -A 15 'PID USERNAME' | tail -n 16}
.conkyrc.1.9.T420s.LOG.9:${color #eeeeee}${exec gstat -p -I 345678}
.conkyrc.1.9.T420s.LOG.a:${color #eeeeee}${exec df -g | awk '{print $5,$6}' | column -t}
.conkyrc.1.9.T420s.LOG.b:${color #eeeeee}${exec pciconf -l}
.conkyrc.1.9.T420s.LOG.c:${color #eeeeee}${exec for I in $( ifconfig -l -u | sed s/lo0//g ); do ifconfig ${I}; done}

… and here is the diagram showing where these commands are placed.

I will use twelve (12) Conky configuration files for this purpose, each with one of the commands from above list.


 a df(1)       | b pciconf(8)             | c ifconfig(8)
---------------+--------------------------+---------------------
 7 sockstat(1) | 8 top(1)                 | 9 gstat(8)
---------------+--------------------------+---------------------
 4 vmstat(8)   | 5 usbconfig(8)           | 6 ps(1)
---------------+--------------------------+---------------------
 1 mount(8)    | 2 /var/log/automount.log | 3 /var/log/messages

Next are the full Conky configuration files.

~/.conkyrc.1.9.T420s.LOG.1

alignment                bottom_left
background               yes
gap_x                    3
gap_y                    3
minimum_size             279 193
maximum_width            280
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.1
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /sbin/mount -p
${color #eeeeee}${exec mount -p | awk '{print $1, $2, $3}' | column -t}

~/.conkyrc.1.9.T420s.LOG.2

alignment                bottom_left
background               yes
gap_x                    288
gap_y                    3
minimum_size             513 193
maximum_width            514
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.2
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /var/log/automount.log
${color #eeeeee}${exec tail -n 16 /var/log/automount.log}

~/.conkyrc.1.9.T420s.LOG.3

alignment                bottom_left
background               yes
gap_x                    807
gap_y                    3
minimum_size             789 193
maximum_width            790
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.3
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /var/log/messages
${color #eeeeee}${exec grep -v -E 'pulseaudio|message repeated|null_update_chw|route failed:|send_packet: |gen6_gt_|feeder_|cdce0: (Su|Re)' /var/log/messages | tail -16}

~/.conkyrc.1.9.T420s.LOG.4

alignment                bottom_left
background               yes
gap_x                    3
gap_y                    201
minimum_size             279 193
maximum_width            280
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.4
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /usr/bin/vmstat -i
${color #eeeeee}${exec vmstat -i}

~/.conkyrc.1.9.T420s.LOG.5

alignment                bottom_left
background               yes
gap_x                    288
gap_y                    201
minimum_size             513 193
maximum_width            514
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.5
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /usr/sbin/usbconfig
${color #eeeeee}${exec doas usbconfig}

~/.conkyrc.1.9.T420s.LOG.6

alignment                bottom_left
background               yes
gap_x                    807
gap_y                    201
minimum_size             789 193
maximum_width            790
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.6
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /bin/ps axwww -o %cpu,rss,command
${color #eeeeee}${exec ps axwww -o %cpu,rss,command | head -1; ps axwww -o %cpu,rss,command | grep -v conky | grep -v '%CPU' | sort -n -r | head -15 }

~/.conkyrc.1.9.T420s.LOG.7

alignment                bottom_left
background               yes
gap_x                    3
gap_y                    399
minimum_size             279 193
maximum_width            280
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.7
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /usr/bin/sockstat -l -4
${color #eeeeee}${exec sockstat -l -4 | cut -c 1-50}

~/.conkyrc.1.9.T420s.LOG.8

alignment                bottom_left
background               yes
gap_x                    288
gap_y                    399
minimum_size             513 193
maximum_width            514
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.8
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /usr/bin/top -m io -o total
${color #eeeeee}${exec top -m io -o total -b -s 1 -d 2 | grep -A 15 'PID USERNAME' | tail -n 16}

~/.conkyrc.1.9.T420s.LOG.9

alignment                bottom_left
background               yes
gap_x                    807
gap_y                    399
minimum_size             789 193
maximum_width            790
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.9
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /usr/sbin/gstat -p -I 300000
${color #eeeeee}${exec gstat -p -I 345678}

~/.conkyrc.1.9.T420s.LOG.a

alignment                bottom_left
background               yes
gap_x                    3
gap_y                    597
minimum_size             279 272
maximum_width            280
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.7
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /bin/df -g
${color #eeeeee}${exec df -g | awk '{print $5,$6}' | column -t}

~/.conkyrc.1.9.T420s.LOG.b

alignment                bottom_left
background               yes
gap_x                    288
gap_y                    597
minimum_size             513 272
maximum_width            514
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.8
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /usr/sbin/pciconf -l
${color #eeeeee}${exec pciconf -l}

~/.conkyrc.1.9.T420s.LOG.c

alignment                bottom_left
background               yes
gap_x                    807
gap_y                    597
minimum_size             789 272
maximum_width            790
double_buffer            yes
draw_outline             no
draw_shades              no
default_outline_color    444444
default_shade_color      444444
own_window               yes
own_window_class         conky
own_window_colour        222222
own_window_type          override
own_window_transparent   no
update_interval          2.9
use_xft                  yes
xftfont                  ubuntu mono-10
border_inner_margin      0
border_outer_margin      0
border_width             2

TEXT
${color #ee0000}% /sbin/ifconfig wlan0/em0/tun0
${color #eeeeee}${exec for I in $( ifconfig -l -u | sed s/lo0//g ); do ifconfig ${I}; done}

Thats a quite a lot configuration files but I think that this configuration done once will serve many many times in the future 🙂

These Conky configuration files are suited for the 1600×900 resolution, you will have to modify values of the gap_x/gap_y/minimum_size/maximum_width parameters to make it fit into other resolution.

Initially I wanted to write a script/generator for that, but lets face it – I will not be able to properly cover each possible resolution 🙂

UPDATE 2 – Latest Conky 1.11 Also Works

When I wrote this article I wrote that older Conky 1.9 version is needed (The conky-1.9.0_6 exactly which could be retrieved using portdowngrade sysutils/conky r419144 command).

Conky 1.10 introduced many bugs along with entirely new configuration format.

Latest Conky 1.11 (its conky-1.11.4_1 package on my box to be exact) works like a charm with Conky 1.9 configuration. It still has bug of NOT passing the mouse clicks to the desktop so of you want to make a left/middle/right click on the desktop aim on the place other then the Conky Dashboard space.

You can of course still follow the original article and fetch/build Conky with 1.9 version and have working left/middle/right mouse clicks on the desktop.

EOF

Ghost in the Shell – Part 3

5 Replies

Time to bring some life into the Ghost in the Shell series with Part 3 article.

Query Functions

I haven’t found better name for that solution. There are generally two types of UNIX people. These that prefer to navigate and operate with basic ls/cd/mv/mkdir/rm commands and those who use some file manager like Midnight Commander (mc) or ranger or vifm or … you get the idea. I have tried various CLI file managers but always came back to navigate without them. If you are one of those people then these Query Functions are for you 🙂

The so called Query Functions are for filter the information you look for. For example if you have directory with large number of files, then you would probably do something like that.

% ls | grep QUERY

… or if you also want to include subdirectories then something like that.

% find . | grep QUERY

For both of these examples you would also probably want to sometimes search case sensitive or insensitive depending on the need.

That leads us to four Query Functions:

q is an equivalent of ls | grep -i QUERY command.
Q is an equivalent of ls | grep QUERY command.
qq is an equivalent of find . | grep -i QUERY command.
QQ is an equivalent of find . | grep QUERY command.

Thus if I need to query the contents of directory while searching for something is very fast with q SOMETHING.

These are definitions of these Query Functions:

# SHORT QUERY FUNCTIONS q()
  q() {
    if [ ${#} -eq 1 ]
    then
      ls | grep --color -i ${1} 2> /dev/null
    else
      echo "usage: q string"
    fi
  }
     
# SHORT QUERY FUNCTIONS Q()
  Q() {
    if [ ${#} -eq 1 ]
    then
      ls | grep --color ${1} 2> /dev/null
    else
      echo "usage: Q string"
    fi
  }

# SHORT QUERY FUNCTIONS qq()
  qq() {
    if [ ${#} -eq 1 ]
    then
      find . \
        | grep -i ${1} 2> /dev/null \
        | cut -c 3-999 \
        | grep --color -i ${1} 2> /dev/null
    else
      echo "usage: qq string"
    fi
  }

# SHORT QUERY FUNCTIONS QQ()
  QQ() {
    if [ ${#} -eq 1 ]
    then
      find . \
        | grep ${1} 2> /dev/null \
        | cut -c 3-999 \
        | grep ${1} 2> /dev/null
    else
      echo "usage: QQ string"
    fi
  }

The qq and QQ functions uses grep(1) two times to make sure the output is colored.

I assume that You use colored grep(1) described in Ghost in the Shell – Part 2 article.

If you prefer to use alias(1) instead then they would look like that.

# SHORT QUERY FUNCTIONS q() Q() qq() QQ()
  alias q="ls | grep --color -i"
  alias Q="ls | grep --color"
  alias qq="find . | grep -i"
  alias QQ="find . | grep"

The qq and QQ will be little more limited as with functions its possible to trim the output to the exact needs with cut(1).

Lots of people use recursive history search which also helps, but what if you used/typed needed command long ago with the arguments you need now? You would probably search the command with history(1) command and then using grep(1) to limit the results to what you look for. I keep enormous large list of commands to keep in history – with my current setting of 655360 the ~/.zhistory (ZSH) file takes about 2.7 MB size. I also wanted to be sure that two identical commands would not be kept in history hence the setopt hist_ignore_all_dups ZSH option enabled. When I wc -l my ~/.zhistory file it currently has 75695 lines of commands.

% grep HISTSIZE /usr/local/etc/zshrc
export HISTSIZE=655360
export SAVEHIST=${HISTSIZE}

% grep dups /usr/local/etc/zshrc
setopt hist_ignore_all_dups

Now back to Query Functions for history:

h is an equivalent of cat ~/.zhistory | grep -i QUERY command.
H is an equivalent of cat ~/.zhistory | grep QUERY command.

They fit in aliases this time. In alias(1) we will use just grep(1) to not ‘do’ Useless Use of Cat.

Here are the Query Functions for history.

# SHORT HISTORY ALIASES h() H()
  alias h='< ~/.zhistory grep -i'
  alias H='< ~/.zhistory grep'

… but what if we would like to filter the outputs of q family and h family Query Functions? The obvious response is using grep(1) like q QUERY | grep ANOTHER or h QUERY | grep ANOTHER for example. To make that faster we will make g and G shortcuts.

g is an equivalent of grep -i command.
G is an equivalent of just grep command.

Here they are.

# SHORT GREP FUNCTIONS g() G()
  alias g='grep -i'
  alias G='grep'

Now it will be just q QUERY | g ANOTHER and h QUERY | G ANOTHER for example.

To clear terminal output you may use clear(1) command, some prefer [CTRL]-[L] shortcut but I find ‘c‘ alias to be the fastest solution.

# SHORT GREP FUNCTIONS c()
  alias c='clear'

To make the solution complete I would also add exa(1) here with an alias of ‘e‘.

# SHORT LISTING WITH e()
  alias e='exa --time-style=long-iso --group-directories-first'

Why exa(1) will you ask while there is BSD ls(1) and GNU ls(1) (installed as gls(1) on FreeBSD to not confuse). To add GNU ls(1) to FreeBSD system use the coreutils package.

Well, the BSD ls(1) has two major cons:

It is not able to sort directories first.
It selects width for ALL columns based on single longest file name.

The BSD ls(1) was used as following alias:

alias ls='ls -p -G -D "%Y.%m.%d %H:%M"'

The GNU ls(1) does not have these two problems but it does color the output only on the very limited pattern like:

Not executable file.
Executable file.
Directory.
Link.
Device.

The GNU ls(1) was used as following alias:

gls -p -G --color --time-style=long-iso --group-directories-first --quoting-style=literal

Here is where exa(1) comes handy as it does not have any cons like FreeBSD’s ls(1) and it colors a lot more types of files.

exa --time-style=long-iso --group-directories-first

Its still very simple coloring based on file extension and not magic number as plain (empty) text file SOME-NOT-FILE.pdf is colored like PDF document.

But even this ‘limited’ coloring helps in 99% of the cases and while with BSD ls(1) and GNU ls(1) all of these files ‘seem’ like plain text files with exa(1) its obvious from the start which are plain files, which are images and which are ‘documents’ like PDF files for example.

Where Is My Space

On all UNIX and Linux systems there exists du(1) command. Combined with sort(1) it is universal way of searching for space eaters. Example for the / root directory with -g flag to display units in gigabytes.

# cd /
# du -sg * | sort -n
1       bin
1       boot
1       compat
1       COPYRIGHT
1       data
1       dev
1       entropy
1       etc
1       lib
1       libexec
1       media
1       mnt
1       net
1       proc
1       rescue
1       root
1       sbin
1       sys
1       tmp
1       var
2       jail
8       usr
305     home

Contents of UNIX System Resources directory with -m flag to display unit in megabytes.

# cd /usr
# du -sm * | sort -n
1       libdata
1       obj
1       tests
3       libexec
11      sbin
13      include
45      lib32
56      lib
58      share
105     bin
1080    ports
1343    src
5274    local

But its PITA to type cd and du all the time, not to mention that some oldschool UNIX systems does not provide -g or -m flags so on HP-UX you are limited to kilobytes at most.

You may also try -h (human readable) with sort -h (sort human readable) du(1) variant.

# du -smh * | sort -h
512B    data
512B    net
512B    proc
512B    sys
4.5K    COPYRIGHT
4.5K    entropy
5.5K    dev
6.5K    mnt
 53K    media
143K    tmp
205K    libexec
924K    bin
2.2M    etc
3.9M    root
4.6M    sbin
6.2M    rescue
6.6M    lib
 90M    boot
117M    compat
564M    jail
667M    var
5.4G    usr
297G    home

This is where ncdu(1) comes handy. Its ncurses based disk usage analyzer which helps finding that space eaters in very fast time without typing the same commands over and over again. Here is ncdu(1) in action.

First it calculates the sizes of the files.

After a while you get the output sorted by size.

If you hit [ENTER] on the directory you will be instantly moved into that directory.

If you delete something with ‘d‘ then remember to recalculate the output with ‘r‘ letter.

It also has great options such as spawning shell ‘b‘ in the current directory or toggle between apparent size and disk usage with ‘a‘ option. The latter is very useful when you use filesystem with builtin compression like ZFS.

       up, k  Move cursor up
     down, j  Move cursor down
 right/enter  Open selected directory
  left, <, h  Open parent directory
           n  Sort by name (ascending/descending)
           s  Sort by size (ascending/descending)
           C  Sort by items (ascending/descending)
           d  Delete selected file or directory
           t  Toggle dirs before files when sorting
           g  Show percentage and/or graph
           a  Toggle between apparent size and disk usage
           c  Toggle display of child item counts
           e  Show/hide hidden or excluded files
           i  Show information about selected item
           r  Recalculate the current directory
           b  Spawn shell in current directory
           q  Quit ncdu

The apparent size using the du(1) command.

Disk usage.

% du -sm books
39145   books

Apparent size.

% du -smA books
44438   books

So I have 1.13 compression ratio on the ZFS filesystem. More then 5 GB saved just in that directory 🙂

Where Are My Files

Once I got some space back I also wanted to know if there are some directories with enormous amount of very small files.

First I came up with my own files-count.sh script solution which is not that long.

#! /bin/sh

export LC_ALL=C

if [ ${#} -eq 0 ]
then
  DIR=.
else
  DIR="${1}"
fi

find "${DIR}" -type d -maxdepth 1 -mindepth 1 \
  | cut -c 3- \
  | while read I
    do
      find "${I}" | wc -l | tr -d '\n'
      echo " ${I}"
    done | sort -n

It works reliably but same as with du | sort tandem you have to retype it (or at least use cd(1) and hit [UP] arrow again) … but then I discovered that ncdu(1) also counts files! It does not provide ‘startup’ argument to start in this count files mode but when you hit ‘c‘ letter it will instantly display count of files in each scanned directory. To sort this output by the count of files hit the ‘C‘ letter (large ‘C‘ letter).

The files-count.sh script still has one advantage over ncdu(1) – the latter stops counting files at 100k which is shown on the screenshot so if You need to search for really big amount of files or just about 100k then files-count.sh script will be more accurate/adequate.

% cd /usr
% files-count.sh 
       1 obj
      36 libdata
     299 sbin
     312 libexec
     390 tests
     498 bin
     723 lib32
     855 lib
    2127 include
   16936 share
  159945 src
  211854 ports
  266021 local

… but what if there were some very big files hidden somewhere deep in the directories tree? The du(1) or ncdu(1) will not help here. As usual I though about short files-big.sh script that will do the job.

#! /bin/sh

export LC_ALL=C

if [ ${#} -eq 0 ]
then
  DIR=.
else
  DIR="${1}"
fi

find "${DIR}" -type f -exec stat -f "%16z; doas rm -f \"%N\"" {} ';' | sort -n

An example usage on the /var directory.

# cd /var
# files-big.sh | tail
        10547304; doas rm -f "./tmp/kdecache-vermaden/icon-cache.kcache"
        29089823; doas rm -f "./db/clamav/clamav-2671b72fce703c2133c61e5bf85aad19.tmp/clamav-373e311ca7f610a39c7cf5c5c5a4fd83.tmp/daily.hdb"
        30138884; doas rm -f "./tmp/pkg-provides-wyK2"
        48271360; doas rm -f "./db/pkg/repo-HardenedBSD.sqlite"
        54816768; doas rm -f "./db/pkg/repo-FreeBSD.sqlite"
        66433024; doas rm -f "./db/pkg/local.sqlite"
        82313216; doas rm -f "./db/clamav/clamav-2671b72fce703c2133c61e5bf85aad19.tmp/clamav-373e311ca7f610a39c7cf5c5c5a4fd83.tmp/daily.hsb"
       117892267; doas rm -f "./db/clamav/main.cvd"
       132431872; doas rm -f "./db/clamav/daily.cld"
       614839082; doas rm -f "./db/pkg/provides/provides.db"

The output is in ‘executable’ format so if you select whole line and paste it into terminal, then this file will be deleted. By default it uses doas(1) but nothing can stop you from putting sudo(8) there. Not sure if you will find it useful but it helped me at least dozen times.

How Many Copies Do You Keep

I often find myself keeping the same files in several places which also wastes space (unless you use ZFS deduplication of course).

The dedup.sh script I once made is little larger so I will not paste it here and just put a link to it.

It has the following options available. You may search/compare files by name or size (fast) or by its MD5 checksum (slow).

% dedup.sh
usage: dedup.sh OPTION DIRECTORY
  OPTIONS: -n   check by name (fast)
           -s   check by size (medium)
           -m   check by md5  (slow)
           -N   same as '-n' but with delete instructions printed
           -S   same as '-s' but with delete instructions printed
           -M   same as '-m' but with delete instructions printed
  EXAMPLE: dedup.sh -s /mnt

Simple usage example.

% cd misc/man
% cp zfs-notes zfs-todo
% dedup.sh -M .
count: 2 | md5: 4ff4be66ab7e5484de2bf7c168ff995a
  doas rm -rf "./zfs-notes"
  doas rm -rf "./zfs-todo"

count: 2 | md5: 6d87f5b1317ea189165fcdc71380735c
  doas rm -rf "./x11"
  doas rm -rf "./xinit"

By copying the zfs-notes file into the zfs-todo file I wanted to show you what dedup.sh will print on the screen, but accidentally I also found another duplicate 🙂

The output of dedup.sh is simple and like with files-big.sh script selecting the while line and pasting it into the terminal will remove the duplicate. By default it uses doas(1) but you can change it into sudo(8) if that works better for you.

Unusual `cron(1)` Intervals

Most of us already remember what the five fields of crontab(5) file mean, but what if you would like to run command every second … or after reboot only? The answer lies in the man 5 crontab page. Here are these exotic options.

string          meaning
------          -------
@reboot         Run once, at startup of cron.
@yearly         Run once a year, "0 0 1 1 *".
@annually       (same as @yearly)
@monthly        Run once a month, "0 0 1 * *".
@weekly         Run once a week, "0 0 * * 0".
@daily          Run once a day, "0 0 * * *".
@midnight       (same as @daily)
@hourly         Run once an hour, "0 * * * *".
@every_minute   Run once a minute, "*/1 * * * *".
@every_second   Run once a second.

Check `cron(1)` Environment

Many times I found myself lost lots of time debugging what went wrong when my script was run by the crontab(5) file. Often it was some variable missing or some command or script I used was not in the PATH variable.

To make that debugging faster You can use ENV.sh script to just store the cron(1) environment.

% cat ENV.sh
env > /tmp/ENV.out

The ENV.sh script will write current environment in the /tmp/ENV.out file.

Lets put it into the crontab(5) for a test.

% crontab -l | grep ENV
@every_second ~/ENV.sh

Now after at most a second you can check for the contents of the /tmp/ENV.out file.

% cat /tmp/ENV.out
LOGNAME=vermaden
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
PWD=/home/vermaden
HOME=/home/vermaden
USER=vermaden
SHELL=/bin/sh

Now you can easily debug the scripts run by the crontab(5) … at least on the environment part 🙂

Simple HTTP Server

I found myself many times in a situation that I would want to allow download of some files from my machine and SSH could not be used.

This is when python(1) comes handy. It has SimpleHTTPServer (or http.server in Python 3 version) so you can instantly start HTTP server in any directory!

Here are the commands for both Python versions.

Python 2.x – python -m SimpleHTTPServer PORT
Python 3.x – python -m http.server PORT

I even made a simple http.sh wrapper script to make it even more easy.

#! /bin/sh

if ${#} -ne 1 ]
then
  echo "usage: ${0##*/} PORT"
  exit 1
fi

python -m SimpleHTTPServer ${1}

Example usage.

% cd misc/man
% http.sh 8080
Serving HTTP on 0.0.0.0 port 8080 ...
127.0.0.1 - - [14/Sep/2018 23:06:50] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [14/Sep/2018 23:06:50] code 404, message File not found
127.0.0.1 - - [14/Sep/2018 23:06:50] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [14/Sep/2018 23:09:15] "GET /bhyve HTTP/1.1" 200 -

To stop it simply hit [CTRL]-[C] interrupt sequence.

Here is how it looks in the Epiphany browser.

Simple FTP Server

Similarly with FTP service, another Python goodie called pyftpdlib (Python FTP Server Library) provides that.

Mine ftp.py wrapper is little bigger as you can write quite comlicated setups with pyftpdlib but mine is simple, it starts in the current directory and adds read only anonymous user and read/write user named writer with WRITER password.

#! /usr/bin/env python

from sys                   import argv,exit
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers    import FTPHandler
from pyftpdlib.servers     import FTPServer

if len(argv) != 2:
  print "usage:", argv[0], "PORT"
  print
  exit(1)
  
authorizer = DummyAuthorizer()
authorizer.add_user("writer", "WRITER", ".", perm="elradfmw")
authorizer.add_anonymous(".")
handler = FTPHandler
handler.authorizer = authorizer
handler.passive_ports = range(60000, 60001)
address = ("0.0.0.0", argv[1])
ftpd = FTPServer(address, handler)
ftpd.serve_forever()

The ftp.py is handy if you want to enable someone to upload something for you (or you are doing it o the other machine) when SSH/SCP is not possible for some reason.

To stop it simply hit [CTRL]-[C] interrupt sequence.

Here is its terminal startup and logs.

% cd misc/man
% ftp.py 2121
[I 2018-09-14 23:21:53] }}} starting FTP server on 0.0.0.0:2121, pid=64399 {{{
[I 2018-09-14 23:21:53] concurrency model: async
[I 2018-09-14 23:21:53] masquerade (NAT) address: None
[I 2018-09-14 23:21:53] passive ports: 60000->60000

… and how Firefox renders its contents.

Hope you will find some of these useful, see you at Part 4 some day.

UPDATE 1 – More Short Functions

As time flies by I also added several other ‘short functions’ that make my life easier. They are related to mine Universal File Opener named see.sh.
This is the part that I added to mine ~/.zshrc shell config.

# SHORT see.sh OPEN ALIASES
  alias s='see.sh'
  alias o='see-pipe-open.sh'

The additional see-pipe-open.sh helper script is meant to be used in pipes to open all files from stdin.
Example below

% ls
bsd.1.pdf  bsd.2.pdf  bsd.png  unix.1.pdf  unix.2.pdf  NOTES.txt

% q bsd
bsd.1.pdf
bsd.2.pdf
bsd.png

% q bsd | g pdf
bsd.1.pdf
bsd.2.pdf

% q bsd | g pdf | o
// see.sh will open bsd.1.pdf with mupdf(1)
// see.sh will open bsd.2.pdf with mupdf(1)

Now – the q bsd | g pdf | o will open bsd.1.pdf and bsd.2.pdf files according to what is configured in the see.sh handler. In my case it mupdf(1) would be used to open both of them.

As for the other s shortcut – its just faster to type s bsd.1.pdf than see.sh bsd.1.pdf to open a file at terminal 🙂

Regards.

EOF

Ghost in the Shell – Part 2

12 Replies

The article in the Ghost in the Shell series was the first post on my blog, so while I was busy by writing various server related articles and recently the FreeBSD Desktop series its about time for the Part 2 of the Ghost in the Shell series.

Lets start with something simple – yet powerful and time saving.

Alias with Arguments

One may of course write any function to do similar job, but keeping track and ‘maintaining’ all those functions becomes complicated and one has to organize itself. This partially applies to aliases, but they are smaller and easier to maintain then whole functions. In any modern shell an alias(1) can also have arguments, while You will not be able to parse them as appropriate as with functions, they do the job for their basic use.

Here is an example of such alias(1) with arguments.

% ls
gfx/ info/ misc/ scripts/ tmp/

% alias lsg='ls | grep'

% lsg gfx
gfx/

Color `grep(1)` Patterns

As we already ‘touched’ the grep(1) command topic, lets make it more usable by highlighting the found results in color. The ${GREP_COLOR} variable is used for that purpose and it expects a number for a color, here is the table with number-color format.

Color    Number
Black    30
Red      31
Green    32
Yellow   33
Blue     34
Magenta  35
Cyan     36
White    37

You may as well use ‘bold’ output by adding ‘1;‘ before the number, for example.

% echo ${GREP_COLOR}
1;31

You will also have to make an alias(1) to grep(1) with --color argument, like that:

% alias grep='grep --color'

Here is how it looks in practice.

% export GREP_COLOR=31
% alias grep='grep --color'
% dmesg | grep SMP
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
SMP: AP CPU #1 Launched!

Here is how it looks on the xterm(1) terminal.

ghost-terminal

Process Management

This one is very useful on any UNIX system, does not matter if its server or desktop.

These are commands and operands that will help us manage processes started by hand:

&
fg
bg
jobs
kill
disown
nohup
[CTRL]+[Z]
[CTRL]+[C]

As you probably already know to start command ‘in the background’ – which means do what I tell you but do not block the terminal – you have to add ‘&‘ (ampersand) at the end of such command. That command does not magically go away and as long as its running its visible by the jobs(1) command. You may use ‘-l‘ switch to also show the PID of background processes.

% galculator &
[1] 8449

% jobs
[1]  + running    galculator

% jobs -l
[1]  + 8449 running    galculator

Now, what of you forget to add ‘&‘ (ampersand) at the end of command but you wanted to put it into the background? Hit [CTRL]+[Z] shortcut (Control key with ‘small’ Z letter) and the process will be put into the suspended state. Now you have several options, you can out that process into the background with bg(1) command – by default it uses last suspended job – %1, you can also bring it back into the foreground blocking the terminal with fg(1) command. You can also list its state with jobs(1) and of course kill(1) it either with PID showed by jobs -l command or by specifying the process number – %1 in that case.

Here is an example.

% galculator
^Z
zsh: suspended  galculator

% jobs
[1]  + suspended  galculator

% bg
[1]  + continued  galculator

% jobs -l
[1]  + 72892 running    galculator

% kill %1
[1]  + terminated  galculator

%

While fg(1) and bg(1) allow you to put command in the background or foreground respectively when the process is in suspended state, one may ask how to ‘switch’ a process to suspended state while its already running in the background. Its done with kill -17 signal called SIGSTOP. You can also bring back such suspended process to running state with kill -19 signal called SIGCONT … or just again use fg(1) or bg(1) command. Other difference between fg(1)/bg(1) commands and more ‘direct’ kill -17/kill -19 commands are that kill(1) does not inform the user what has changed to the process. You may as well use kill -SIGCONT syntax or kill -s SIGCONT if that is more readable for you.

% galculator
^Z
zsh: suspended  galculator

% bg
[1]  + continued  galculator

% xcalc
^Z
zsh: suspended  xcalc

% jobs -l
[1]  - 19537 running    galculator
[2]  + 20563 suspended  xcalc

% kill -17 %1
[1]  + suspended (signal)  galculator

% jobs -l
[1]  + 19537 suspended (signal)  galculator
[2]  - 20563 suspended  xcalc

% kill -SIGCONT %1
% bg %2
[2]  - continued  xcalc

% jobs -l
[1]  + 19537 running    galculator
[2]  - 20563 running    xcalc

Also check man kill and man signal for more information.

What about disown(1) then? Its a ‘magic’ helper when you start some long running jobs directly at the terminal without Screen or Tmux and you need to disconnect that terminal, for example because you are taking your laptop with you. When you do this – depending on the settings of the current shell – the processes in the background may be killed or ‘moved’ to PID 1 (the init(1) of course) as the PPID (Parent PID). To achieve that we will used that disown(1) command. Once you ‘disown’ a process it will no longer be show by the jobs(1) command, but it will run ‘pinned’ to the init(1) process after you disconnect the terminal session.

% galculator
^Z
zsh: suspended  galculator

% bg
[1]  + continued  galculator

% jobs -l
[1]  + 98556 running    galculator

% disown %1

% jobs -l

% pgrep galculator
98556

% pstree -p 98556
─┬◆ 00001 root /sbin/init --
 └─┬─ 48708 vermaden xterm
   └─┬◆ 52463 vermaden -zsh (zsh)
     └──◆ 98556 vermaden galculator

Now its still pinned to the shell in the xterm(1) terminal. After we close the xterm(1) window (or kill that zsh(1) shell) it will switch to init(1) as PPID (Parent PID).

% pstree -p 98556
─┬◆ 00001 root /sbin/init --
 └──◆ 98556 vermaden galculator

% pgrep -P 1 galculator
98556

We are left with nohup(1) then, when and why to use it as we already has great disown(1) magic? Well, disown(1) is not always available, so when You need to put some command into the long background run and disconnect after it its the best possible option. By default the nohup(1) command will log the output of started command into the nohup.out file. Remember that nohup(1) will still run the process in the foreground, to put it into the background use ‘&‘ (ampersand) or [CTRL]+[Z] with bg(1) combo.

% nohup galculator
appending output to nohup.out
^Z
zsh: suspended  nohup galculator

% bg
[1]  + continued  nohup galculator

% jobs -l
[1]  + 22322 running    nohup galculator

% pstree -p 22322
─┬◆ 00001 root /sbin/init --
 └─┬─ 89568 vermaden xterm
   └─┬◆ 91486 vermaden -zsh (zsh)
     └──◆ 22322 vermaden galculator

… and after disconnect out process switched to init(1) as PPID.

% pstree -p 22322
─┬◆ 00001 root /sbin/init --
 └──◆ 22322 vermaden galculator

You may of course end a running process in the foreground with [CTRL]+[C] shortcut, but that is probably already known to you. I just mention it for the ‘completeness’ of the guide.

% galculator
^C

%

Which Which

While the which(1) command shows the full path of the executable found in the first directory of the ${PATH} variable, it also shows what alias is used for that command it there is one. One may ask how then to find information about absolute executable path if it shows and alias(1) instead. Well, you have to use unalias(1) on that command, so which(1) would be showing full path again.

% which caja
caja: aliased to caja --browser --no-desktop

% unalias caja

% which caja
/usr/local/bin/caja

Also be sure to check Smylers comment below about the difference between shell builtin which and /usr/bin/which command.

The difference is that by typing which you are executing your shell builtin command (ZSH in my case) which also takes aliases into account. If you want to omit the unalias part then use /usr/bin/which which will ignore any existing aliases.

% which caja
caja: aliased to caja --browser --no-desktop

% /usr/bin/which caja
/usr/local/bin/caja

Record Session

If you have used PuTTY or MobaXterm in your work, then you appreciate the possibility of saving the terminal output to a file, foe example for the documentation purposes. This is also available ‘natively’ in the shell by using the script(1) command. Remember that script(1) will record also ‘special’ characters like colors, so to properly ‘replay’ the session you may want to either use script(1) or cat(1) commands for that or use less with -R argument.

Here is example recorded script(1) session.

% script script.out
Script started, output file is script.out

% ls
gfx info misc scripts tmp unix.png

% uname -spr
FreeBSD 11.2-RELEASE amd64

% exit
Script done, output file is script.out

% cat script.out
Script started on Sun Jul  8 08:24:06 2018
You have mail.
% ls | grep gfx
gfx
% uname -spr
FreeBSD 11.2-RELEASE amd64
% exit
exit

Script done on Sun Jul  8 08:24:20 2018

% less -R script.out
Script started on Sun Jul  8 08:24:06 2018
You have mail.
% ls | grep gfx
gfx
% uname -spr
FreeBSD 11.2-RELEASE amd64
% exit
exit

Script done on Sun Jul  8 08:24:20 2018

% less script.out
Script started on Sun Jul  8 08:24:06 2018
You have mail.
% ls | grep gfx
ESC[1;31mgfxESC[00mESC[K
% uname -spr
FreeBSD 11.2-RELEASE amd64
% exit
exit

Script done on Sun Jul  8 08:24:20 2018

Edit Command Before Executing

Sometimes you have long multi-line command to execute, so often it is crafted in you favorite ${EDITOR} and then pasted into the terminal. To omit copying and pasting yo may want to check fc(1) command which serves similar purpose. After you type a command, for example simple ls(1) command, and then you type fc(1) command, then fc(1) will take that ls(1) command into your favorite text editor from ${EDITOR} variable, will allow you to edit it and if you save and exit the that editor, it will execute it.

Lets see how it behave by example.

% ls
gfx   books   download   scripts

% fc

Now you are taken into the ${EDITOR} which is vi(1) in my case.

      1 ls
~
~
~
/tmp/zsh999EQ6: unmodified: line 1

Lets made some changes.

      1 ls -l \
      2    -h
~
~
~
~

:wq

After you hit [ENTER] it will exit from ${EDITOR} and execute that command.

total 6181
drwxr-xr-x    87 vermaden  vermaden    87B 2017.12.18 15:30 books/
drwxr-xr-x    12 vermaden  vermaden    12B 2018.06.19 16:02 download/
drwxr-xr-x    19 vermaden  vermaden    20B 2018.05.24 11:52 gfx/
drwx------    12 vermaden  vermaden   310B 2018.07.07 03:23 scripts/

You may show that command by pressing [Up] key to check what has been executed.

% ls -l -h

Edit or Just View

When working in multi-admin environment – especially while debugging – one admin may block other admin’s work by using vi(1) – or just their favorite editor to ‘browse’ the file contents. Good practice in that case is using more(1) or less(1) instead of vi(1), but that frustrates some admins to type vi(1) again if they need to change something.

… and by the way, on FreeBSD more(1) is less(1) 🙂

% uname -spr
FreeBSD 11.2-RELEASE amd64

% ls -i `which less` `which more`
492318 /usr/bin/less  492318 /usr/bin/more

A blocked ‘example’ is shown below when the second admin wanted to browse the /etc/rc.conf file while the first one already did that.

# vim /etc/rc.conf

E325: ATTENTION
Found a swap file by the name "/etc/.rc.conf.swp"
          owned by: root   dated: Sun Jul  8 08:38:35 2018
         file name: /etc/rc.conf
          modified: no
         user name: root   host name: t420s.local
        process ID: 54219 (still running)
While opening file "/etc/rc.conf"
             dated: Fri Jul  6 00:51:11 2018

(1) Another program may be editing the same file.  If this is the case,
    be careful not to end up with two different instances of the same
    file when making changes.  Quit, or continue with caution.
(2) An edit session for this file crashed.
    If this is the case, use ":recover" or "vim -r /etc/rc.conf"
    to recover the changes (see ":help recovery").
    If you did this already, delete the swap file "/etc/.rc.conf.swp"
    to avoid this message.

Swap file "/etc/.rc.conf.swp" already exists!
[O]pen Read-Only, (E)dit anyway, (R)ecover, (Q)uit, (A)bort:

This is where less(1) comes handy because of you open a file in it, you do not ‘block’ access to it and if you need to edit something just hi [V] key (small ‘v’ letter). It will open that file in your ${EDITOR} editor and you can make any changes now.

Reset

Last but not least, often when you paste ‘too much’ into the terminal it becomes ‘fragile’ or ‘broken’. To reset it into the ‘stable’ and ‘proper’ state just use the reset(1) command.

% reset

Hope You find it useful, see you at the Part 3 sometime 😉

EOF

FreeBSD Desktop – Part 3 – X11 Window System

37 Replies

In this article I would like to cover setting up the X11 Window System on FreeBSD.

x11

I always get the impression that the X11 configuration/setup seems to be the seen as hard or weird. I will try to gather all useful information about that topic regarding the FreeBSD operating system. A lot of that is covered in the FreeBSD Handbook – 5.4. Xorg Configuration – section.

BIOS or UEFI

First things first, if You find a device that is not supported by any ‘accelerated’ driver like ‘intel‘ or ‘nvidia‘ You would use ‘vesa‘ driver (Video Electronics Standards Association) while booting in BIOS mode and You will use ‘scfb‘ driver (System Console Frame Buffer) while booting on UEFI mode. This can be checked by machdep.bootmethod sysctl(8) parameter.

% sysctl machdep.bootmethod
machdep.bootmethod: BIOS

This way You will know if you will use ‘scfb‘ driver or ‘vesa‘ driver when troubleshooting or when just running the ARM system on which ‘scfb‘ will be used.

Packages

We will install the X11 server, the XDM login manager, the XTERM terminal emulator and the Openbox window manager. We will also install the Intel graphics card driver. To achieve that type the command below.

# pkg install xorg xdm xterm openbox xf86-video-intel

Of course You will need to have network connection configured but we covered that in earlier parts.

Group

To have hardware accelerated X11 You will also need to add your regular user to the ‘video‘ group.

# pw groupmod video -m username

Login Class

Remember to also add UTF-8 as the default encoding (charset and lang parameters) in the /etc/login.conf file as show below. Also, after adding them, run the cap_mkdb /etc/login.conf command as root and relogin with your regular user.

Below are shown the lines that need to be added to the default: profile in the /etc/login.conf file.

(...)

 default:\
         :passwd_format=sha512:\
         :copyright=/etc/COPYRIGHT:\
         :welcome=/etc/motd:\
         :setenv=MAIL=/var/mail/$,BLOCKSIZE=K:\
         :path=/sbin /bin /usr/sbin /usr/bin /usr/local/sbin /usr/local/bin ~/bin:\
         :nologin=/var/run/nologin:\
         :cputime=unlimited:\
         :datasize=unlimited:\
         :stacksize=unlimited:\
         :memorylocked=64K:\
         :memoryuse=unlimited:\
         :filesize=unlimited:\
         :coredumpsize=unlimited:\
         :openfiles=unlimited:\
         :maxproc=unlimited:\
         :sbsize=unlimited:\
         :vmemoryuse=unlimited:\
         :swapuse=unlimited:\
         :pseudoterminals=unlimited:\
         :kqueues=unlimited:\
         :umtxp=unlimited:\
         :priority=0:\
         :ignoretime@:\
+        :charset=UTF-8:\
+        :lang=en_US.UTF-8:\
         :umask=022:
(...)

The diff(1) after/before modification looks like that.

# diff -u login.conf.ORG login.conf
--- login.conf.ORG      2018-05-22 07:10:09.864831000 +0200
+++ login.conf  	2017-07-07 04:47:04.643412000 +0200
@@ -46,6 +46,8 @@
        :umtxp=unlimited:\
        :priority=0:\
        :ignoretime@:\
+       :charset=UTF-8:\
+       :lang=en_US.UTF-8:\
        :umask=022:

The UTF-8 encoding – same as in the lang parameter in the /etc/login.conf file – must be added at the beginning of your ~/.xinitrc file, like that:

% grep LC_ALL ~/.xinitrc
# SET PROPER locale(1) with LC_ALL VARIABLE.
  export LC_ALL=en_US.UTF-8

Kernel Module

In 2010 Intel introduced Nehalem architecture which have integrated GPU in the CPU (and memory controller which AMD had in CPU since 2003). This means that most computers/laptops/servers (Intel has about 80% market share) will have integrated GPU which should be supported by the X11 ‘intel‘ driver and:

/boot/kernel/i915.ko from base system (older)
/boot/kernel/i915kms.ko from base system (newer)
/boot/modules/i915kms.ko from ‘drm-next-kmod‘ package

The ‘drm-next-kmod‘ package is available for FreeBSD 12-CURRENT development branch and upcoming FreeBSD 11.2-RELEASE (currently at BETA2 stage) about which You can read here on FreeBSD Mailing Lists on https://lists.freebsd.org/pipermail/freebsd-stable/2018-February/088406.html and https://svnweb.freebsd.org/ports?view=revision&revision=462202 accordingly.

You will probably be using the builtin KMS module which is the /boot/kernel/i915kms.ko file. You may configure loading of this module in either /boot/loader.conf (loaded at boot menu) or in the /etc/rc.conf main FreeBSD configuration file – which will load modules during the startup process. I prefer the second option as it is faster for FreeBSD system to boot while loading the kernel modules from the /etc/rc.conf file along with starting needed services then lading them one by one from /boot/loader.conf file. To keep system startup time to minimum we will have only one kernel module configured to load from the /boot/loader.conf file – the ZFS module with zfs_load=YES line.

If you found yourself with situation that KMS modules from the base system will not support your graphics card then You will have to try the ‘drm-next-kmod‘ package for latest support of Intel graphics chips, that sometimes requires running FreeBSD CURRENT version which is not a problem, you will get all the new stuff instantly as a bonus 😉

To add the KMS (Kernel Mode Setting) Intel module loading into the /etc/rc.conf file put the line below into it.

# KERNEL MODULES
  kld_list="${kld_list} i915kms"

We can also add other useful kernel module loading such as fuse (for alien filesystems), aesni (for encryption acceleration if You CPU has these extensions) geom_eli (for the encryption itself) or coretemp (for the temperature sensor). I will cover all of these kernel module in the future series with exact explanation what are they for.

With all of them the /etc/rc.conf kld_list variables will look like that:

# grep kld_list /etc/rc.conf 
  kld_list="${kld_list} i915kms aesni geom_eli"
  kld_list="${kld_list} fuse coretemp cpuctl"

In case of need to use ‘drm-next-kmod‘ module instead of the base system one, your /etc/rc.conf file would look like that one below.

# grep kld_list /etc/rc.conf
  kld_list="${kld_list} /boot/modules/i915kms.ko aesni geom_eli"
  kld_list="${kld_list} fuse coretemp cpuctl"

If you find yourself in the situation that the ‘drm-next-kmod‘ would still not support your latest graphics card then you may try the latest version of the ‘drm-next-kmod‘. Current ‘drm-next-kmod‘ is on par with Linux 4.11 when it comes to graphics support, the latest WIP (work in progress) tree is at 4.15, so that may enable some graphics cards, anyway, here it is – https://github.com/FreeBSDDesktop/freebsd-base-graphics/tree/drm-v4.15-WIP – use at your own risk. 😉

X11 Window System Configuration

Historically You would create entire /etc/X11/xorg.conf file which would include complete X11 Window System configuration. Recently to comply with FreeBSD hier(7) directory structure and logic this can be also configured as /usr/local/etc/X11/xorg.conf file … and even more recently You can just configure these parts of X11 server that You need without touching other parts. This ‘individual’ configuration is done in the /usr/local/etc/X11/xorg.conf.d directory with individual files for each setting, like card.conf for graphics card configuration.

As the latter method is the most modern one we will use that in this guide for the sake of future installations.

# cat /usr/local/etc/X11/xorg.conf.d/flags.conf
Section "ServerFlags"
  Option "DontZap" "off"
EndSection

The DontZap option is to enable the X11 server abort with [CTRL] + [ALT] + [BACKSPACE] shortcut. I found it useful several times when none other option worked. Only X11 server required restart. I could of course login to this laptop via SSH and kill needed processes, but I do not carry two laptops with me to be prepared for such situations 😉

Next we will configure the keyboard with Polish layout and two additional options, terminate:ctrl_alt_bksp to allow termination of the X11 server with [CTRL] + [ALT] + [BACKSPACE] shortcut and ctrl:nocaps to ignore the Caps Lock key as I also find that useful, if you not, then do not enable that option.

# cat /usr/local/etc/X11/xorg.conf.d/keyboard.conf
Section "InputDevice"
  Identifier "Keyboard0"
  Driver "kbd"
  Option "XkbLayout" "pl"
  Option "XkbOptions" "terminate:ctrl_alt_bksp,ctrl:nocaps"
EndSection

The last one is the graphics card configuration, we will put ‘intel‘ here as our graphics is fully supported by it.

# cat /usr/local/etc/X11/xorg.conf.d/card.conf
Section "Device"
  Identifier "Card0"
  Option "DPMS"
  Driver "intel"
EndSection

We will also enable DPMS to save power when its possible.

XDM or XINIT/STARTX

You will also have to decide how You want to start your X11 Window Server, you may login in plan text console and then type xinit(1) or startx(1) to read your ~/.xinitrc configuration and daemons or You may want to use X11 Login manager such as xdm/sddm/slim with ~/.xsession configuration to load after successful login. The difference between xinit and startx is that startx command executes xinit command with arguments like -nolisten tcp to not listen for the external connections for example. Its pointless to start another layer of binary (startx) just to pass an argument to command, so we will omit startx and go directly to xinit command with suitable alias.

Which is better – some login manager or directly starting X11 with xinit? Both are good, it only depends on your preferences. For example I used text login and then typed ‘x‘ as an alias to xinit command and recently switched to xdm along with symlink of ~/.xsession which leads to the very same ~/.xinitrc file and also like it.

When I use xinit to directly start the X11 server I use an alias to xinit with these parameters:

alias x='xinit ~/.xinitrc -- -dpi 75 -nolisten tcp'

Of course I keep that alias in the ~/.zshrc file (for the ZSH shell) so everytime I login in the text console I just type ‘x‘ and X11 session starts reading contents of my ~/.xinitrc file.

While xinit run commands based on the ~/.xinitrc file the XDM login manager looks for the ~/.xsession file. As You will be loading same stuff regardless of the startup method we will create a link of ~/.xsession pointing to the ~/.xinitrc file. This way either method You choose You will always end with started X11 session.

% ln -s ~/.xinitrc ~/.xsession

% ls -l ~/.xsession
lrwxr-xr-x  1 vuk  vuk  8 2018.05.22 00:27 .xsession -> .xinitrc

One more case about the ~/.xinitrc (or ~/.xsession) file. It is interpreted as a shell script (and yes you can do if/then/else/fi and case/esac or for/while POSIX shell scripting in it) but it does not need to be executable. The last command in this file MUST NOT to be put in the background (must be without the & char at the end) because the X11 session will end.

XDM Black Theme

I prepared two nice looking themes for XDM login manager, the first one uses black background and grey/white fonts.

To configure this black XDM theme You will need to modify contents of two files as show below.

# cat /usr/local/etc/X11/xdm/Xsetup_0
xsetroot -solid black

This we will set black background on the login screen on the :0 X11 server – the default one.

# cat /usr/local/etc/X11/xdm/Xresources
xlogin.Login.greeting:
xlogin.Login.unsecureGreeting:
xlogin.Login.fail:                Fail.
xlogin.Login.changePasswdMessage: Change.
xlogin.Login.namePrompt:          Username:
xlogin.Login.passwdPrompt:        Password:
xlogin.Login.echoPasswd:          true
xlogin.Login.background:          black
xlogin.Login.foreground:          white
xlogin.Login.failColor:           #cccccc
xlogin.Login.inpColor:            black
xlogin.Login.promptColor:         #aaaaaa
xlogin.Login.face:                courier:size=13:style=Bold
xlogin.Login.failFace:            courier:size=13:style=Bold
xlogin.Login.promptFace:          courier:size=13:style=Normal
xlogin.Login.greetFace:           courier:size=13
xlogin.Login.width:               400

… and the rest of the XDM login manager configuration is kept in the Xresources file, at least the appearance part 😉

Wonder what font will be chosen when we put ‘courier‘ in the *face options? Lets check with fc-match(1) command.

% fc-match courier  
c0419bt_.pfb: "Courier 10 Pitch" "Regular"

You have the answer, You do not always have to type whole font name in the configuration files.

Here is the screenshot of this black theme in action.

xdm.BLACK

XDM Grey Theme

If You did not liked the black XDM theme maybe the grey one will suit your taste better.

# cat /usr/local/etc/X11/xdm/Xsetup_0
xsetroot -solid gray50

We will use the gray50 predefined X11 server color for the background.

# cat /usr/local/etc/X11/xdm/Xresources
xlogin.Login.greeting:
xlogin.Login.unsecureGreeting:
xlogin.Login.fail:                Fail.
xlogin.Login.changePasswdMessage: Change.
xlogin.Login.namePrompt:          Username:
xlogin.Login.passwdPrompt:        Password:
xlogin.Login.echoPasswd:          true
xlogin.Login.background:          gray50
xlogin.Login.foreground:          white
xlogin.Login.failColor:           #dddddd
xlogin.Login.inpColor:            gray50
xlogin.Login.promptColor:         #cccccc
xlogin.Login.greetColor:          gray50
xlogin.Login.promptColor          gray50
xlogin.Login.hiColor              gray50
xlogin.Login.shdColor             gray50
xlogin.Login.face:                courier:size=13:style=Bold
xlogin.Login.failFace:            courier:size=13:style=Bold
xlogin.Login.promptFace:          courier:size=13:style=Bold
xlogin.Login.greetFace:           courier:size=13:style=Bold
xlogin.Login.frameWidth:          0
xlogin.Login.innerFramesWidth:    0
xlogin.Login.sepWidth:            0
xlogin.Login.width:               32768
xlogin.Login.height:              32768
xlogin.Login.x:                   0
xlogin.Login.y:                   0

… and knowing the XDM theming limitations we will customize to mimic the fullscreen of grey color. The only downside of that approach is that Username: and Password: fields will be placed in the top/left side of the screen.

Here is the screenshot of this grey theme in action.

xdm.GREY

Client Configuration

Last but not least, we will configure the simple ~/.xinitrc file with Openbox window manager, prestarted xterm terminal and black background for our simple session.

Here are the contents of the ~/.xinitrc file (to which ~/.xsession links of course).

% cat ~/.xinitrc
# SET PROPER locale(1) with LC_ALL VARIABLE.
  export LC_ALL=en_US.UTF-8

# PRESTARTED APPS
  xterm &
  xsetroot -solid black &

# WINDOW MANAGER
  openbox

It will not be very pretty – yet – but we will cover that later in the series.

desktop

That’s all for today’s X11 configuration, hope that was useful.

UPDATE 1 – The `devd(8)` Backend and `AutoAddDevices` Option

As user seschwar on Lobsters suggested:

“Xorg now has a devd(8) backend it can use to get informed about hotplugged devices instead of hald(8): https://lists.freebsd.org/pipermail/freebsd-x11/2017-March/018978.html That’s working fine for me, even without moused(8).”

Thank you for that information, this may be also verified by checking the X11 log /var/log/Xorg.0.log file.

# grep devd /var/log/Xorg.0.log
[ 96890.425] (II) The server relies on devd to provide the list of input devices.
        If no devices become available, reconfigure devd or disable AutoAddDevices.
[ 96890.955] (II) config/devd: probing input devices...
[ 96890.955] (II) config/devd: adding input device (null) (/dev/kbdmux)
[ 96890.956] (**) Option "config_info" "devd:kbdmux"
[ 96890.960] (II) config/devd: kbdmux is enabled, ignoring device ukbd0
[ 96890.960] (II) config/devd: kbdmux is enabled, ignoring device atkbd0
[ 96890.960] (II) config/devd: adding input device (null) (/dev/sysmouse)
[ 96890.961] (**) Option "config_info" "devd:sysmouse"
[ 96890.962] (II) config/devd: device /dev/ums0 already opened
[ 96890.962] (II) config/devd: device /dev/psm0 already opened

I will modify the original post to not confuse future readers.

UPDATE 2 – Using `modesetting` Driver

As user alx82 on Reddit suggested:

“If you use i915kms kernel module, you don’t need to install xf86-video-intel, the modesetting generic DDX driver that is built in the X server, will do just well. The xf86-video-intel might even cause you troubles, as you mentioned, for example my x230 was rebooting instead of suspending when I had installed the xf86-video-intel (I had installed it to solve the tearing problem with xfwm4 using the xf86-video-intel’s SNA acceleration). Now removed, my x230 suspends no problem.”

Thank you for that information.

I have tried below configuration with ‘modesetting’ driver and it works without a problem. I did not had any issues with ‘intel’ driver either but I will stick with ‘modesetting’ driver for a while see how it behaves on my machine.

# cat /usr/local/etc/X11/xorg.conf.d/card.conf
Section "Device"
  Identifier "Card0"
  Option "DPMS"
  Driver "modesetting"
EndSection

# cat /usr/local/etc/X11/xorg.conf.d/flags.conf
Section "ServerFlags"
  Option "DontZap" "off"
EndSection

# cat /usr/local/etc/X11/xorg.conf.d/keyboard.conf
Section "InputDevice"
  Identifier "Keyboard0"
  Driver "kbd"
  Option "XkbLayout" "pl"
  Option "XkbOptions" "terminate:ctrl_alt_bksp,ctrl:nocaps"
EndSection

To verify which driver You use You can check the X11 log /var/log/Xorg.0.log file.

Here is ‘intel‘ driver being used:

# grep -c intel /var/log/Xorg.0.log
98

# grep -c modeset /var/log/Xorg.0.log
8

# grep DRI /var/log/Xorg.0.log
[ 95305.483] (II) intel(0): [DRI2] Setup complete
[ 95305.483] (II) intel(0): [DRI2]   DRI driver: i965
[ 95305.483] (II) intel(0): [DRI2]   VDPAU driver: va_gl
[ 95305.484] (II) intel(0): DRI2: Enabled
[ 95305.484] (II) intel(0): DRI3: Disabled
[ 95305.549] (II) GLX: Initialized DRI2 GL provider for screen 0

Here is ‘modesetting‘ driver being used:

# grep -c intel /var/log/Xorg.0.log
0

# grep -c modeset /var/log/Xorg.0.log
80

# grep DRI /var/log/Xorg.0.log
[ 95556.446] (II) glamor: EGL version 1.4 (DRI2):
[ 95556.671] (II) modeset(0): [DRI2] Setup complete
[ 95556.671] (II) modeset(0): [DRI2]   DRI driver: i965
[ 95556.671] (II) modeset(0): [DRI2]   VDPAU driver: i965
[ 95556.714] (II) GLX: Initialized DRI2 GL provider for screen 0

UPDATE 3 – Using Dedicated `video:` Login Class Instead of Global One

As Lorenzo suggested in the comments below.

“If you modify /etc/login.conf, you are modifying the systemwide locale for the default login class to which all users belong (also user www for Apache, for example).”

We can of course created dedicated login class for the X11 Window System, as we use group video already we will create video: login class for the X11 Window System.

I assume that the default: login class is untouched – if You already modified it with lang and charset options, remove them.

Add this login class to the /etc/login.conf file.

video:\
        :charset=UTF-8:\
        :lang=en_US.UTF-8:\
        :tc=default:

Rebuild the login class database.

# cap_mkdb /etc/login.conf

How the account looks before setting the login class.

# grep vuk /etc/master.passwd
vuk:{REMOVED}:1000:1000::0:0:vuk:/home/vuk:/bin/sh

Lets set the login class to video for the vuk user.

# pw usermod -L video -n vuk

How the account looks after setting the login class.

# grep vuk /etc/master.passwd
vuk:{REMOVED}:1000:1000:video:0:0:vuk:/home/vuk:/bin/sh

Now logout and login again to make that work.

Hope that helps.

UPDATE 4 – Tap to Click on Synaptics Touchpad

After upgrade FreeBSD from 12.1-RELEASE to 12.2-RELEASE the tap to click on (not only mine touchpad on ThinkPad W520 machine stopped working.

There is also option called NaturalScrolling which enables the natural scrolling mode. It means that if you use two fingers to slide up on the touchpad the content of the window will move up (like scrolling down with mouse).

The ‘not’ natural scrolling mode means that sliding two fingers up on the touchpad will move the content of window down (like scrolling up with the mouse).

After some time I fell that natural scrolling is more … natural/logical for me so I switched to it.

Below configuration fixes that and tap to click works again.

# pkg install xf86-input-synaptics

# cat << BSD > /usr/local/etc/X11/xorg.conf.d/touchpad.conf
Section "InputClass"
  Identifier "touchpad"
  MatchIsTouchpad "on"
  Driver "libinput"
  Option "Tapping" "on"
  Option "NaturalScrolling" "on"
EndSection
BSD

After these steps restart the X11 server and tap to click should work properly again.

Hope that helps.

UPDATE 5 – VAAPI and Tear Free on Intel Cards

The VAAPI is also known as Video Acceleration API – its open source API that allows applications such as mpv(1) or vlc(1) media players to use hardware video acceleration capabilities – usually provided by the GPU.

About Tear Free option … some people praise Wayland as tear free experience … well guess what you can have the same on X11 too. It will provide at most 1 frame of latency to the X11 output while using little more memory for additional copying but besides that nothing changes. The input latency does not change.

Here is the detailed information about Tear Free from Intel.

Disable or enable TearFree updates. This option forces X to perform all rendering to a backbuffer prior to updating the actual display. It requires an extra memory allocation the same size as a framebuffer, the occasional extra copy, and requires Damage tracking. Thus enabling TearFree requires more memory and is slower (reduced throughput) and introduces a small amount of output latency, but it should not impact input latency. However, the update to the screen is then performed synchronously with the vertical refresh of the display so that the entire update is completed before the display starts its refresh. That is only one frame is ever visible, preventing an unsightly tear between two visible and differing frames. Note that this replicates what the compositing manager should be doing, however TearFree will redirect the compositor updates (and those of fullscreen games) directly on to the scanout thus incurring no additional overhead in the composited case. Also note that not all compositing managers prevent tearing, and if the outputs are rotated, there will still be tearing without TearFree enabled.

To achieve this we will need to add these two packages:

# pkg install libvdpau-va-gl libva-intel-driver

The vlc(1) media player does not need any configuration to use VAAPI. It will just ‘pick it up’ automatically.

% vlc VIDEO.mp4
(...)
libva info: va_openDriver() returns 0
(...)

To use VAAPI in mpv(1) you will need to add hwdec=vaapi option in the ~/.config/mpv/mpv.conf config file.

% cat << MPV >> ~/.config/mpv/mpv.conf
hwdec=vaapi

MPV

% mpv VIDEO.mp4
(...)
Using hardware decoding (vaapi).
(...)

Above terminal outputs from mpv(1) or vlc(1) media players will let you know that they are using VAAPI successfully.

Lets get back to Intel Tear Free option – when enabled – will be shown as follows in the /var/log/Xorg.0.log file.

% grep -i tear /var/log/Xorg.0.log
[  6356.991] (**) intel(0): Option "TearFree" "true"
[  6356.993] (**) intel(0): TearFree enabled

To enable it use the following X11 config for the graphics card.

% cat << X11 > /usr/local/etc/X11/xorg.conf.d/card.conf
Section "Device"
  Identifier "Card0"
  Driver "intel"
  Option "TearFree" "true"
  Option "DPMS"
EndSection

X11

After these steps restart the X11 server to make these changes work.

Also a note about the Tear Free option. I do not recall any tearing before enabling it so I generally do not see any ‘benefit’ of enabling it, but maybe its just me. I have read that some people see noticeable difference (for the good) when they enabled the Tear Free option.

Regards.

EOF

Ghost in the Shell – Part 1

11 Replies

I wanted to post this earlier, but the busy daily life does not help 😉

This will be first article in the series about efficient working in the shell environment. There are actually a lot articles and blog posts about efficient working in the terminal, but a lot of them are biased towards very specific uses, like hints only for Bash shell or only for specific terminal emulator. For example Moving efficiently in the CLI.

These series are about universal knowledge that would work on most shells and environments. Lets start with hint that I use many times a day that saves a lot time for not having to type …

Recall Last Argument of Previous Command

Imagine most simple scenario, creating directory and entering it. Typically its like that:

% mkdir clear-place-for-new-work
% cd clear-place-for-new-work
%

The longer the name, the bigger the chance that You would type mkdir, then hit the [UP] arrow, then [HOME] or [CTRL]+[A] keys and then put cd in the place of mkdir.

With the use of !$ You can recall last argument of the precious command, so it will now look like that.

% mkdir clear-place-for-new-work
% cd !$
cd clear-place-for-new-work
%

Faster isn’t it?

Swap First Occurrence of a Word

The upper example can be used for the next advice as well. By typing ^fromwhat^towhat in the terminal You will swap the first occurrence of word fromwhat word to towhat word in the previous command, lets see how its working.

% mkdir clear-place-for-new-work
% ^mkdir^cd
cd clear-place-for-new-work
%

It still takes more time to write then using the !$ so its useful mostly when there are short things to swap, like numbers, for example ^3^4 to ‘move’ from one target to another. … or also if You can not recall to the last argument of previous command.

There and Back Again

A lot people does not know, that You can go back to previous working directory with dash. Lets assume that You need to get to /tmp directory for one command and get back to where You were to continue the work. Here is an example.

% pwd
/usr/local/etc/bareos/bareos-dir.d/jobdefs
% cd /tmp
% pwd
/tmp% (do needed work in /tmp dir)
% cd -
/usr/local/etc/bareos/bareos-dir.d/jobdefs
% pwd
/usr/local/etc/bareos/bareos-dir.d/jobdefs

You can even create entire directory stack with pushd/popd commands if needed, check Wikipedia article on that for more information. You can also use ${OLDPWD} variable. Useful with umount command for example.

% pwd
/media/backup-pendrive-key
% cd ~
% umount $OLDPWD
% pwd
/home/vermaden

Repeat Command from History

With exclamation mark (!) You can re-invoke the command from history with all its arguments (which sometimes can be risky). For example.

% !pkg
pkg update -f
(runs actual command)
%

Its better to first check what arguments have been used in that command, that is where :p comes handy. Here is its example usage.

% !pkg:p
pkg update -f
(just prints command without running it)
% !pkg
pkg update -f
(runs actual command)
%

Now, as arguments are known its safe to re-invoke the command with arguments. When this can be dangerous? Can ls command can be dangerous, that depends what You have on Your history, check the example below.

% ls | while read I; do rm -f ${I}; done

This command first lists the contents of the current working directory with ls command, then the output is piped to the while loop which invokes rm -f command for each item listed by ls command, which efficiently removes all non-hidden files in current working directory … which probably is not what we mean by typing !ls on the command prompt ;). That is why its valuable to first check what arguments were used with !ls:p syntax.

Enough for now, I will write more parts with more hints on how to efficiently work in the shell/terminal environment.

UPDATE 1

The Ghost in the Shell – Part 1 article was included in the BSD Now 241 – Bowling in the LimeLight episode.

Thanks for mentioning!

UPDATE 2

About Recall last argument of previous command section … there is also $_ that does similar thing as !$ but there is little difference. The !$ is ‘line oriented’ while $_ is ‘previous command oriented’. Below is an example that shows the difference in the behavior.

The $! takes value from last command in ‘previous line’ which means that '-l' value will be used from line 001 and not 'asd' from the current line 002 from previously executed command.

001 % ls -l
002 % echo asd; ls !$ | tail -2
echo asd; ls -l | tail -2
asd
// ls output //

The $_ takes value from last executed command, thus it points at 'asd' used on line 002 and not at '-l' used at previous 001 line.

001 % ls -l
002 % echo asd; ls $_ | tail -2
asd
ls: asd: No such file or directory

On BASH shell there is also [ALT]-[.] shortcut that switches between $! from previous lines. To achieve the same shortcut on ZSH use this line below in ZSH config.

bindkey '\e.' insert-last-word

Thank you Zachery Purnell for pointing that out.

EOF

Test Environment and Process

RAM Usage Results

Clean Text Console FreeBSD

XFCE

MATE

KDE/Plasma

Openbox

GNOME

CPU Time Usage Results

Subjective Experience

Summary

UPDATE 1 – Added freecolor(1) and htop(1) RAM Measurements

Share this:

Like this:

Why ZSH Shell Anyway?

System Config

User Config

Summary

UPDATE 1 – fish(1) Coloring and Completion in zsh(1)

Share this:

Like this:

Basics

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Summary

Share this:

Like this:

xterm(1)

Proper Font Selection

Selecting Text for Copy/Paste

Selection Buffers and Keyboard Shortcuts

Increase/Decrease Font Size on the Fly

Copy/Paste with [CTRL]+[SHIFT]+[C/V]

Interactive Menus

Random xterm(1) Theme

urxvt(1)

Tabbed Interface

Daemon and Client Mode

Random urxvt(1) Theme and Background

sakura(1)

Different sakura(1) Themes

RAM Usage Comparison

CPU Time Usage Comparison

Share this:

Like this:

Less More Useful

Detox These Filenames

Man Up the Info Pages

Real UNIX Sorting

Faster Better Uptime

Filter Huge Files

Share this:

Like this:

Conky

Xbindkeys

Scripts and Configs

UPDATE 2 – Latest Conky 1.11 Also Works

Share this:

Like this:

Query Functions

Where Is My Space

Where Are My Files

How Many Copies Do You Keep

Unusual cron(1) Intervals

Check cron(1) Environment

Simple HTTP Server

Simple FTP Server

UPDATE 1 – More Short Functions

Share this:

Like this:

Alias with Arguments

Color grep(1) Patterns

Process Management

UPDATE 1 – Added `freecolor(1)` and `htop(1)` RAM Measurements

`xterm(1)`

Random `xterm(1)` Theme

`urxvt(1)`

Random `urxvt(1)` Theme and Background

`sakura(1)`

Different `sakura(1)` Themes

Unusual `cron(1)` Intervals

Check `cron(1)` Environment

Color `grep(1)` Patterns

UPDATE 1 – The `devd(8)` Backend and `AutoAddDevices` Option

UPDATE 2 – Using `modesetting` Driver

UPDATE 3 – Using Dedicated `video:` Login Class Instead of Global One