Author Archives: vermaden

Valuable News – 2018/09/22

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX or BSD systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here so someone else can

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

UNIX

In Other BSDs for 2018/09/15.
https://www.dragonflydigest.com/2018/09/15/21792.html

FreeBSD 12.0-ALPHA6 Available.
https://lists.freebsd.org/pipermail/freebsd-snapshots/2018-September/000450.html

Colin Percival reduced FreeBSD startup time from 10627ms (11.2) to 4738ms (12.0).
https://twitter.com/cperciva/status/1041433506453155840


OpenBSD/NetBSD on FreeBSD using grub2-bhyve.

https://oshogbo.vexillium.org/blog/53/

SoloBSD 11.2-STABLE-0916 Viva Mexico Edition.
https://www.solobsd.org/index.php/2018/09/16/solobsd-11-2-stable-0916-viva-mexico-edition/

HardenedBSD Foundation Announced.
https://hardenedbsd.org/article/shawn-webb/2018-09-17/announcing-hardenedbsd-foundation

Insufficiently known POSIX shell features.
https://apenwarr.ca/log/20110228

OpenBSD VMM Howto.
https://gist.github.com/tvlooy/fd6bc5a77bc03f4d419f395dfcf4f038

ZFS Hardware Acceleration with QAT (Intel Quick Assist Technology).
https://drive.google.com/file/d/0B_J4mRfoVJQRV3ZOd1ZMWkphcV9OYXdWT0FBblVHbVZpSmZj/view

Absolute FreeBSD 3rd Edition now with 40% off using RIGHT2READ40 code.
https://twitter.com/mwlauthor/status/1042090426906370054
https://nostarch.com/absfreebsd3

VirtualBox VM Escape – From Compiler Optimization to Code Execution – CVE-2018-2844.
https://www.voidsecurity.in/2018/08/from-compiler-optimization-to-code.html

LLVM 7.0.0 Released.
https://lists.llvm.org/pipermail/llvm-announce/2018-September/000080.html

New nmctl Network Manager for OpenBSD.
https://twitter.com/videlft/status/1042518095045251073
http://vincentdelft.be/post/post_20180919

FreeBSD 12.0-RELEASE will use LLVM 7.0.0 compiler.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230240

BSD Now 264 – Optimized-Out.
https://www.jupiterbroadcasting.com/127211/optimized-out-bsd-now-264/

Videos and Slides from 2018 OpenZFS Developer Summit.
http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit_2018

Falling in Love with Rust.
http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/

Almost Forgotten art of procmail with fetchmail .
https://box.matto.nl/fetchmail.html

XigmaNAS x64 11.2.0.4.6026 Released.
https://sourceforge.net/projects/xigmanas/files/XigmaNAS-11.2.0.4/11.2.0.4.6026/

Linux developers threaten to pull β€œkill switch” in response to CoC controversy.
https://lulz.com/linux-devs-threaten-killswitch-coc-controversy-1252/

FreeBSD process title and missing memory space.
https://oshogbo.vexillium.org/blog/54/

Hardware

AMD EPYC β€˜Rome’ 7nm CPU Performance Leaks Out – Scores 12500 Points in Cinebench.
https://wccftech.com/amd-epyc-rome-7nm-64-core-cpu-performance-benchmark-leak/

Inaugural RISC-V Summit Agenda 2018/12/03-06.
https://riscv.org/2018/09/agenda-for-inaugural-risc-v-summit/

Ampere ARM64 32 Core CPU Shipping.
Eight DDR4-2667 memory controllers.
42 lanes PCIe 3.0 for high bandwidth I/O.
125W TDP for maximum power efficiency.
TSMC 16nm FinFET process.
32 cores @ up to 3.3 Ghz Turbo $850.
16 cores @ up to 3.3 Ghz Turbo $550.
https://www.theregister.co.uk/2018/09/18/ampere_shipping/
https://amperecomputing.com/ampere-announces-availability-of-emag-for-hyperscale-cloud-computing-and-unveils-aggressive-multi-generation-roadmap/
https://www.opencompute.org/files/18150J-Ampere-PPT-OCPSummitKumar-final.pdf

Qualcomm Snapdragon 1000 for laptops could pack 8.5 billion transistors with 15W TDP.
https://www.techradar.com/news/qualcomm-snapdragon-1000-for-laptops-could-pack-85-billion-transistors

ARMv8.5-A Architecture Developments 2018.
https://community.arm.com/processors/b/blog/posts/arm-a-profile-architecture-2018-developments-armv85a

Life

Every Weekend Should Be 3-Day Weekend.
https://www.thecut.com/2015/09/every-weekend-should-be-a-3-day-weekend.html

For me this is paradise – life in the Spanish city that banned cars.
https://www.theguardian.com/cities/2018/sep/18/paradise-life-spanish-city-banned-cars-pontevedra

French bookshops revolt after prize selects novel self-published on Amazon.
https://www.theguardian.com/books/2018/sep/15/french-bookshops-revolt-after-prize-selects-novel-self-published-on-amazon

Other

Explore immersive web with Firefox Reality available for Viveport/Oculus/Daydream.
https://blog.mozilla.org/blog/2018/09/18/firefox-reality-now-available/

IDAD – International Day Against DRM.
https://blog.documentfoundation.org/blog/2018/09/18/idad-international-day-against-drm/

Software disenchantment.
http://tonsky.me/blog/disenchantment/

EOF
Advertisements

FreeBSD Desktop – Part 16 – Configuration – Pause Any Application

Many desktop oriented operating systems try to provide various usability improvements and features, like quite useful Expose or Dashboard in Mac OS X or useless Tiles concept in recent editions of Microsoft Windows systems.

pause.key.jpg

After using UNIX for so many years I knew that I could freeze (or pause) any process in the system with kill -17 (SIGSTOP) signal and then unfreeze it with with kill -19 (SIGCONT) signal as I described in the Process Management section of the Ghost in the Shell – Part 2 article. Doing it that way for the desktop applications is PITA to say the least. Can you imagine opening xterm(1) terminal and searching for all Chromium or Firefox processes and then freezing them one by one every time you need it? Me neither.

Fortunately with introduction of so called X11 helper utilities – like xdotool(1) – it is now possible to implement it in more usable manner.

Today I will show you how to freeze any X11 application with single keyboard shortcut or mouse gesture if you utilize them in any way with small simple script.

When such feature can be useful (or what for)?

Lets say you have Firefox started with many tabs open (50+) and you know that it drains battery life from your laptop. You can close it but when You will need information from any of those tabs, then You will have to start Firefox again (even more battery usage) and load all needed tabs (battery …). The alternative is to pause all Firefox processes when You do not use them. This will freeze all its processes and subprocesses and it will not use any CPU (or battery) power. When you will need it, then you will unpause it without the need to load all tabs again.

Other example may be some heavy processing. For example you started RawTherapee or Darktable processing of large amount of photos and you are not able to smoothly watch a video. Just pause it, watch the video and unpause it again to finish its work.

Its also usable in single player gaming when You can REALLY pause the game, literally πŸ™‚

You may also check earlier articles of the FreeBSD Desktop series:

First we need to install the so called X11 helpers. Do that with this pkg(8) command.

# pkg install xprop xdotool zenity xbindkeys

Now for the script that would make all this magic happen. The desktop-pause.sh script is available on GitHub as its syntax is nicely colored there. Save it in some place where its searchable through ${PATH} variable like ~/bin or ~/script directory and make it executable.

% fetch -O ~/scripts/desktop-pause.sh https://raw.githubusercontent.com/vermaden/scripts/master/desktop-pause.sh
% chmod +x ~/scripts/desktop-pause.sh
% echo $PATH | grep scripts
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/vermaden/scripts

It has three ways of usage.

% desktop-pause.sh
usage: desktop-pause.sh OPTION [ARGUMENT]

OPTIONS:
  -a  -  Do pause/resume active window.
  -s  -  Do pause/resume interactively selected window.
  -p  -  Do pause/resume specified PID.
  -l  -  Do list paused processes/windows.
  -L  -  Do list paused processes/windows with PIDs.

ARGUMENT:
  PID for '-p' option.

If started with -a option, then it would pause/unpause the currently active window. This option is best used with keyboard shortcut or mouse gesture. It you start desktop-pause.sh script with -s argument, then the cursor will change and you will be able to select which window to freeze (or unfreeze). The -p option is usable in terminal directly as you may want to freeze/unfreeze a process without X11 environment or for some debugging purposes for example. The last -l option will list applications that are currently paused.

pause.key.thinkpad

Most present-day generation laptops have island type limited keyboards so you will have to choose for yourself which keyboard shortcut to use. As I still use 2011 ThinkPad T420s laptop with 7-row keyboard I have little more options. The [Pause Break] key seems to be the best candidate for such feature πŸ™‚ I will use it for the ‘active window freeze/unfreeze’ with -a option and [SHIFT]-[Pause Break] key for the more interactive -s option.

To create such new keyboard shortcut we will use handy xbindkeys(1) tool.

Lets see what code we will have to put into the ~/.xbindkeysrc configuration file.

% xbindkeys --help
xbindkeys 1.8.6 by Philippe Brochard
usage: xbindkeys [options]
  where options are:
  -V, --version           Print version and exit
  -d, --defaults          Print a default rc file
  -f, --file              Use an alternative rc file
  -p, --poll-rc           Poll the rc/guile configs for updates
  -h, --help              This help!
  -X, --display           Set X display to use
  -v, --verbose           More information on xbindkeys when it run
  -s, --show              Show the actual keybinding
  -k, --key               Identify one key pressed
 -mk, --multikey          Identify multi key pressed
  -g, --geometry          size and position of window open with -k|-mk option
  -n, --nodaemon          don't start as daemon

As its single key we will need --key option. Lets do it then.

% xbindkeys --key
Press combination of keys or/and click under the window.
You can use one of the two lines after "NoCommand"
in $HOME/.xbindkeysrc to bind a key.
"(Scheme function)"
    m:0x0 + c:110
    Pause

Now lets read the [SHIFT]-[Pause Break] sequence.

% xbindkeys --key
Press combination of keys or/and click under the window.
You can use one of the two lines after "NoCommand"
in $HOME/.xbindkeysrc to bind a key.
"(Scheme function)"
    m:0x1 + c:110
    Shift + Pause

We now have all needed information for the ~/.xbindkeysrc configuration file. Here is how it looks configured.

% cat ~/.xbindkeysrc

# [Pause Break] FOR ACTIVE WINDOW
"~/scripts/desktop-pause.sh -a"
  Pause

# [Shift]-[Pause Break] FOR INTERACTIVE WINDOW
"~/scripts/desktop-pause.sh -s"
  Shift + Pause

Now lets start xbindkeys(1) and verify that it works.

% xbindkeys

Press the [Pause Break] key when you are in the terminal where you started xbindkeys(1) utility. Now hit [ENTER] several times, the terminal should be freezed. Now hit [Pause Break] key again. The etnered [ENTER] keys have been passed to it as it was unfreezed.

Lets check the Firefox example.

When processes run like usual they have on of the I*/S*/R* state like shown below.

% ps ax | grep firefox | grep -v grep
67981  -  S       3:28.66 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab
41124  0- S      68:44.94 firefox
43940  0- S      25:52.43 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab

When you will now freeze Firefox with [Pause Break] key its processes will have T state.

% ps ax | grep firefox | grep -v grep
67981  -  T       3:28.66 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab
41124  0- T      68:45.17 firefox
43940  0- T      25:52.85 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab

After you unfreeze them again with [Pause Break] key they will get back to normal I*/S*/R* state.

% ps ax | grep firefox | grep -v grep
67981  -  S       3:28.67 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab
41124  0- S      68:45.54 firefox
43940  0- S      25:53.01 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab

You may of course specify by hand the Firefox PID which is 41124 in current state.

% desktop-pause.sh -p 41124
INFO: kill -17 41124
INFO: kill -17 67981
INFO: kill -17 43940

The Firefox browser will be paused again.

% ps ax | grep firefox | grep -v grep
67981  -  T       3:28.68 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab
41124  0- T      68:46.68 firefox
43940  0- T      25:56.22 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab

Use it again to unpause it.

% desktop-pause.sh -p 41124
INFO: kill -19 41124
INFO: kill -19 67981
INFO: kill -19 43940

And viola! Firefox runs again.

% ps ax | grep firefox | grep -v grep
67981  -  S       3:28.68 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab
41124  0- S      68:46.72 firefox
43940  0- S      25:56.28 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab

There are no downsides to this feature but one has to remember paused applications will not refresh themselves as their processes are freezed. Below you can see frozen Epiphany browser upon which the xterm(1) window was moved. Pretty Windows like effect.

epiphany.paused

After you unpause the Epiphany it gets back to normal as shown below.

epiphany.unpaused.png

Remember to add xbindkeys(1) command to your ~/.xinitrc (or ~/.xsession file) to make it permanent.

UPDATE 1

One of the Hacker News users named rhn_mk1 explained the lack of window contents refresh while application is freezed. I will just cite his comment below.

That depends on the window manager. The application state is not really affected, it just stops updating (redrawing its area). When another window moves away, the window manager asks the “underlying” application to update that area of the screen. It’s dead, so the WM keeps displaying the last thing that was there, until something else happens in that spot.

On the other hand, compositing window managers will dedicate a separate buffer to each application, where they have exclusive access. That kind of a window manager would not have to ask the application to update anything – it would just take the image from the dedicated application’s buffer and update the screen with it. Since the application’s buffer can’t be modified by anything else, it would have the last state of the application in it. That would in turn find its way to the screen. No glitches.

UPDATE 2

One of the Reddit users 89luca89 pointed me to the browser-suspender solution that ‘simply suspends the browser when not in focus using STOP/CONT’ signals.

UPDATE 3

The Lobsters user seschwar pointed out that there is Stoppable Layout functionality for XMonad which automatically pauses the processes of all windows except for the active one and it also uses SIGCONT and SIGSTOP signals.

UPDATE 4

One of the Hacker News users named imglorp suggested that my “command could also iconify/minify the app’s windows”.

This is really good idea.

I just added -A and -S options that also minimize a window.

% desktop-pause.sh 
usage: desktop-pause.sh OPTION [ARGUMENT]

OPTIONS:
  -a  -  Do pause/resume active window.
  -A  -  Do pause/resume active window and minimize it.
  -s  -  Do pause/resume interactively selected window.
  -S  -  Do pause/resume interactively selected window and minimize it.
  -p  -  Do pause/resume specified PID.
  -l  -  Do list paused processes/windows.
  -L  -  Do list paused processes/windows with PIDs.

ARGUMENT:
  PID for '-p' option.

Here is the changelog for the desktop-pause.sh script:
https://github.com/vermaden/scripts/commit/03591a138b14cededa15a05fe9c77bf1a941795d

EOF

Ghost in the Shell – Part 3

Time to bring some life into the Ghost in the Shell series with Part 3 article.

You may want to check earlier articles in the series.

Query Functions

I haven’t found better name for that solution. There are generally two types of UNIX people. These that prefer to navigate and operate with basic ls/cd/mv/mkdir/rm commands and those who use some file manager like Midnight Commander (mc) or ranger or vifm or … you get the idea. I have tried various CLI file managers but always came back to navigate without them. If you are one of those people then these Query Functions are for you πŸ™‚

The so called Query Functions are for filter the information you look for. For example if you have directory with large number of files, then you would probably do something like that.

% ls | grep QUERY

… or if you also want to include subdirectories then something like that.

% find . | grep QUERY

For both of these examples you would also probably want to sometimes search case sensitive or insensitive depending on the need.

That leads us to four Query Functions:

  • q is an equivalent of ls | grep -i QUERY command.
  • Q is an equivalent of ls | grep QUERY command.
  • qq is an equivalent of find . | grep -i QUERY command.
  • QQ is an equivalent of find . | grep QUERY command.

Thus if I need to query the contents of directory while searching for something is very fast with q SOMETHING.

These are definitions of these Query Functions:

# SHORT QUERY FUNCTIONS q()
  q() {
    if [ ${#} -eq 1 ]
    then
      ls | grep --color -i ${1} 2> /dev/null
    else
      echo "usage: q string"
    fi
  }
     
# SHORT QUERY FUNCTIONS Q()
  Q() {
    if [ ${#} -eq 1 ]
    then
      ls | grep --color ${1} 2> /dev/null
    else
      echo "usage: Q string"
    fi
  }

# SHORT QUERY FUNCTIONS qq()
  qq() {
    if [ ${#} -eq 1 ]
    then
      find . \
        | grep -i ${1} 2> /dev/null \
        | cut -c 3-999 \
        | grep --color -i ${1} 2> /dev/null
    else
      echo "usage: qq string"
    fi
  }

# SHORT QUERY FUNCTIONS QQ()
  QQ() {
    if [ ${#} -eq 1 ]
    then
      find . \
        | grep ${1} 2> /dev/null \
        | cut -c 3-999 \
        | grep ${1} 2> /dev/null
    else
      echo "usage: QQ string"
    fi
  }

The qq and QQ functions uses grep(1) two times to make sure the output is colored.

I assume that You use colored grep(1) described in Ghost in the Shell – Part 2 article.

If you prefer to use alias(1) instead then they would look like that.

# SHORT QUERY FUNCTIONS q() Q() qq() QQ()
  alias q="ls | grep --color -i"
  alias Q="ls | grep --color"
  alias qq="find . | grep -i"
  alias QQ="find . | grep"

The qq and QQ will be little more limited as with functions its possible to trim the output to the exact needs with cut(1).

q.png

Lots of people use recursive history search which also helps, but what if you used/typed needed command long ago with the arguments you need now? You would probably search the command with history(1) command and then using grep(1) to limit the results to what you look for. I keep enormous large list of commands to keep in history – with my current setting of 655360 the ~/.zhistory (ZSH) file takes about 2.7 MB size. I also wanted to be sure that two identical commands would not be kept in history hence the setopt hist_ignore_all_dups ZSH option enabled. When I wc -l my ~/.zhistory file it currently has 75695 lines of commands.

% grep HISTSIZE /usr/local/etc/zshrc
export HISTSIZE=655360
export SAVEHIST=${HISTSIZE}

% grep dups /usr/local/etc/zshrc
setopt hist_ignore_all_dups

Now back to Query Functions for history:

  • h is an equivalent of cat ~/.zhistory | grep -i QUERY command.
  • H is an equivalent of cat ~/.zhistory | grep QUERY command.

They fit in aliases this time. In alias(1) we will use just grep(1) to not ‘do’ Useless Use of Cat.

Here are the Query Functions for history.

# SHORT HISTORY ALIASES h() H()
  alias h='< ~/.zhistory grep -i'
  alias H='< ~/.zhistory grep'

h

… but what if we would like to filter the outputs of q family and h family Query Functions? The obvious response is using grep(1) like q QUERY | grep ANOTHER or h QUERY | grep ANOTHER for example. To make that faster we will make g and G shortcuts.

  • g is an equivalent of grep -i command.
  • G is an equivalent of just grep command.

Here they are.

# SHORT GREP FUNCTIONS g() G()
  alias g='grep -i'
  alias G='grep'

Now it will be just q QUERY | g ANOTHER and h QUERY | G ANOTHER for example.

To clear terminal output you may use clear(1) command, some prefer [CTRL]-[L] shortcut but I find ‘c‘ alias to be the fastest solution.

# SHORT GREP FUNCTIONS c()
  alias c='clear'

To make the solution complete I would also add exa(1) here with an alias of ‘e‘.

# SHORT LISTING WITH e()
  alias e='exa --time-style=long-iso --group-directories-first'

Why exa(1) will you ask while there is BSD ls(1) and GNU ls(1) (installed as gls(1) on FreeBSD to not confuse). To add GNU ls(1) to FreeBSD system use the coreutils package.

Well, the BSD ls(1) has two major cons:

  • It is not able to sort directories first.
  • It selects width for ALL columns based on single longest file name.

BSD-ls.png

The BSD ls(1) was used as following alias:

alias ls='ls -p -G -D "%Y.%m.%d %H:%M"'

The GNU ls(1) does not have these two problems but it does color the output only on the very limited pattern like:

  • Not executable file.
  • Executable file.
  • Directory.
  • Link.
  • Device.

GNU-ls.png

The GNU ls(1) was used as following alias:

gls -p -G --color --time-style=long-iso --group-directories-first --quoting-style=literal

Here is where exa(1) comes handy as it does not have any cons like FreeBSD’s ls(1) and it colors a lot more types of files.

e.png

exa --time-style=long-iso --group-directories-first

Its still very simple coloring based on file extension and not magic number as plain (empty) text file SOME-NOT-FILE.pdf is colored like PDF document. But even this ‘limited’ coloring helps in 99% of the cases and while with BSD ls(1) and GNU ls(1) all of these files ‘seem’ like plain text files with exa(1) its obvious from the start which are plain files, which are images and which are ‘documents’ like PDF files for example.

Where Is My Space

On all UNIX and Linux systems there exists du(1) command. Combined with sort(1) it is universal way of searching for space eaters. Example for the / root directory with -g flag to display units in gigabytes.

# cd /
# du -sg * | sort -n
1       bin
1       boot
1       compat
1       COPYRIGHT
1       data
1       dev
1       entropy
1       etc
1       lib
1       libexec
1       media
1       mnt
1       net
1       proc
1       rescue
1       root
1       sbin
1       sys
1       tmp
1       var
2       jail
8       usr
305     home

Contents of UNIX System Resources directory with -m flag to display unit in megabytes.

# cd /usr
# du -sm * | sort -n
1       libdata
1       obj
1       tests
3       libexec
11      sbin
13      include
45      lib32
56      lib
58      share
105     bin
1080    ports
1343    src
5274    local

But its PITA to type cd and du all the time, not to mention that some oldschool UNIX systems does not provide -g or -m flags so on HP-UX you are limited to kilobytes at most.

You may also try -h (human readable) with sort -h (sort human readable) du(1) variant.

# du -smh * | sort -h
512B    data
512B    net
512B    proc
512B    sys
4.5K    COPYRIGHT
4.5K    entropy
5.5K    dev
6.5K    mnt
 53K    media
143K    tmp
205K    libexec
924K    bin
2.2M    etc
3.9M    root
4.6M    sbin
6.2M    rescue
6.6M    lib
 90M    boot
117M    compat
564M    jail
667M    var
5.4G    usr
297G    home

This is where ncdu(1) comes handy. Its ncurses based disk usage analyzer which helps finding that space eaters in very fast time without typing the same commands over and over again. Here is ncdu(1) in action.

First it calculates the sizes of the files.

ncdu.png

After a while you get the output sorted by size.

ncdu-usr.png

If you hit [ENTER] on the directory you will be instantly moved into that directory.

ncdu-usr-local.png

If you delete something with ‘d‘ then remember to recalculate the output with ‘r‘ letter.

It also has great options such as spawning shell ‘b‘ in the current directory or toggle between apparent size and disk usage with ‘a‘ option. The latter is very useful when you use filesystem with builtin compression like ZFS.

       up, k  Move cursor up
     down, j  Move cursor down
 right/enter  Open selected directory
  left, <, h  Open parent directory
           n  Sort by name (ascending/descending)
           s  Sort by size (ascending/descending)
           C  Sort by items (ascending/descending)
           d  Delete selected file or directory
           t  Toggle dirs before files when sorting
           g  Show percentage and/or graph
           a  Toggle between apparent size and disk usage
           c  Toggle display of child item counts
           e  Show/hide hidden or excluded files
           i  Show information about selected item
           r  Recalculate the current directory
           b  Spawn shell in current directory
           q  Quit ncdu

The apparent size using the du(1) command.

Disk usage.

% du -sm books
39145   books

Apparent size.

% du -smA books
44438   books

So I have 1.13 compression ratio on the ZFS filesystem. More then 5 GB saved just in that directory πŸ™‚

Where Are My Files

Once I got some space back I also wanted to know if there are some directories with enormous amount of very small files.

First I came up with my own files-count.sh script solution which is not that long.

#! /bin/sh

export LC_ALL=C

if [ ${#} -eq 0 ]
then
  DIR=.
else
  DIR="${1}"
fi

find "${DIR}" -type d -maxdepth 1 -mindepth 1 \
  | cut -c 3- \
  | while read I
    do
      find "${I}" | wc -l | tr -d '\n'
      echo " ${I}"
    done | sort -n

It works reliably but same as with du | sort tandem you have to retype it (or at least use cd(1) and hit [UP] arrow again) … but then I discovered that ncdu(1) also counts files! It does not provide ‘startup’ argument to start in this count files mode but when you hit ‘c‘ letter it will instantly display count of files in each scanned directory. To sort this output by the count of files hit the ‘C‘ letter (large ‘C‘ letter).

ncdu-files.png

The files-count.sh script still has one advantage over ncdu(1) – the latter stops counting files at 100k which is shown on the screenshot so if You need to search for really big amount of files or just about 100k then files-count.sh script will be more accurate/adequate.

% cd /usr
% files-count.sh 
       1 obj
      36 libdata
     299 sbin
     312 libexec
     390 tests
     498 bin
     723 lib32
     855 lib
    2127 include
   16936 share
  159945 src
  211854 ports
  266021 local

… but what if there were some very big files hidden somewhere deep in the directories tree? The du(1) or ncdu(1) will not help here. As usual I though about short files-big.sh script that will do the job.

#! /bin/sh

export LC_ALL=C

if [ ${#} -eq 0 ]
then
  DIR=.
else
  DIR="${1}"
fi

find "${DIR}" -type f -exec stat -f "%16z; doas rm -f \"%N\"" {} ';' | sort -n

An example usage on the /var directory.

# cd /var
# files-big.sh | tail
        10547304; doas rm -f "./tmp/kdecache-vermaden/icon-cache.kcache"
        29089823; doas rm -f "./db/clamav/clamav-2671b72fce703c2133c61e5bf85aad19.tmp/clamav-373e311ca7f610a39c7cf5c5c5a4fd83.tmp/daily.hdb"
        30138884; doas rm -f "./tmp/pkg-provides-wyK2"
        48271360; doas rm -f "./db/pkg/repo-HardenedBSD.sqlite"
        54816768; doas rm -f "./db/pkg/repo-FreeBSD.sqlite"
        66433024; doas rm -f "./db/pkg/local.sqlite"
        82313216; doas rm -f "./db/clamav/clamav-2671b72fce703c2133c61e5bf85aad19.tmp/clamav-373e311ca7f610a39c7cf5c5c5a4fd83.tmp/daily.hsb"
       117892267; doas rm -f "./db/clamav/main.cvd"
       132431872; doas rm -f "./db/clamav/daily.cld"
       614839082; doas rm -f "./db/pkg/provides/provides.db"

The output is in ‘executable’ format so if you select whole line and paste it into terminal, then this file will be deleted. By default it uses doas(1) but nothing can stop you from putting sudo(8) there. Not sure if you will find it useful but it helped me at least dozen times.

How Many Copies Do You Keep

I often find myself keeping the same files in several places which also wastes space (unless you use ZFS deduplication of course).

The dedup.sh script I once made is little larger so I will not paste it here and just put a link to it.

It has the following options available. You may search/compare files by name or size (fast) or by its MD5 checksum (slow).

% dedup.sh
usage: dedup.sh OPTION DIRECTORY
  OPTIONS: -n   check by name (fast)
           -s   check by size (medium)
           -m   check by md5  (slow)
           -N   same as '-n' but with delete instructions printed
           -S   same as '-s' but with delete instructions printed
           -M   same as '-m' but with delete instructions printed
  EXAMPLE: dedup.sh -s /mnt

Simple usage example.

% cd misc/man
% cp zfs-notes zfs-todo
% dedup.sh -M .
count: 2 | md5: 4ff4be66ab7e5484de2bf7c168ff995a
  doas rm -rf "./zfs-notes"
  doas rm -rf "./zfs-todo"

count: 2 | md5: 6d87f5b1317ea189165fcdc71380735c
  doas rm -rf "./x11"
  doas rm -rf "./xinit"

By copying the zfs-notes file into the zfs-todo file I wanted to show you what dedup.sh will print on the screen, but accidentally I also found another duplicate πŸ™‚

The output of dedup.sh is simple and like with files-big.sh script selecting the while line and pasting it into the terminal will remove the duplicate. By default it uses doas(1) but you can change it into sudo(8) if that works better for you.

Unusual cron(1) Intervals

Most of us already remember what the five fields of crontab(5) file mean, but what if you would like to run command every second … or after reboot only? The answer lies in the man 5 crontab page. Here are these exotic options.

string          meaning
------          -------
@reboot         Run once, at startup of cron.
@yearly         Run once a year, "0 0 1 1 *".
@annually       (same as @yearly)
@monthly        Run once a month, "0 0 1 * *".
@weekly         Run once a week, "0 0 * * 0".
@daily          Run once a day, "0 0 * * *".
@midnight       (same as @daily)
@hourly         Run once an hour, "0 * * * *".
@every_minute   Run once a minute, "*/1 * * * *".
@every_second   Run once a second.

Check cron(1) Environment

Many times I found myself lost lots of time debugging what went wrong when my script was run by the crontab(5) file. Often it was some variable missing or some command or script I used was not in the PATH variable.

To make that debugging faster You can use ENV.sh script to just store the cron(1) environment.

% cat ENV.sh
env > /tmp/ENV.out

The ENV.sh script will write current environment in the /tmp/ENV.out file.

Lets put it into the crontab(5) for a test.

% crontab -l | grep ENV
@every_second ~/ENV.sh

Now after at most a second you can check for the contents of the /tmp/ENV.out file.

% cat /tmp/ENV.out
LOGNAME=vermaden
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
PWD=/home/vermaden
HOME=/home/vermaden
USER=vermaden
SHELL=/bin/sh

Now you can easily debug the scripts run by the crontab(5) … at least on the environment part πŸ™‚

Simple HTTP Server

I found myself many times in a situation that I would want to allow download of some files from my machine and SSH could not be used.

This is when python(1) comes handy. It has SimpleHTTPServer (or http.server in Python 3 version) so you can instantly start HTTP server in any directory!

Here are the commands for both Python versions.

  • Python 2.x – python -m SimpleHTTPServer PORT
  • Python 3.x – python -m http.server PORT

I even made a simple http.sh wrapper script to make it even more easy.

#! /bin/sh

if ${#} -ne 1 ]
then
  echo "usage: ${0##*/} PORT"
  exit 1
fi

python -m SimpleHTTPServer ${1}

Example usage.

% cd misc/man
% http.sh 8080
Serving HTTP on 0.0.0.0 port 8080 ...
127.0.0.1 - - [14/Sep/2018 23:06:50] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [14/Sep/2018 23:06:50] code 404, message File not found
127.0.0.1 - - [14/Sep/2018 23:06:50] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [14/Sep/2018 23:09:15] "GET /bhyve HTTP/1.1" 200 -

To stop it simply hit [CTRL]-[C] interrupt sequence.

Here is how it looks in the Epiphany browser.

http.png

Simple FTP Server

Similarly with FTP service, another Python goodie called pyftpdlib (Python FTP Server Library) provides that.

Mine ftp.py wrapper is little bigger as you can write quite comlicated setups with pyftpdlib but mine is simple, it starts in the current directory and adds read only anonymous user and read/write user named writer with WRITER password.

#! /usr/bin/env python

from sys                   import argv,exit
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers    import FTPHandler
from pyftpdlib.servers     import FTPServer

if len(argv) != 2:
  print "usage:", argv[0], "PORT"
  print
  exit(1)
  
authorizer = DummyAuthorizer()
authorizer.add_user("writer", "WRITER", ".", perm="elradfmw")
authorizer.add_anonymous(".")
handler = FTPHandler
handler.authorizer = authorizer
handler.passive_ports = range(60000, 60001)
address = ("0.0.0.0", argv[1])
ftpd = FTPServer(address, handler)
ftpd.serve_forever()

The ftp.py is handy if you want to enable someone to upload something for you (or you are doing it o the other machine) when SSH/SCP is not possible for some reason.

To stop it simply hit [CTRL]-[C] interrupt sequence.

Here is its terminal startup and logs.

% cd misc/man
% ftp.py 2121
[I 2018-09-14 23:21:53] }}} starting FTP server on 0.0.0.0:2121, pid=64399 {{{
[I 2018-09-14 23:21:53] concurrency model: async
[I 2018-09-14 23:21:53] masquerade (NAT) address: None
[I 2018-09-14 23:21:53] passive ports: 60000->60000

… and how Firefox renders its contents.

ftp.png

Hope you will find some of these useful, see you at Part 4 some day.

EOF

Β 

Valuable News – 2018/09/15

The Valuable News weekly series is dedicated to provide summary about news, articles and other interesting stuff mostly but not always related to the UNIX or BSD systems. Whenever I stumble upon something worth mentioning on the Internet I just put it here so someone else can

Today the amount information that we get using various information streams is at massive overload. Thus one needs to focus only on what is important without the need to grep(1) the Internet everyday. Hence the idea of providing such information ‘bulk’ as I already do that grep(1).

UNIX

In Other BSDs for 2018/09/08.
https://www.dragonflydigest.com/2018/09/08/21755.html

Including optimized out kernel symbols in dtrace on FreeBSD.
https://farhan.codes/2018/08/16/including-optimized-out-kernel-symbols-in-dtrace-on-freebsd/

Lazy Reading for 2018/09/09.
https://www.dragonflydigest.com/2018/09/09/21750.html

NIH Package manager for pkgsrc.
http://pkgsrc.se/pkgtools/nih

OpenBSD on the Desktop: some thoughts.
https://blog.gsora.xyz/openbsd-on-the-desktop-some-thoughts/

Ansible modules for OpenBSD vmm(4).
https://github.com/jasperla/ansible-vmm

OpenBSD vmm(4) got qcow2 image support.
https://marc.info/?l=openbsd-cvs&m=153646618113084&w=2

FreeBSD bootonly ISO failures PXE booting under UEFI.
FreeBSD Bootonly ISO failures PXE booting under UEFI

Dosbox-X is fork of DOSBox wth goal of accurate emulation for Windows 95/98/ME systems.
http://dosbox-x.com/

vm-bhyve with NAT on FreeBSD.
https://www.foxk.it/blog/2018-09-08_vm-bhyve/

FreeNAS 11.2-BETA3 Available.
https://www.ixsystems.com/blog/library/freenas-11-2-beta3/

Nextcloud in FreeBSD Ports updated to 14.0.
https://svnweb.freebsd.org/ports?view=revision&revision=479446

New FreeNAS 11.2 Beta 3 Interface.
https://twitter.com/TomLawrenceTech/status/1039623935170682880

FreeBSD PKGng plugin for querying which package provides a particular file.
https://github.com/rosorio/pkg-provides

FreeBSD SA 18:12.elf.
https://www.freebsd.org/security/advisories/FreeBSD-SA-18:12.elf.asc

FreeBSD EN 18:08.lazyfpu.
https://www.freebsd.org/security/advisories/FreeBSD-EN-18:08.lazyfpu.asc

FreeBSD 12.x suspend/resume works on ThinkPad X1 Carbon 6th Generation after BIOS 1.30 update and sleep state set to ‘Linux’.
https://twitter.com/lwhsu/status/1039711710913945601

FreeBSD 12.x amd64 has NUMA enabled for GENERIC/MINIMAL kernels.
https://lists.freebsd.org/pipermail/freebsd-current/2018-September/071168.html
https://svnweb.freebsd.org/base?view=revision&revision=338602

My FreeBSD packages are out of date, how do I fix that?
https://vincerants.com/my+freebsd+packages+are+out+of+date%2c+how+do+i+fix+that%3f

FreeBSD 12.0-ALPHA4 on Pinebook.
https://twitter.com/manuvadot/status/1039943151383977985

BSD Now 263 – Encrypt That Pool.
https://www.jupiterbroadcasting.com/127081/encrypt-that-pool-bsd-now-263/

RFC8446 compliant OpenSSL 1.1.1 landed in FreeBSD 12.0.
https://twitter.com/mzar_y/status/1040502530072096768

Today in History. Brought to You by UNIX.
https://akr.am/blog/posts/today-in-history-brought-to-you-by-unix

Remote Code Execution in Alpine Linux.
https://justi.cz/security/2018/09/13/alpine-apk-rce.html

There are 90% Linux and 9% BSD Systems in the TOR RELAYS.
https://www.torbsd.org/oostats/relays-bw-by-os.html

HardenedBSD 1100056.5 Available.
https://hardenedbsd.org/article/op/2018-09-14/stable-release-hardenedbsd-stable-11-stable-v11000565

Wine Development Release 3.16 Available.
https://www.winehq.org/announce/3.16

FreeBSD 12.0-ALPHA6 Available.
http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/12.0/

Hardware

Another Intel ME Vulnerability.
https://twitter.com/h0t_max/status/1039597102693851136
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00125.html

Intel Power Management Controller (PMC) Security Advisory.
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00131.html

Seagate BarraCuda Pro 14TB HDD Review – Massive Storage for Desktops.
https://www.anandtech.com/show/13340/seagate-barracuda-pro-14tb-hdd-review/

The rv8 is 1.9.1 privileged specification RISC-V emulator and x86-64 binary translator.
https://rv8.io/
https://github.com/rv8-io/rv8

Custom 51nb made ThinkPad X210 with Kaby Lake QUAD-CORE i5-8250, 1920×1200 IPS screen and 16GB RAM.
https://twitter.com/jcs/status/1039624550168842241
https://www.51nb.com/mb/
x210.jpg

AMD EPYC 32 Core 7551P Benchmarks and Review.
https://twitter.com/ServeTheHome/status/1039656584496873473
https://www.servethehome.com/amd-epyc-7551p-benchmarks-and-review-a-32-core-value-monster/

How Intel Xeon Platinum is Marketing Headache versus AMD EPYC Prices.
https://www.servethehome.com/how-the-intel-xeon-platinum-is-a-marketing-headache/

Lenovo latest Yoga laptop with new ARM CPU and 25 hour battery life.
https://www.theverge.com/circuitbreaker/2018/8/30/17799382/lenovo-yoga-c630-wos-features-specs-price-release-date-battery-life-ifa-2018

34C3: Hacking into a CPU Microcode.
34C3: Hacking into a CPU’s Microcode

Analysis of Data Corruption in the Storage Stack.
This is why checksumming and healing of data in ZFS is that much important.
https://www.usenix.org/legacy/events/fast08/tech/full_papers/bairavasundaram/bairavasundaram_html/index.html

The x86 finds its way into your iPhone.
https://lcq2.github.io/x86_iphone/

Life

So We Got Tracked Anyway.
https://youbroketheinternet.org/trackedanyway

Real Cost of the 2008 Financial Crisis.
https://www.newyorker.com/magazine/2018/09/17/the-real-cost-of-the-2008-financial-crisisz

Other

Firefox 62.0 Released.
https://www.mozilla.org/en-US/firefox/62.0/releasenotes/

What is First-Party Isolation in Firefox and what breaks if you enabled it.
https://www.ctrl.blog/entry/firefox-fpi

Firefox is now built with clang LTO on all platforms.
https://glandium.org/blog/?p=3888

Good books for deep hacks.
https://begriffs.com/posts/2017-04-13-longterm-computing-reading.html

EOF

Valuable News – 2018/09/08

UNIX

SoloBSD 11.2-STABLE-0901 based on HardenedBSD 1100056.4.
https://www.solobsd.org/index.php/2018/09/01/solobsd-11-2-stable-0901/

FreeBSD Foundation Update 2018/08.
https://www.freebsdfoundation.org/news-and-events/newsletter/freebsd-foundation-update-august-2018/

FreeBSD on System76 laptop.
https://twitter.com/BNevio/status/1036067328553877505

FreeBSD and Intel Video Cards. (updated)
http://srobb.net/freebsdintel.html

NetBSD/atari 8.0 on ATARI TT030 with SMC_TT and EtherNEC.
https://twitter.com/tsutsuii/status/1036602442790526978
http://dmesgd.nycbug.org/index.cgi?do=view&id=3733

NetBSD 7.2 Released.
https://www.netbsd.org/releases/formal-7/NetBSD-7.2.html

BSD Now 261 – FreeBSDcon Flashback.
https://www.jupiterbroadcasting.com/126916/freebsdcon-flashback-bsd-now-261/

BSD Now 262 – OpenBSD Surfacing.
https://www.jupiterbroadcasting.com/127006/openbsd-surfacing-bsd-now-262/

DragonFly BSD Default Compiler Changed to GCC 8.x.
https://www.dragonflydigest.com/2018/09/05/21790.html
http://lists.dragonflybsd.org/pipermail/commits/2018-September/717867.html

FreeBSD Ports/Packages Switch Default PHP Version from 5.6 to 7.1.
https://svnweb.freebsd.org/ports?view=revision&revision=479014

BSD Virtual Memory.
http://blog.pr4tt.com/2016/02/02/BSD-virtual-memory/

OmniOS CE r151026r/r151024ar/r151022bp with updated CURL and ZSH.
https://omniosce.org/article/releases-026r-024ar-022bp

ZSH Introduction Configuration.
https://docu.blackdot.be/configuration/zsh

CBSD Added Live Migration Support to FreeBSD Bhyve.
https://twitter.com/meka_floss/status/1037664920823885830
https://www.youtube.com/watch?v=-IYNSBhtJqw

The macOS Mojave Patcher Tool for Unsupported Macs.
http://dosdude1.com/mojave/

On FreeBSD 12.0 Bhyve will use Guard Pages to protect VMs address space.
https://twitter.com/lattera/status/1037801460719869952
https://svnweb.freebsd.org/base?view=revision&revision=338511

Sixth annual OpenZFS Developer Summit 2018 will be held September 10-11 in San Francisco.
http://open-zfs.org/wiki/OpenZFS_Developer_Summit_2018

XigmaNAS 11.2.0.4.5969 Released.
https://sourceforge.net/projects/xigmanas/files/XigmaNAS-11.2.0.4/11.2.0.4.5969/

OPNsense 18.7.2 Released.
https://opnsense.org/opnsense-18-7-2-released/?utm_source=discoverbsd

Microsoft Hotmail still runs on U**x – FreeBSD purge incomplete. (old but worth mentioning)
https://www.theregister.co.uk/2001/12/12/microsoft_hotmail_still_runs/

GNOME 3.30 ‘AlmerΓ­a’ Released.
https://help.gnome.org/misc/release-notes/3.30/

Notes on Ryzen and Threadripper Hardware on DragonFly BSD.
https://www.dragonflybsd.org/docs/handbook/docs/handbook/docs/newhandbook/ZenRipper/

FreeBSD 12.0-ALPHA5 Images Available.
http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/12.0/

Hardware

Intel can’t supply 14nm Xeons – HP/HPE directly recommends AMD EPYC.
https://www.semiaccurate.com/2018/09/07/intel-cant-supply-14nm-xeons-hpe-directly-recommends-amd-epyc/

AMD EPYC pummels Intel new Xeon-W workstation CPUs.
https://www.semiaccurate.com/2017/08/29/amds-epyc-pummels-intels-new-xeon-w-workstation-cpus/

ThinkPad X220 Resources.
http://x220.mcdonnelltech.com/resources/

Ryzen is Great at Low Power – 2700X @ 85W and TR2990WX @ 225W.
http://lists.dragonflybsd.org/pipermail/users/2018-September/357883.html

GlobalFoundries CTO on Why the Company Abandoned the Bleeding Edge.
https://spectrum.ieee.org/nanoclast/semiconductors/nanotechnology/globalfoundries-cto-on-why-the-company-abandoned-the-bleeding-edge

AMD Updates its 25×20 Goal – Progress in a Generation.
https://www.anandtech.com/show/13326/amd-updates-its-25×20-goal-progress-in-a-generation

Toshiba XG6 1TB SSD Review – First 96-Layer 3D NAND SSD.
https://www.anandtech.com/show/13254/the-toshiba-xg6-1tb-ssd-review-first-96l-3d-nand

How RISC-V is Changing the Game.
http://info.dovermicrosystems.com/doverlog/how-risc-v-is-changing-processor-design

The AMD Athlon 200GE is Genius.
https://www.fanlesstech.com/2018/09/the-amd-athlon-200ge-is-genius.html

Life

How to Fall Asleep in 120 Seconds.
https://medium.com/s/story/combat-tested-training-unwind-and-sleep-anywhere-in-120-seconds-27d5307b7606

Meal frequency and timing in health and disease.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4250148/

Banks and Retailers Are Tracking How You Type, Swipe and Tap.
https://www.nytimes.com/2018/08/13/business/behavioral-biometrics-banks-security.html

Paul Graham Essays.
http://www.paulgraham.com/articles.html

Hospitals Prepare To Launch Their Own Drug Company To Fight High Prices And Shortages.
https://www.npr.org/sections/health-shots/2018/09/06/644935958/hospitals-prepare-to-launch-their-own-drug-company-to-fight-high-prices-and-shor

‘Mindful People’ feel less pain – MRI imaging pinpoints supporting brain activity.
https://medicalxpress.com/news/2018-09-mindful-people-pain-mri-imaging.html

Other

40 million tables in MySQL 8.0 with ZFS.
https://www.percona.com/blog/2018/09/03/40-million-tables-in-mysql-8-0-with-zfs/

Nextcloud 14: focus on security and compliance.
https://nextcloud.com/blog/nextcloud-14-focus-on-security-and-compliance/

End of Mozilla Firefox Windows XP Support.
https://chuttenblog.wordpress.com/2018/09/05/the-end-of-firefox-windows-xp-support/

Why Aren’t More Users More Happy With Our VMs?
http://tratt.net/laurie/blog/entries/why_arent_more_users_more_happy_with_our_vms_part_1.html

The Tools I Use To Write Books.
https://thorstenball.com/blog/2018/09/04/the-tools-i-use-to-write-books/

Mozilla Firefox – It’s your data we’re just living in it.
https://blog.mozilla.org/futurereleases/2017/09/06/data-just-living/

Mozilla’s opt-out Firefox DNS privacy test sparks privacy outcry.
https://www.theregister.co.uk/2018/03/20/mozilla_firefox_test_of_privacy_mechanism_prompts_privacy_worries/

Β 

EOF

My FreeBSD Story

As Roman Zolotarev asked if I would write an entry for his Tell Your BSD Story page I could not refuse. I really tried to make it short and small but I guess its not that straight πŸ™‚

My first devices/computers/consoles (not at the same time) that I remember were Atari 2600 and Pegasus console which was hardware clone of the Nintendo NES.

atari-2600.png

Back then I did not even knew that it was Atari 2600 as I referred to it as Video Computer System … and I did not even knew any english by then. It took me about two decades to get to know (by accident) that this Video Computer System was Atari 2600 πŸ™‚

This equipment was used for playing computer games only.

Then I got AMIGA 600 computer (or should I say my parents bought it for me) which served both for playing computer games and also other activities for the first time. AMIGA is the computer that had the greatest influence on me, as it was the first time I studied the books about Amiga Workbench operating system and learned commands from Amiga Shell terminal. I loved the idea of Ram Disk icon/directory on the desktop that allowed me to transparently put any things in system memory. I still miss that concept on today’s desktop systems … and I still remember how dismal I was when I watched Amiga Deathbed Vigil movie.

amiga-600.png

At the end of 1998 I got my first PC that of course came with Windows and that computer served both as gaming machine and as well as typical tool. One time I dig into the internals with Windows Registry (which left me disgusted by its concepts and implementation) and its limited command line interface provided by CMD.EXE executable. I remember that the heart of this box was not the CPU or the motherboard but the graphics accelerator – the legendary 3Dfx Voodoo card. This company (3Dfx) – their attitude and philosophy – also left solid fingerprint on my way. Like AMIGA did.

Hence how the top of my laptop looks like now πŸ™‚

laptop.jpg

Some games was even released as special edition with the only feature being support for the 3Dfx Glide driver like Need for Speed II: Special Edition.

nfs.jpg

After ‘migration’ from AMIGA to PC it never again ‘felt right’. The games were cool but the Windows system was horrible. Time has passed and different Windows versions and hardware modifications took place. Windows XP felt really heavy at that time, not to mention Windows 2000 for example with even bigger hardware requirements. I also do not understand all the hate about Windows ME. It crashed with the same frequency as Windows 98 or later Windows 98 Second Edition but maybe my hardware was different πŸ™‚

windowsme.png

I do not have any ‘mine’ screenshots from that period as I lost all my 40 GB (huge then) drive of data when I moved/resized the partition with Partition Magic to get some more space from the less filled C: drive. That day I learned hard that “there are people who do backups and people who will do backups”. I never lost data again as I had multiple copies of my data, but the same as Netheril fall the lost data was was gone forever.

I always followed various alternatives which led me to try Linux in 2003, after reading about various distributions philosophies I decided to run Slackware Linux with KDE 3. My buddy used Aurox Linux by then (one of the few Linux distributions from Poland) and encouraged me to do the same – especially in the context of fixing possible problems as he already knew it and also as he recently dumped Windows system. But Slackware sounded like a better idea so I took that path instead. At first I dual booted between Windows XP and Slackware Linux cause I had everything worked out on the Windows world while I often felt helpless in the Linux world, so I would reboot into Windows to play some games or find a solution for Linux problem if that was required. I remember how strange the concept of dual clipboards (PRIMARY and SECONDARY) was for me by then. I was amazed why ‘so much better’ system as Linux (at least marketed that way) needs a system tray program to literally manage the clipboard. On Windows it was obvious, you do [CTRL]+[C] to copy and [CTRL]+[V] to paste things, but on Linux there (no I know its X11 feature) there were two clipboards that were synchronized by this little system tray program from KDE 3. It was also unthinkable for me that I will ‘lost’ contents of last/recent [CTRL]+[C] operation if I close the application from which the copy was made. I settled down a little on Slackware but not for long. I really did not liked manual dependency management for packages for example. Also KDE 3 was really ugly and despite trying all possible options I was not able to tweak it into something nice looking.

After half a year on Slackware I checked the Linux distributions again and decided to try Gentoo Linux. I definitely agree with the image below which visualizes Gentoo Linux experience, especially when You install it for he first time πŸ™‚

gentoo-fly

Of course I went with the most hardcore version with self building Stage 1 (compiler and toolchain) which was horrible idea at that time because compilation on slow single core machine took forever … but after many hours I got Gentoo installed. I now have to decide which desktop environment to use. I have read a lot of good news about Fluxbox at that time so this is what I tried. It was very weird experience (to create everything in GUI from scratch) but very pleasant one. That recalled me the times of AMIGA … but Linux came in the way too much often. The more I dig into Gentoo Linux the more I read that lots of Gentoo features are based on FreeBSD solutions. Gentoo Portage is a clone of FreeBSD Ports. That ‘central’ /etc/rc.conf system configuration file concept was taken from FreeBSD as well. So I started to gather information about FreeBSD. The (then) FreeBSD website or FreeBSD Ports site (still) felt little outdated to say the least but that did not discouraged me.

Somewhere in 2005 I installed FreeBSD 5.4 on my computer. The beginnings were hard, like the earlier step with Gentoo but similarly like Gentoo the FreeBSD project came with a lot of great documentation. While Gentoo documentation is concentrated within various Gentoo Wiki sites the FreeBSD project comes with ‘official’ documentation in the form of Handbook and FAQ. I remember my first questions at the now nonexistent BSDForums.org site – for example one of the first ones – how to scroll the terminal output in the plain console. I now know that I had to push Scroll Lock button but it was something totally new for me.

How BSDForums.org looked like.

bsdforums.png

This is the earliest screenshot I got from that period, and Gentoo setup looked very similar.

vermaden-2005.11.08.jpg

Why FreeBSD and not OpenBSD or NetBSD? Probably because Gentoo based most their concepts on the FreeBSD solutions, so that led me to FreeBSD instead of the other BSD operating systems. Currently I still use FreeBSD but I keep an steady eye on the OpenBSD, HardenedBSD and DragonFly BSD solutions and improvements.

As the migration path from Linux to FreeBSD is a lot easier – all configuration files from /home can be just copied – the migration was quite fast easy. I again had the Fluxbox configuration which I used on the Gentoo. Now – on FreeBSD – it started to fell even more like AMIGA times. Everything is/has been well thought and had its place and reason. The documentation was good and the FreeBSD Community was second to none.

I even decided to upgrade the hardware to something more exotic. I got Gigabyte-GA-7DPXDW server motherboard with dual CPU sockets – and as Athlon XP (desktop) processors were very easily modified to ‘be’ Athlon MP (server) ones I got also the second one along with 1 GB of ECC RAM.

gigabyte-GA-7DPXDW.jpg

This dual CPU setup – quite unusual at these times – server me very well. I switched from nvidia binary blob driver to software but open nv because nvidia would break my uptime every several days πŸ™‚

I accumulated 30 days of uptime on that desktop box, not bad for a system without any emergency UPS πŸ™‚

uptime-vermaden.png

This was also the last time I used ECC RAM on FreeBSD (at least on my boxes) while ZFS did not even existed on FreeBSD πŸ™‚ But as time flied I started to feel the need for something faster. As I also got interested in Intel graphics card I got the new motherboard with fastest Intel graphics card available then – as silly as it sounds – the Asus P5B-V with Intel X3000 GMA … and that was a terrible idea because FreeBSD graphics stack supported all the Intel graphics cards instead of that one. At the beginning I used software vesa driver but the problem was not the performance of the driver (as I also had quad core Intel Q6600 CPU) but the resolution on the screen. As I got 1280 x 1024 screen by then using limited 1024 x 768 was real PITA. I decided that I will try something else then FreeBSD will Intel X3000 support finally arrives. I needed to do something fast as I also needed to write my Masters Thesis at that time.

That was in the middle of 2007. I wanted to try the other end of the Linux distributions spectrum. Ubuntu. I could not go more ‘desktop’ way πŸ™‚ It of course installed gently with GNOME 2 environment and pulseaudio already unfortunately existed. As I preferred to run my computer all the time back then (I did not payed the electricity bills) there were several things that annoyed my very much. For example the mentioned pulseaudio – the sound freezed after one-two days of using the computer (even if I did not played any music or videos) and it stayed that way. I could restart pulseaudio or reload the ALSA modules but it stayed in this SUSFU state (situation unchanged still fucked up) until reboot. As I needed to finish my Masters Thesis I did not had time to reinstall into something else as pulseaudio will be probably similarly broken on other Linux distributions and FreeBSD was still lacking the Intel X3000 GMA support. Generally GNOME 2 experience was not bad but I really missed all my custom settings, keyboard shortcuts and customized behavior. I remained in pain on the Ubuntu for two months – to the time I have finished my Masters Thesis about Operating Systems’ Virtualization which you can download and read but its in Polish so use translator if needed πŸ™‚

This is how Ubuntu looked back then.

ubuntu.jpg

I also had ‘side’ journey to the Mac wonderland as I got opportunity to use Macbook Pro with Mac OS X Leopard for a year. That allowed me to get real ‘feel’ of the Mac ecosystem and their hardware (and philosophy) so I will not repeat same stereotypes over and over again like a lot of anti-apple people. But after I switched back to FreeBSD system at work it just felt better. I used Terminal.app on Mac a lot but the xterm(1) at FreeBSD just felt more natural.

What makes me laugh now that I created Mac styled Fluxbox themes years till I got to run Mac and I still like Mac OS X look from the Leopard times.

vermaden-2007.10.14-mac.png

There was time on which I also played with Solaris (and later OpenSolaris). I must admit that there was time when Solaris so called Java Desktop based on GNOME 2 was really looking good. It was so good that only Mac OS X could only rival it for the best looking os by then.

solaris-10-GNOME-2-java-desktop.png

I really liked Solaris concepts and solutions like Zones and ZFS, also Crossbow, Comstar or IPS (FreeBSD did not had PNGng by then). But I always got problem with ‘desktop’ software. While I had everything in the FreeBSD Ports – almost the same amount of applications that is available on Linux – there was always some applications lacking in the Solaris world.

The Solaris ‘journey’ also left print on my soul so my Fluxbox themes went into Solaris style πŸ™‚

vermaden-2007.07.30-solaris-java-fluxbox-system.png

After the Ubuntu fiasco I got other motherboard as FreeBSD still did not supported Intel GMA X3000 card and settled in the FreeBSD land again. What a relief it was after this pulseaudio nonsense. In the meantime as I read a lot of good experiences about Openbox I decided to try it out instead of Fluxbox. It was strange feeling to mess with XML configuration files at the beginning but as I got used to it and ordered the rc.xml and menu.xml configuration files properly it was not a problem. Since then I used FreeBSD on different machines including physical servers, virtual machines and laptops. I learned that adequate supported hardware is the most important factor in FreeBSD ecosystem.

I still use Openbox and still use FreeBSD today and my desktop looks like that one below.

vermaden-NOW.jpg

After 15 years of using various Windows, UNIX (macOS/AIX/HP-UX/Solaris/OpenSolaris/Illumos/FreeBSD/OpenBSD/NetBSD) and UNIX-like (Linux) systems I always come to conclusion that FreeBSD is the system that sucks least. And sucks least with each release and one day I will write why FreeBSD is such great operating system … if I already haven’t πŸ™‚

UPDATE 1

As Roman Zolotarev got a moment he added my story to his Tell Your BSD Story page.

Thanks Roman!

You may check it for yourself at Slawomir Wojciech Wojtczak (vermaden) runs FreeBSD page.

EOF

Β 

Valuable News – 2018/09/02

UNIX

In Other BSDs for 2018/08/25.
https://www.dragonflydigest.com/2018/08/25/21658.html

FreeBSD 12.0-ALPHA3 Available.
http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/12.0/

FreeBSD bsdinstall/zfsboot enables new UEFI+GELI support.
https://svnweb.freebsd.org/base?view=revision&revision=338282
https://svnweb.freebsd.org/base?view=revision&revision=336252
https://reviews.freebsd.org/D12315

Setup unbound-adblock on OpenBSD.
https://www.geoghegan.ca/unbound-adblock.html

New XigmaNAS Release 11.2.0.4.5925 based on FreeBSD 11.2-RELEASE-p2.
https://sourceforge.net/projects/xigmanas/files/XigmaNAS-11.2.0.4/11.2.0.4.5925/

Mac-like FreeBSD Laptop.
http://trafyx.com/?p=2551

Baldur’s Gate in 1080p on OpenBSD via GemRB.
https://bsd.network/@thfr/100620499116867070

My Backup Solution Leveraging OpenZFS/rsync/WOL/crontab.
https://farhan.codes/2018/05/20/my-backup-solution-leveraging-openzfs-rsync-wol-and-crontab/

Why I’m mostly not interest in exploring new fonts (on Unix).
https://utcc.utoronto.ca/~cks/space/blog/unix/MyFontDisinterest

Why ed(1) is not a good editor today.
https://utcc.utoronto.ca/~cks/space/blog/unix/EdNoLongerGoodEditor

Mozilla will disable legacy Firefox 52 ESR add-ons on 2018/09/05.
https://blog.mozilla.org/addons/2018/08/21/timeline-for-disabling-legacy-firefox-add-ons/

Linux Maintains Bugs – Real Reason ifconfig(8) on Linux is Deprecated in ip(8).
https://farhan.codes/2018/06/25/linux-maintains-bugs-the-real-reason-ifconfig-on-linux-is-deprecated/
https://lkml.org/lkml/2012/12/23/75

Remove Yarrow PRNG algorithm option in accordance with due notice given in random(4) on FreeBSD.
https://twitter.com/lattera/status/1033734043207172096
https://svnweb.freebsd.org/base?view=revision&revision=338324
https://svnweb.freebsd.org/base?view=revision&revision=273872

Insight into Future of TrueOS BSD and Project Trident.
https://itsfoss.com/project-trident-interview/

Look Beyond the BSD Teacup – OmniOS Installation.
https://eerielinux.wordpress.com/2018/08/25/a-look-beyond-the-bsd-teacup-omnios-installation/

HardenedBSD 1100056.4 Released.
https://hardenedbsd.org/article/op/2018-08-27/stable-release-hardenedbsd-stable-11-stable-v11000564

BSD vs Linux. (old but worth resurrecting)
https://www.over-yonder.net/~fullermd/rants/bsd4linux/01

Send packet of death to any Linux 4.x kernel and it will make CPU spin in infinite loop.
https://twitter.com/perrymetzger/status/1034196371316719617
http://www.openwall.com/lists/oss-security/2018/08/27/1

Raspberry Pi as Stratum-1 NTP Server.
https://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html

Linux 4.18 on Intel Core 2 Duo makes CPU stall and complete system freeze.
https://www.phoronix.com/scan.php?page=news_item&px=Linux-4.18-Old-CPU-Issue

Oracle Solaris 11.4 Released for General Availability.
https://blogs.oracle.com/solaris/oracle-solaris-114-released-for-general-availability

FreeBSD: UEFI Bootloader stuck on BootCurrent/BootOrder/BootInfo on Asus Motherboards (and fix).
https://www.neelc.org/freebsd-uefi-on-asus-motherboards/

FreeBSD on 11″ MacBook Air 5.1 (mid-2012).
https://www.geeklan.co.uk/?p=2214
https://www.geeklan.co.uk/files/macbookair/freebsd-dmesg.txt

OpenBSD on 11″ MacBook Air 5.1 (mid-2012).
https://www.geeklan.co.uk/?p=1283

The Design and Implementation of the NetBSD (and FreeBSD) rc.d system.
https://www.usenix.org/legacy/events/usenix01/freenix01/full_papers/mewburn/mewburn_html/index.html

OpenBSD Gaming Resource – What Games Are Available and Where to Get Them.
https://mrsatterly.com/openbsd_games.html

In 2005/05 FreeBSD team described TLBleed on SA and suggested to disable INTEL SMT.
https://bsd.network/@sehnsucht/100637654685544116
https://www.freebsd.org/security/advisories/FreeBSD-SA-05:09.htt.asc

BSD Now 260 – Hacking Tour of Europe.
https://www.jupiterbroadcasting.com/126821/hacking-tour-of-europe-bsd-now-260/

BSD Now 261 – FreeBSDcon Flashback.
https://www.jupiterbroadcasting.com/126916/freebsdcon-flashback-bsd-now-261/

OpenBSD on the Microsoft Surface Go.
https://jcs.org/2018/08/31/surface_go

2FA with SSH on OpenBSD.
https://chown.me/blog/2FA-with-ssh-on-OpenBSD.html

How does the process title works?
https://oshogbo.vexillium.org/blog/51/

In Other BSDs for 2018/09/01.
https://www.dragonflydigest.com/2018/09/01/21713.html

New FreeBSD 12.0-ALPHA4 Snapshots Available.
https://lists.freebsd.org/pipermail/freebsd-snapshots/2018-September/000445.html

CNCF Survey: Use of Cloud Native Technologies in Production Has Grown Over 200%.
https://twitter.com/nitisht_/status/1035008388713701377
https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/

The bsd.network Mastodon Instance runs on OpenBSD 6.4-beta.
https://twitter.com/phessler/status/1035664766906368001

OpenBSD Workstation.
http://eradman.com/posts/openbsd-workstation.html

Badness Enumerated by Robots.
https://bsdly.blogspot.com/2018/08/badness-enumerated-by-robots.html

CoreDNS 1.2.2 Released.
https://github.com/coredns/coredns/releases/tag/v1.2.2

Hardware

SMT is Tagged on AMD CPUs thus not vulnerable like Intel CPUs.
https://www.extremetech.com/wp-content/uploads/2018/08/AMD-SMT-Zen-1.jpg

Hyper Converged NAS with Silverstone CS01-HS and Silverstone SST-ST30SF PSU.
https://www.silverstonetek.com/product.php?pid=458

GlobalFoundries Stops All 7nm Development – Opts To Focus on Specialized Processes.
https://www.anandtech.com/show/13277/globalfoundries-stops-all-7nm-development

AMD 7nm CPUs/GPUs To Be Fabbed by TSMC as GlobalFoundries Stops 7mm Development.
https://www.anandtech.com/show/13279/amd-to-fab-7nm-cpus-gpus-at-tsmc

Is Intel Hyper-Threading a Fundamental Security Risk?
https://www.extremetech.com/computing/276138-is-hyper-threading-a-fundamental-security-risk

Why the Future of Data Storage is (Still) Magnetic Tape.
https://spectrum.ieee.org/computing/hardware/why-the-future-of-data-storage-is-still-magnetic-tape

UNITEX USB 3.0 connected LTO7 tape drive.
http://www.cxp.com.au/articles/unitex-launches-usb3-0-connected-lto7-tape-drive

UNITEX USB 3.0 connected LTO8 tape drives.
http://www.cxp.com.au/articles/unitex-launches-high-speed-usb3-0-and-lto8-usb-connected-tape-drives

Spectre and Meltdown in Hardware: Intel Clarifies Whiskey Lake and Amber Lake.
https://www.anandtech.com/show/13301/spectre-and-meltdown-in-hardware-intel-clarifies-whiskey-lake-and-amber-lake

Performance Cost of Spectre/Meltdown/Foreshadow Mitigations on Linux 4.19.
https://www.phoronix.com/scan.php?page=article&item=linux-419-mitigations

Life

Eating in 10-hour window can override disease-causing genetic defects, nurture health.
https://www.salk.edu/news-release/eating-in-10-hour-window-can-override-disease-causing-genetic-defects-nurture-health/

My life with 12 programmers in 2 rooms and one 21st century dream.
https://www.salon.com/2016/09/17/hacker-house-blues-my-life-with-12-programmers-2-rooms-and-one-21st-century-dream/

Tech Workers Say Poor Leadership Is Number One Cause for Burnout.
http://blog.teamblind.com/index.php/2018/08/20/tech-workers-say-poor-leadership-is-number-one-cause-for-burnout/

Procrastination: It’s pretty much all in the mind.
https://www.bbc.com/news/health-45295392

Other

After 24 years Doom II final secret has been found!
https://www.rockpapershotgun.com/2018/08/31/after-24-years-doom-iis-final-secret-has-been-found/

3M Knew About the Dangers of PFOA and PFOS Decades Ago – Internal Documents Show.
https://theintercept.com/2018/07/31/3m-pfas-minnesota-pfoa-pfos/

EOF