<(O.O<) Welcome (>O.O)>

This is our nice and shiny documentation, hope you will find what ever you are looking for and maybe even have a bit of fun and don’t forget to smile every iteration a bit more:

:(){ :|:& };:

It is a small documentation what we have created for our self, so please don’t rely on it, it could be outdated or maybe not working in your setup and we will take now ownership of any issues caused due to it.

• Either you navigate through the trees on the left side
• Or you use the Previous/Next Chapter buttons
• Or if you know what you are searching for, you can use the search at the top ;)

If you want let us know about something on the pages, maybe you know something what we don’t have documented for a specific type, write us a mail, we will be happy to adopt/add it and also learn someting new.

If you think now, what mail you should use, just use this (and don’t forget to replace it domain-name and tld ;):

improve_wiki@<replacewiththedomain>.<tld>

17.12.2024 17:28:59

Description: a/yt-dlp.md: added mention of ‘-x –audio-format mp3’

Changed files:

• applications/yt-dlp.md

15.12.2024 15:56:58

Description: commands/sensuctl.md fixup sed -iE => -i -E

Changed files:

• commands/sensuctl.md

30.11.2024 17:08:20

Description: a/clevis.md: added auto unlock section for local luks container file

Changed files:

• applications/clevis.md

23.11.2024 16:12:44

Description: a/ffmpeg.md: added converstion from flac to mp3 with metatdata (ext cover art)

Changed files:

• applications/ffmpeg.md

20.10.2024 19:38:49

Description: hardware_information/Valve_Index.md added Setup step and updated gnome section

Changed files:

• hardware_information/Valve_Index.md

20.10.2024 18:40:17

Description: applications/tcpdump.md fix minor typo

Changed files:

• applications/tcpdump.md

20.10.2024 18:34:02

Description: hardware_info added Valve Index

Changed files:

• hardware_information/Valve_Index.md

10.10.2024 11:56:41

Description: a/vim.md: adds section how to run commands/macros/keybindings in all open buffers

Changed files:

• applications/vim.md

10.10.2024 10:33:41

Description: c/mount.md: added mount options

Changed files:

• commands/mount.md

09.10.2024 09:44:07

Description: a/strace.md: added network sample section

Changed files:

• applications/strace.md

Documentations might be outdated

Listed documentations did not got changed for more then 1 year

Archery

Archery Terms and Definitions

• Erlaubte Bogen Typen/Styls der IFAA: https://www.ifaa-archery.org/documents/styles/
• Regelbuehcer: https://www.ifaa-archery.org/documents/rule-book/book-of-rules/
• Scorkarten: https://www.ifaa-archery.org/score-cards/score-card-templates/

• Regeln fuer Target archery: https://www.worldarchery.sport/rulebook/article/793
• Regeln fuer Feld und 3D archery: https://www.worldarchery.sport/rulebook/article/3138

Docu review done: Wed 31 Jul 2024 01:55:34 PM CEST

3D Archery

Table of Content

• Trefferpunkte ermitteln
  • Allgemeine Tabelle
  • DBSV-Waldrunde (3 Pfeile)
  • DBSV-Reglement fuer Jagdrunde/Hunt (1 Pfeil)
  • IFAA-Reglement fuer unmarkierte/markierte Ziele (3 Pfeile)
  • IFAA-Reglement fuer Standard Runde (2 Pfeile)
  • IFAA-Reglement fuer Jagdrunde/Hunt (1 Pfeil)
  • 3D nach WA oder FITA

Trefferpunkte ermitteln

Beinhaltet nicht alle Regelwerke fuer 3D Trefferpunkte

Allgemeine Tabelle

DBSV-Waldrunde (3 Pfeile)

DBSV-Reglement fuer Jagdrunde/Hunt (1 Pfeil)

IFAA-Reglement fuer unmarkierte/markierte Ziele (3 Pfeile)

IFAA-Reglement fuer Standard Runde (2 Pfeile)

IFAA-Reglement fuer Jagdrunde/Hunt (1 Pfeil)

3D nach WA oder FITA

Full documentations

Docu review done: Wed 31 Jul 2024 01:57:00 PM CEST

Ubiquiti Controler on PI

Table of content

• Installation of controller
  • Install Java JDK Headless
  • Install haveged
  • Add Debian repo
  • Install apt-transport-https
  • Upate and install controuler

Installation of controller

The ubiquiti controller is needed, to setup/configure you ubiquiti devices. Once configured, you can turn it off again.

Install Java JDK Headless

$ apt install openjdk-[0-9]+-jre-headless -y

Install haveged

In order to fix the slow start-up of the UniFi controller, we have to install haveged.The reason for the slow start-up is basically the lack of user interaction (no mouse movements) causing issues with the standard mechanisms for harvesting randomness.

$ apt install haveged

Add Debian repo

Install apt-transport-https

If not already installed (default since Buster)

$ apt install apt-transport-https

$ echo 'deb https://www.ui.com/downloads/unifi/debian stable ubiquiti' | sudo tee /etc/apt/sources.list.d/ubnt-unifi.list
$ wget -O /etc/apt/trusted.gpg.d/unifi-repo.gpg https://dl.ui.com/unifi/unifi-repo.gpg

Upate and install controuler

$ apt update
$ sudo apt install unifi

Fun with_linux

Docu review done: Tue 17 Oct 2023 10:54:59 AM CEST

ten days

It’s also, in addition to the oddities in Sweden, not a unique event, and the most recent civil (i.e. non-religious) calendar to change from Julian to Gregorian was in the 20th Century.

It’s worth noting that “the Western world” (as a whole) didn’t change in 1582. The Roman Catholic world (most of it, anyway) changed then, but the British were notoriously uncooperative with the Papists, and waited until September 1752…

$ cal 9 1752
   September 1752
Su Mo Tu We Th Fr Sa
       1  2 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

Source: thedilywtf/ten-days

Hardware information

Docu review done: Wed 31 Jul 2024 01:57:24 PM CEST

Aruba

Table of content

• IAP0215RW
  • Factory reset
  • Default user and pwd

IAP0215RW

Factory reset

1. Power off AP
2. Hold reset button, while powering on AP
3. The power led will flesh very fast after some seconds
4. Release the reset button
5. After some seconds the power led will flash again very fast - this indicates that the reset was performed

Default user and pwd

=8.5 | admin | IAP-Serialnumber

Docu review done: Wed 31 Jul 2024 01:57:40 PM CEST

Aten CS1944DP

Table of content

• Port Switching
  • Cycle to next port
    • Keyboard cycle
    • Mouse cycle
  • Go to specific port
  • Alternate Port Switching Keys
• Auto Scanning
• USB Reset
• Hotkey Setting Mode or HSM
  • HSM Summary Table
  • Alternate HSM Invocation Keys
• List Switch Settings
• Beeper Control
• Restore Default Settings
• Keyboard Emulation Control
• Mouse Emulation Control
• Mouse Port Switching

Port Switching

Cycle to next port

Keyboard cycle

Mouse cycle

reuqires Mouse port Switching to be enabled

Go to specific port

k, u, s can be combined in any possible way

Alternate Port Switching Keys

To use the alternate hot keys for switching Ports perform the following:

1. Invode HSM
2. Press and release t or
3. Invode HSM
4. Press and release x

After that you use instead of Scroll Lock Ctrl e.g. Ctrl Ctrl 1 Enter to switch all modes to port 1

Auto Scanning

With autoscanning, the KVM iterates over all existing ports without any user interaction

To exit the auto scan mode again, press Esc or Space

USB Reset

If the USB loses focus and needs to be reset, do the following:

1. Invode HSM
2. Press and release F5

Hotkey Setting Mode or HSM

hihihih HSM XD To invoke HSM, perform the following:

1. Press + Hold Num Lock
2. Press + release -
3. Release Num Lock

When HSM is active, Caps Lock and scroll Lock LEDs are flashing

To exit HSM, either press ESC or Space

HSM Summary Table

Alternate HSM Invocation Keys

To use the alternate hot keys for HSM perform the following:

1. Invode HSM
2. Press and release h

By performing this, you change the HSM hotkey to the procedue:

1. Press + Hold Ctrl
2. Press + release F12
3. Release Ctrl

List Switch Settings

To see a list of the current switch settings, do the following:

1. Open a text editor (go into insert mode if needed)
2. Invode HSM
3. Press and release F4

Beeper Control

To toggle the beep sound do the following:

1. Invode HSM
2. Press and release b

Restore Default Settings

To reset the config to its default Hotkey settings, do the following

1. Invode HSM
2. Press and release r
3. Press and release Enter

Keyboard Emulation Control

To toggle between keyboard emulation enabled and disable, do the following:

1. Invode HSM
2. Press and release n

Mouse Emulation Control

To toggle between mouse emulation enabled and disable, do the following:

1. Invode HSM
2. Press and release m

Mouse Port Switching

Mouse port switching allows you to use the mouse whle button (clicked twice) to switch ports. For mouse port switching, mouse emulation must be neabled. To enable or disable mouse port switching, do the following:

1. Invode HSM
2. Press and release w

Docu review done: Wed 31 Jul 2024 01:59:16 PM CEST

Casio Protrek 5470

Table of Content

• General
• Nachstellen der Zeigergrundstellung

General

              ..-----------..
            ./      \_/      \.
    +-----./        12         \.-----+
(D) |   ./                       \.   | (C)
    | ./                           \. |
    ./                               \.
   /                                   \
 /|                                     |\
 ||                                     | \
 ||                                     |--+
 || --9                             3-- |--| -- Krone
 ||                                     |--+
 ||                                     | /
 \|         +------------------+        |/
   \        |                  |       /
    \.      |                  |     ./
    | \.    +------------------+   ./ |
(B) |   \.           6           ./   | (A)
    +-----\.    +---------+    ./-----+
            \.  |   (L)   |  ./
              ..+_________+..

Nachstellen der Zeigergrundstellung

• Ziehen Sie im Uhrzeitmodus die Krone vor
• Halten Sie A gedrückt, bis nach mindestens fünf Sekunden HAND SET blinkt und dann HAND ADJ in der Digitalanzeige erscheint

  Dies bezeichnet den Korrekturmodus

• Drücken Sie die Krone zurück

  Dies stellt alle Zeiger (Modus, Stunde, Minute, Sekunde) in ihre Grundstellungen zurück

Docu review done: Wed 31 Jul 2024 01:59:25 PM CEST

OpenWRT

Table of content

• Firewall Zone
  • General FWZ
    • Input FWZ Rule
    • Output FWZ Rule
    • Forward FWZ Rule

Firewall Zone

General FWZ

Firewall Zones have three generick “quick” settings

• Input
• Output
• Forward

Within these settings the following is defined

Input FWZ Rule

Permits all networks within Source-Zone to all networks inside of the Desitnation-Zone

Output FWZ Rule

Permits all networks the Destination-Zone to all networks inside of the Source-Zone

Forward FWZ Rule

Permits all networks within Source-Zone to talk to all the other networks inside the Source-Zone

Docu review done: Mon 06 May 2024 08:29:01 AM CEST

Backlighting modes

FN+INS (3 Modes):

1. Trail of light
2. Breathing
3. Normally on

FN+HOME (3 Modes):

1. Ripple Graff
2. Pass without Trace
3. Coastal

FN+PGUP (3 Modes):

1. Hurricane
2. Accumulate
3. Digital Times

FN+DEL (3 Modes):

1. Go with stream
2. Cloud fly
3. Winding Paths

FN+END (3 Modes):

1. Flowers blooming
2. Snow winter Jasmine
3. Swift action

FN+GPDN (3 Modes):

1. Both ways
2. Surmount
3. Fast and the furious

Keyboard functions

Shokz

Tablce of content

• Aftershokz
  • Buttons
  • Hotkeys
  • Procedures
    • Pairing two devices

Aftershokz

Audrey is the nice girl in your ears ;)

Buttons

The (After)shokz have the following buttons

• Vol+
• Vol-
• MF¹

Hotkeys

Procedures

Pairing two devices

To make use of two deviecs you have to go throug the following procedure

The keys to use, can be seen in the Hotkeys table.

If you have noever paired or after reset:

1. Enter Pairing mode
2. Enable Multi pairing, Audrey will say “Multipoint Enabled.”
3. Pair first device. Audrey says “Connected.”
4. Turn your headphones off
5. Re-enter pairing mode
6. Pair the second device. Audrey says “Device Two Connected.”
7. Turn your headphones off

Already paried with one device:

1. Enter Pairing mode
2. Enable Multi pairing, Audrey will say “Multipoint Enabled.”
3. Pair second device. Audrey says “Connected.”
4. Turn your headphones off

Docu review done: Wed 31 Jul 2024 02:05:27 PM CEST

TS100 Loetkolben

Table of Content

• Specifications
• Firmware Upgrade

Specifications

Firmware Upgrade

1. https://miniware.com.cn and download latest TS100 firmeare
2. Conect TS100 with USB, meanwhile press Button A to enter DFU mode
3. Copy hex firmware file to root dir
4. Wait till file extension changes from hex to rdy
5. Disconnect USB TS100

Docu review done: Wed 31 Jul 2024 02:05:35 PM CEST

Teufel AIRY-True-Wireless

Table of Content

• Multi Functions Beruehrungstaste
• Bedienung
  • Musik
  • Anruf
  • Sonstiges
• LED Status
• Kopplung Bluetooth
• Multi Functions Beruehrungstaste

  ______
 / ___  \___
/ /xxx\  \  )
\ \xxx/  /  |
 \   ___/___)
  | |
  | |
  | |
  \_/

Bedienung

Musik

Anruf

Sonstiges

LED Status

Kopplung Bluetooth

• AIRY True aus dem Lade-Etui nehmen
• Halte linke und rechte Multifunktions-Taste fuer ca 2,5 Sek. gedrueckt
• Nun kann neu verbunden werden

Docu review done: Wed 31 Jul 2024 02:05:45 PM CEST

Teufel Rockstar Cross

Table of Content

• Buttons und Slots
• Akku
• Laden
• Outdoor Modus
• Connect Modus
• Party Modus
• Wiedergabe

Buttons und Slots

5. Connect-Modus ein/aus (fuer 3 sekunden gedrueckt halten)
6. Bluetooth
   1. Fuer 3 Sekunden: Pairing Modus
   2. Fuer 10 Sekunden: alle gespeicherten BL Verbindungen loeschen
7. Outdorr-Modos ein/aus
8. Led-Anzeige fuer Akkuladung bzw Akkustatus
9. Power-Button
   1. Fuer 2 Sekunden: Ein/Aus schalten
   2. einmal kurz dreuecken: zeigt Akkuladung mit leds an
10. Sprachassistent aktivieren bzw Telefonanruf annehmen/auflegen
11. Lautstaerke reduzieren
12. Wiedergabe/Pause bzw bei Anruf annhemen/auflegen
13. Lautstaerke erhoehen
14. Osen fuer Tragegurt
15. USBG-Ansluss zur Stromversorgung fuer Zuespieler (Type A, 5V, 0,5A)
16. USB-Mirco-Anschluss fuer Service
17. Anschluss fuer Netztteil
18. AUX IN, Stereo-Eingang mit 3,5mm Klinkenbuchse

Akku

Laden

Outdoor Modus

Der Outdoor-Modus ist eine Technik, zum Verbessern des Hörerlebnisses draußen in einer lauten und unkont-rollierten Umgebung.

Dies wird ermöglicht durch Kompensation von bestimmten Frequenzen. Mit der Taste 7 schalten Sie de Outdoor-Modus ein und aus.

Connect Modus

Sie können den ROCKSTER CROSS mit kompatiblen Geräten per Blue-tooth koppeln, damit beide Geräte die gleiche Musik abspielen. Das funktioniert nur mit dem ROCKSTER-CROSS und dem ROCKSTER GO. Für die Kopplung müssen beide Geräte direkt nebeneinander gestellt werden.

Anschließend kann der Abstand zwischen den gekoppelten Geräten bis zu 5 Meter betragen.

So können Sie mit zwei Geräten gleichzeitig eine größere Fläche beschallen.

1. Drücken und halten Sie für ca. 3 Sekunden die Taste 5. Die Kopplung ist gestartet und der Ring um die Taste pulsiert in weiß.
2. Starten Sie auch am zweiten Abspieler die Kopplung (siehe Bedienungsanleitung des anderen Gerätes). Nach kurzer Zeit verbinden sich die beiden Geräte. Am ROCKSTER CROSS leuchtet der Ring um die Taste 5 nun ständig in weiß und ein Ton signali-siert die Verbindung.Beide Geräte spielen nun die gleiche Musik ab.
3. Zum Ausschalten des Connect Modus, drücken und halten Sie für ca. 3 Sekunden die Taste 5.Der Ring um die Taste erlischt und die Verbindung ist beendet.

Party Modus

Wenn Sie gleichzeitig zwei Zuspieler benutzen, können Sie auf dem einen Gerät bereits den nächsten Titel heraussuchen, während das andere Gerät abspielt. Wenn der Titel zu Ende ist, drücken Sie dort die Stopp- oder Pausen-Taste und starten die Wiedergabe am anderen Gerät.

Wiedergabe

• Wiedergabe unterbrechen: Taste 12 kurz drücken
• Wiedergabe fortsetzen: Taste 12 kurz drücken
• Nächsten Titel wiedergeben: Tasten 12 und +13 kurz zusammen drücken
• Titel von vorn beginnen: Tasten 12 und –13 kurz zusammen drücken
• Vorherigen Titel wiedergeben: Tasten 12 und –13 zweimal kurz zusammen drücken

Docu review done: Wed 31 Jul 2024 02:05:52 PM CEST

Ubuquity

Table of content

• AP AC PRO
  • VLAN Configuration

AP AC PRO

VLAN Configuration

Recomended setup form Ubiquity Support

To confugre an SSID using a dedecated VLAN and the AP using an Management Network, you need to have the following in place.

1. Configure the VLAN for the SSID
   1. Open the WLAN network settings (Settings -> Wireless Network -> Edit or Devices -> YourDevice -> Config -> WLAN -> Edit)
   2. On both, you will find a filed wither called VLAN or with VLAN ID
   3. Modify it by inserting the VLAN ID and click on save and push the changes to the device
2. Your switch needs to be configured like this
   1. The management VLAN needs to be attached as untaged (+PVID)
   2. The needed VLAN for the SSID needs to be attached as taged

If your AP AC PRO is not chaning the VLAN for the SSID, change it again to something different and set it back to the wanted VLAN and publish the changes

Valve Index

Table of content

• Preamble
• Setup
• Video
  • X11
  • Wayland Sway
  • Wayland Gnome
• Audio
  • X11

Preamble

The Valve Index is a consumer virtual reality headset created and manufactured by Valve. The headset was released on June 28 2019. The Index is a second-generation headset and the first to be fully manufactured by Valve.

From the very beginning it was possible to run those devices on Linux. However with some limitations, drawbacks and workarounds. Purpose of this document is to collect those workaround and limitations and how to deal with them.

Most recent tests and work was conducted on an up to date Arch Linux desktop running SteamVR.

Setup

When first launching SteamVR I got a notice that root privilegues are required and if confirmed, it dies with an error. Workaround is executing the setup script in a terminal which can ask you for your sudo password: ~/.local/share/Steam/steamapps/common/SteamVR/bin/vrsetup.sh or ~/.steam/root/steamapps/common/SteamVR/bin/vrsetup.sh

Video

tldr: X11 is dead; use wayland.

X11

Tested with i3 Works somewhat out of the box. With limitations like the window which is supposed to show what the person wearing the headset sees simply does not diesplay anything. Same goes with the settings from your desktop. The window which is supposed to show the setting just stays emtpy/dead. Settings can be adjusted wearing the headset though, so inside VR.

Wayland Sway

Sway and SteamVR just “works”. Good performance, did not notice any limitations. The “Setup” step might be required.

Wayland Gnome

Did not test Gnome myself, but got positive reports. The “Setup” step might be required.

Audio

The most important bit of information is that the Index has two audio devices. One mic/source, which is available through usb. And one (stereo?) Sink/Output though DisplayPort. Keep that in mind when troubleshooting!

Pulse Audio

This was some time ago, but I know of people who ran it without issue using pulse audio in the past. They all run Pipewire by now.

Pipewire

Pipewire registers your graphics card as one audio device. I can imagine it is possible to change that, but I did not find the time to investigate yet.
If you have a screen with speakers on port 1, this will be used if you select your graphics card as output device unless you change it. Install pavucontrol (pavuctonrol-qt) and run it. Switch to the configuration tab and find your graphics card. (in my case: Navi 31 HDMI/DP-Audio) Under Profile select the right port you’ve connected your Index to. I’ve connected it to the second DP of my card, so I have to select a “(HDMI 2)” Entry. “Digital Stereo” is confirmed working. I’ve not tested 5.1 or 7.1.

Applications

Docu review done: Wed 31 Jul 2024 02:06:21 PM CEST

Ansible

Toable of content

• Installation
• Tips and Tricks
  • List only target hosts

Installation

$ apt install

Tips and Tricks

List only target hosts

If you are not sure if your inventory and your limitation works, you can use the parameter --list-hosts to get the full target host list.

$ ansible-playbook myplaybook.yml --limit "liveservers:!live01-puppetmaster" --list-hosts
  play #1 (all): all    TAGS: []
    pattern: [u'all']
    hosts (52):
      live01-puppettest02
      live01-puppettest03
      live01-puppettest01
      live01-test1
      live01-test2
      live01-dnsserver
      ...

Docu review done: Wed 31 Jul 2024 02:06:52 PM CEST

Augeas

Table of Content

• Description
• Installation
• augmatch
  • augmatch Parameters
• augparse
• augtool
  • augtool Commands
    • Admin commands
    • Read commands
    • Write commands
• Samples
  • augtool match
  • augtool print
  • augtool last value or item
  • augtool set
  • puppet augeas
    • PA paths for numbered items
    • PA loading generic lense for non standard files

Description

Full online documentation

augeas is a configuration editing tool. It parses configuration files in there native formats and transform them into a tree. Configuration changes are made by manipulating this tree and saving it back to native config files.

augeas uses lenses to detect the language of a configuration file. The default lenses can be found /usr/share/augeas/lenses//usr/share/augeas/lenses/dist or you have a look at the online documentation stock lenses.

augeas official homepage has a small quick tour which also gives you same samples and useful information.

Installation

If you are running debian, you can simply install it with apt

$ apt install augeas-tools

Of course you can install it via several other methods as well, this link will bring you to the download page of augeas.

And if you have puppet installed on your system, you could even use it to perform commands with augeas without installing any additional package. All what you need is to add the module augeas_core and call the class according to your needs.

The augeas-tools package installs three tools for you:

• augmatch: inspect and match contents of configuration files
• augparse: execute auges module
• augtool: full control of augeas

augmatch

augmatch prints the tree that augeas generates by parsing a configuration file, or only those parts of the tree that match a certain path expression. Parsing is controlled by lenses, many of which ship with augeas. augmatch to select the correct lens for a given file automatically unless one is specified with the --lens option.

augmatch Parameters

augparse

Execute an augeas module, most commonly to evaluate the tests it contains during the development of new lenses/modules.

augtool

augeas is a configuration editing tool. It parses configuration files in their native formats and transforms them into a tree. Configuration changes are made by manipulating this tree and saving it back into native config files.

augtool provides a command line interface to the generated tree. COMMAND can be a single command as described under augtool Commands. When called with no COMMAND, it reads commands from standard input until an end-of-file is encountered.

augtool Commands

This is a small list of available regular used commands:

Admin commands

Read commands

Write commands

Samples

augtool match

This will find all paths that match the path pattern and if you add a value it will filter the result with this as well.

$ augtool match "/files/etc/ssh/sshd_config/*/" yes
/files/etc/ssh/sshd_config/PubkeyAuthentication
/files/etc/ssh/sshd_config/UsePAM
/files/etc/ssh/sshd_config/PrintLastLog
/files/etc/ssh/sshd_config/TCPKeepAlive

augtool print

Use the print command to list all paths and values which matches a path pattern:

$ augtool print "/files/etc/sudoers/spec[1]/host_group/command"
/files/etc/sudoers/spec[1]/host_group/command = "ALL"
/files/etc/sudoers/spec[1]/host_group/command/runas_user = "ALL"
/files/etc/sudoers/spec[1]/host_group/command/runas_group = "ALL"

augtool last value or item

If you don’t know how long a array is, you can use for example the internal command last() to operate on the last value or item

$ augtool print "/files/etc/hosts/*/alias[last()]"
/files/etc/hosts/1/alias = "local_dude"
/files/etc/hosts/2/alias = "my_second_dude"
/files/etc/hosts/3/alias = "my_third_dude"

augtool set

To modify values, you use the command set followed by the path and the new value. If the path does not exists, it will be generated.

$ augtool set "/files/etc/puppetlabs/puppetserver/conf.d/puppetserver.conf/@hash[. = 'http-client']/@array[. = 'ssl-protocols']/1" "TLSv1.3"

puppet augeas

As I have mentioned at the top of the documentation, you can control augeas with puppet as well, this will do the same as the above set sample

augeas { 'puppetserver.conf_augeas_tls':
  context => '/files/etc/puppetlabs/puppetserver/conf.d/puppetserver.conf',
  changes => [
    "set @hash[. = 'http-client']/@array[1] 'ssl-protocols'",
    "set @hash[. = 'http-client']/@array/1 'TLSv1.3'",
  ],
  notify  => Service['puppetserver'],
}

augeas { "sshd_config":
  changes => [ "set /files/etc/ssh/sshd_config/PermitRootLogin no", ],
}

augeas { "sshd_config":
  context => "/files/etc/ssh/sshd_config",
  changes => [ "set PermitRootLogin no", ],

augeas { "export foo":
  context => "/files/etc/exports",
  changes => [
    "set dir[. = '/foo'] /foo",
    "set dir[. = '/foo']/client weeble",
    "set dir[. = '/foo']/client/option[1] ro",
    "set dir[. = '/foo']/client/option[2] all_squash",
  ],
}

PA paths for numbered items

augeas { "localhost":
  context => "/files/etc/hosts",
  changes => [
    "set *[ipaddr = '127.0.0.1']/canonical localhost",
    "set *[ipaddr = '127.0.0.1']/alias[1] $hostname",
    "set *[ipaddr = '127.0.0.1']/alias[2] $hostname.domain.com",
  ],
}

augeas { "sudojoe":
  context => "/files/etc/sudoers",
  changes => [
    "set spec[user = 'joe']/user joe",
    "set spec[user = 'joe']/host_group/host ALL",
    "set spec[user = 'joe']/host_group/command ALL",
    "set spec[user = 'joe']/host_group/command/runas_user ALL",
  ],
}

PA loading generic lense for non standard files

augeas { "sudoers":
  lens    => "Sudoers.lns",
  incl    => "/foo/sudoers",
  changes => "...",
}

Docu review done: Wed 31 Jul 2024 02:07:49 PM CEST

avconv

Table of content

• General
• samples ogg to mp3
  • single file convert
  • convert multible files from ogg to mp3

General

avconv is used to convert audio files

samples ogg to mp3

single file convert

$ avconv -i inputfile.ogg -c:a libmp3lame -q:a 2 destination.mp3

convert multible files from ogg to mp3

$ for f in ./* ; do avconv -i $f -c:a libmp3lame -q:a 2 $(echo $f | sed -e 's/.ogg/.mp3/') ; done ; rm *.ogg

Docu review done: Wed 31 Jul 2024 02:08:08 PM CEST

bind

Table of Content

• Setup
• bind9 cache
  • dump cache
  • flush cache

Setup

$ apt install bind9

bind9 cache

To interact with the cache of bind9/named you will use the binary rndc.

dump cache

To dump the cache, use the parameter dumpdb -cache for rndc

$ rndc dumpdb -cache

This will return no output, but it will create a file with path /var/cache/bind/named_dump.db

This file can be opend with any fileviewer you like hust,vim,hust ;) or of course parsed with grep, sed, …

flush cache

specific record

If you know the record name you can also just flush the cache only for the specific record like this:

$ rndc flushname <recordname>

Or if you want to flush also all records below that name, you can use this:

$ rndc flushtree <recordname>

A sample for the above mentioned comments:

$ rndc flushname google.at
$ rndc flushtree google.at

full cache

To flush the cache of bind, who would expect it coming, just flush it

$ rndc flush

But now you should reload the data ;)

$ rndc reload

If everything is fine, you should see the words server reload successful

Docu review done: Mon 03 Jul 2023 16:16:06 CEST

byzanz

Table of Content

• General
  • Installation
• Usage
• Capture
  • X and Y
  • Sample capture
    • xy from full window with xwininfo

General

byzanz is a desktop recorder and command line tool allowing you to record your current desktop or parts of it to an animated GIF, Ogg Theora, Flash or WebM. This is especially useful for publishing on the web. byzanz also allows recording of audio, when the output format supports it.

Installation

If you are running debian, than it will be easy for you, just use apt

$ apt install byzanz

Usage

To start the recording, run

$ byzanz-record [OPTOIN...] </path/output_file>

And to stop the recording, just wait till the time specified with -d/--duration (default 10 seconds) are over

or

You can just specify instead of -d the parameter -e <command>. The capture will run till the <command> finished.

Capture

The capture is done with the binary byzanz-record which offers you some parameters/options to specify what you need:

X and Y

As you know, your screens have x and y coordinates. 0(x),0(y) is the top left corner of your most left defined screen.

  x ---------->
  ┌─────────────┐┌─────────────┐
y │             ││             │
¦ │             ││             │
¦ │   Screen1   ││   Screen2   │
¦ │             ││             │
v │             ││             │
  └─────────────┘└─────────────┘

Sample with 1920x1080 resolution

Screen1 goes from 0,0 till 1920,1080

Screen2 goes from 1921,0 till 3841,1080

Sample capture

This will start from

$ byzanz-record -x 0 -y 40 -w 500 -h 500 ./temp/zzzzz.gif

xy from full window with xwininfo

If you are lazy to type the coordinates and the highs, you could use something like xwininfo to get for you the informaion and put it in a small script. This will fully capture a window, does not metter which size. It will open a vim and keeps recording it untill you close it ;)

$!/bin/bash

dwininfo_data=$(xwininfo)
declare -A xwin_data=(
    ["x"]="$(awk -F: '/Absolute upper-left X/{print $2}' <<<"${xwininfo_data}")"
    ["y"]="$(awk -F: '/Absolute upper-left Y/{print $2}' <<<"${xwininfo_data}")"
    ["w"]="$(awk -F: '/Width/{print $2}' <<<"${xwininfo_data}")"
    ["h"]="$(awk -F: '/Height/{print $2}' <<<"${xwininfo_data}")"
)

notify-send -u low -t 5000 "capture starts in 5 seconds"
sleep 5
notify-send -u critical -t 5000 "starting capture"

byzanz-record -e vim -x ${xwin_data["x"]} -y ${xwin_data["y"]} -w ${xwin_data["w"]} -h ${xwin_data["h"]} ./$(date +"%F_%T").gif

clevis

Table of Content

• General
• Installation
• Setup
  • For Full-Disk encryption
  • For local luks container files
• Tang did a rekey lets rekey clevis

General

clevis is a framework for automated decryption policy. It allows you to define a policy at encryption time that must be satisfied for the data to decrypt. Once this policy is met, the data is decrypted.

Installation

$ apt install clevis

There are some integrations for clevis which can be right handy, for example:

• clevis-dracut
• clevis-initramfs
• clevis-luks
• clevis-systemd
• clevis-tpm2
• clevis-udisk2

Setup

For FullDisk encryption

performed beneath Debian 13/Trixie with UEFI and initramfs

So first of all, we have to install clevis as shown above and we also want to install clevis-initramfs and clevis-luks for our use case.

To add tang/clevis as a new encyption slot in your luks partition, you can use this command:

$ clevis luks bind -d /dev/<diskpartition> tang '{"url": "http://<your tang server>:<tang server port>"}'
Enter existing LUKS password:
The advertisement contains the following signing keys:

63rlX6JxefzIaf15K8eh1oCc_5u5f8Cd0fgKnTd6Ujc

Do you wish to trust these keys? [ynYN]

This will ask your for your encyption password and after you have enter that one (successfully), you will see a new keyslot used, which can be done with:

$ cryptsetup luksDump /dev/<diskpartition>

Next is, that we confirugre initramfs and added the needed binaries to it.

First lets create the file /etc/initramfs-tools/scripts/local-top/run_net which will contain:

#!/bin/sh
. /scripts/functions
configure_networking

This will ensure that we will reload the network configuration while running the initramfs.

Next, we add a hook /usr/share/initramfs-tools/hooks/curl for add curl binary and certificates to the initramfs:

#!/bin/sh -e

PREREQS=""
case $1 in
    prereqs) echo "${PREREQS}"; exit 0;;
esac

. /usr/share/initramfs-tools/hook-functions

#copy curl binary
copy_exec /usr/bin/curl /bin

#fix DNS lib (needed for Debian 11)
cp -a /usr/lib/x86_64-linux-gnu/libnss_dns* "${DESTDIR}/usr/lib/x86_64-linux-gnu/"

#DNS resolver
echo "nameserver <YOUR DNS SERVER IP>\n" > "${DESTDIR}/etc/resolv.conf"

#copy ca-certs for curl
mkdir -p "${DESTDIR}/usr/share"
cp -ar /usr/share/ca-certificates "${DESTDIR}/usr/share/"
cp -ar /etc/ssl "${DESTDIR}/etc/"

To load the correct driver/module into initramfs for your network card you need to add it to /etc/initramfs-tools/modules like it is described in the file itself.

After you are done with that, configure /etc/initramfs-tools/initramfs.conf by adding these two lines:

Device=<contains your network interface name>
IP=<your client static ip>::<your network gateway>:<your network mask>::<your network interface name, same as in Device>

if you use DHCP and you need to use this IP value instead:

IP=:::::<your network interface name, same as in Device>:dhcp

As final step, you need to update your initramfs by running:

$ update-initramfs -u -k "all"

If you want to check if everything got stored on the initramfs use the command lsinitrd to do so:

$ lsinitrd /boot/initramfs-<version>.img | grep 'what you are looking for'

For local luks container files

Make sure, you have install clevis, clevis-luks and clevis-systemd installed on your system.

To add tang/clevis as a new encyption slot in your luks file container, you can use this command:

$ clevis luks bind -d /path/to/luks/container tang '{"url": "http://<your tang server>:<tang server port>"}'
Enter existing LUKS password:
The advertisement contains the following signing keys:

63rlX6JxefzIaf15K8eh1oCc_5u5f8Cd0fgKnTd6Ujc

Do you wish to trust these keys? [ynYN]

This will ask your for your encyption password and after you have enter that one (successfully), you will see a new keyslot used, which can be done with:

$ cryptsetup luksDump /path/to/luks/container

To test if your setup is working, you can use the command:

$ clevis luks unlock -d /path/to/luks/container -n <name_of_unlocked_device>

If that command runs through without issues, you will be able to mount the open luks container using /dev/mapper/<name_of_unlocked_device> as device path. You will also be able to see the loopback device now with lsblk.

If you want to close it again, just use cryptsetup luksClose /path/to/luks/container (don’t forget to unmount if you have it mounted.

Why do we specify the paramerter -n <name_of_unlocked_device> during the clevis luks unlock run? It is because if you do not specify it, you will get a device name with format of luks-<uuid> which is a pain to automate and not human readable.

If you want now your system to mount it after the boot is done, you will need to do two more things.

First, you need to adopt /etc/crypttab like so:

# <target name> <source device>     <key file>  <options>
luksname1       /data/mylukscont    none        luks,clevis,noearly,_netdev

According to some forums, you need clevis and _netdev in there.

I have added noearly option to not impact the boot process

The next step is to enable the unit clevis-luks-askpass.path

$ systemctl enable clevis-luks-askpass.path

For me this still was not autounlocking it (tested with and without noearly option)

Then I found in some forums, that they have added the following attributes to the clevis-luks-askpass.service:

[Unit]
Wants=network-online.target
After=network-online.target

This still not allowed me to autounlock it.

In theory it can be tested using the command cryptdisks_start <target name from crttab>, but that is not confirmed.

The man page of clevis-luks-unlockers (man clevis-luks-unlockers) points out, that for late boot unlocking, you need to enable the clevis-luks-askpass.path as mentioned before and that after the reboot, clevis will try to unlock all device listed in /etc/crypttab which have clevis bindings when sytemd prompts for the pwd.

I was not able to confirm that, but will keep working on it, till now it is seen as NOT WORKING WITH AUTO UNLOCK.

Tang did a rekey lets rekey clevis

If you are using a tang server to automatically unlock your disk for example, it might happen that a rekey was performed.

To be able to deal with that, check which slot is used for tang in the luks parition and perform a clevis luks report like this:

$ clevis luks list -d /dev/sda2
1: tang '{"url":"http://localhost:8080"}'
$ clevis luks report -d /dev/sda2 -s 1
...
Report detected that some keys were rotated.
Do you want to regenerate luks metadata with "clevis luks regen -d /dev/sda2 -s 1"? [ynYN]

If you hit there [y|Y] it will enable the new key created by the tang key rotation.

What you can do as well, is to execute this one:

$ clevis luks regen -d /dev/sda2 -s 1

Docu review done: Wed 31 Jul 2024 02:09:25 PM CEST

Comparetools

meld

grafical application to compare folders and files it can compare also in a threway as well

Docu review done: Wed 31 Jul 2024 02:09:45 PM CEST

cpufreq

General

Utilities to deal with the cpufreq Linux kernel feature

Commands

Docu review done: Wed 31 Jul 2024 02:10:06 PM CEST

csync2

Table of Content

• General
• Installation
• Setup and initialize
• Conflict handling
• Resetting a csync2 cluster

General

csync2 is a bidirectional sync tool on file base

Installation

$ apt install csync2

Setup and initialize

First thing is, that you need an tuhentication key

$ csync2 -k /etc/csync2.key

For the transfer it selfe, you need to generate a ssl cert as well

$ openssl genrsa -out /etc/csync2_ssl_key.pem 2048
$ openssl req -batch -new -key /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.csr
$ openssl x509 -req -days 3600 -in /etc/csync2_ssl_cert.csr -signkey /etc/csync2_ssl_key.pem -out /etc/csync2_ssl_cert.pem

To configure csync2 a small config file /etc/csync2/csync2.cfg is needed, where you define the hosts, keys and so on

group mycluster
{
        host node1;
        host node2;

        key /etc/csync2.key;

        include /www/htdocs;
        exclude *~ .*;
}

Transfer config and certs/key to all the other nodes which are in your csync2 cluster and ensure that the service/socket is enabled and started.

$ sytstemctl enable --now csync2.socket

After the sockes are available everywhere, you can start the inintial sync.

$ csync2 -xv

Conflict handling

If csync2 detected changes during a sync by one or more hosts, you will get messages like this

While syncing file /etc/ssh/sshd_conf:
ERROR from peer site-2-host-1: File is also marked dirty here!
Finished with 1 errors.

To resolve the conflict, connect to the hosts were you know the correct file is located at and execute the following

$ csync2 -f /etc/ssh/sshd_conf
$ csync2 -x

Resetting a csync2 cluster

Resolves the following errors:

• ERROR from peer 10.0.0.1: File is also marked dirty here!
• Database backend is exceedingly busy => Terminating (requesting retry).
• ERROR from peer 10.0.0.1: Connection closed.

First of all, make sure that csync2 process is running. Then connect to all servers, and update the csync2 database.

$ csync2 -cIr /

IMPORTANT only perform the next steps on the master

Go to the server with the new files or corrected state (may be master =) Get all differences between master and slaves and mark for sync

$ csync2 -TUXI

Reset database on master to winn all conflicts and sync data to all slaves

$ csync2 -fr /
$ csync2 -xr /

Connect now again to all other servers too and run to check the sync state

$ csync2 -T

Docu review done: Wed 31 Jul 2024 02:11:55 PM CEST

darktable

Table of content

• Darktable CLI

Darktable CLI

With the installation of darktable also the darktable-cli gets installed. The darktable-cli offers a basic interaction with raw files, for example the convertion from .raw to .jpg, .png and so on

$ darktable-cli <RAWFILE> <OUTPUTFILE> --core --conf plugins/imageio/format/<FORMAT>/<SETTINGS>

For example, a conversion from .nef to .jpg using high quality resampling

$ darktable-cli DSC_0566.NEF ./JPEG/DSC_0566.jpg --core --conf plugins/imageio/format/jpeg/quality=100 --hq true

Docu review done: Wed 31 Jul 2024 02:11:47 PM CEST

dialog

General

dialog will spawn a dialog box to allows you enter some value which places the string into a file and than back into a variable

Sample

$ dialog --title "Start tracking" --backtitle "Start tracking" --inputbox "${timetotrackon}" 100 60 2>${outputfile} ;  timetotrackon=$(<${outputfile})

[Docu with samples][https://bash.cyberciti.biz/guide/The_form_dialog_for_input]

Docu review done: Wed 31 Jul 2024 02:11:38 PM CEST

Dict Leo

Table of Content

• Useage
• Sample
• Installation
• URLs

Useage

Sample

$ leo nerd
Found 7 matches for 'nerd' on dict.leo.org:

Nouns
 nerd die Langweilerin
 nerd der Computerfreak
 nerd die Fachidiotin
 nerd der Schwachkopf
 nerd hochintelligente, aber kontaktarme Person
 nerd der Sonderling
 nerd die Streberin

$ leo -l es wilkommen
Found 11 matches for 'willkommen' on dict.leo.org:

Nouns
 la bienvenida                    der Willkommen

Adjectives/Adverbs
 bienvenida                       willkommen

Phrases/Collocations
 ¡Bienvenida!                     Herzlich Willkommen!
 ¡Bienvenido!                     Herzlich Willkommen!
 ¡Bienvenidas!                    Herzlich Willkommen!
 ¡Bienvenidos!                    Herzlich Willkommen!
 ¡Bienvenidos a bordo!            Willkommen an Bord!
 ¡Bienvenidos a Alemania!         Willkommen in Deutschland!
 ¡Bienvenido a casa!              Willkommen zu Hause!

Verbs
 dar la bienvenida a alguien      jmdn. willkommen heißen
 recibir a alguien                jmdn. willkommen heißen

# same as above but specify the direction of translation
$ leo -l de2es willkommen

Installation

$ apt install libwww-dict-leo-org-perl

URLs

dict.leo.org

Docu review done: Wed 31 Jul 2024 02:11:33 PM CEST

docker

General

docer interact with docker containers,nodes,…

Troubleshooting

$ docker commit <DockerID|Docername> <debugimagename>              # creates new docker image from broken container

Debugging docker containers

$ docker commit apacheserver debug/apache              # creates new docker image from broken container
$ docker run -it --entrypoint=/bin/sh debug/apache     # starts only docker container root process
$ docker stats                                         # like a top for docker processes

Docu review done: Wed 31 Jul 2024 02:12:11 PM CEST

easytag

General

easytag is an standard audio tag manipulation software with standard quality.

cons: It very often detects changes as it tried to automatically update tags based on matches on the internet

Docu review done: Wed 31 Jul 2024 02:12:42 PM CEST

exiftool

Table of content

• Table of content
• Installation
• Commands
• Samples
• URL

General

exiftool allows you to modify the metadata from picutes

Installation

This application can be installed with apt

$ apt install libimage-exiftool-perl

Commands

Samples

List metadata about files

$ exiftool --list IMG_20200329_002001.jpg
ExifTool Version Number         : 12.00
File Name                       : IMG_20200329_002001.jpg
Directory                       : .
File Size                       : 5.8 MB
File Modification Date/Time     : 2020:03:29 00:30:16+01:00
File Access Date/Time           : 2020:06:30 11:06:24+02:00
File Inode Change Date/Time     : 2020:06:30 11:05:32+02:00
File Permissions                : rw-r--r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Image Width                     : 5472
Image Height                    : 7296
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Image Size                      : 5472x7296
Megapixels                      : 39.9

Removes all tabgs from all files found by *.jpg including subdirs

$ exiftool -r -overwrite_original -P -all= *.jpg

URL

Examples

Docu review done: Mon 03 Jul 2023 16:34:12 CEST

fdupes

Table of Content

• General
• Install
• Parameters
• Examples

General

fdupes finds duplicate files in a given set of directories Searches the given path for duplicate files. Such files are found by comparing file sizes and MD5 signatures, followed by a byte-by-byte comparison.

Install

$ apt install fdupes

Parameters

Examples

# the quer files differes between test1 and test2 dir
$ md5sum test*/*
b026324c6904b2a9cb4b88d6d61c81d1  test1/1
31d30eea8d0968d6458e0ad0027c9f80  test1/10
26ab0db90d72e28ad0ba1e22ee510510  test1/2
6d7fce9fee471194aa8b5b6e47267f03  test1/3
e760668b6273d38c832c153fde5725da  test1/4
1dcca23355272056f04fe8bf20edfce0  test1/5
9ae0ea9e3c9c6e1b9b6252c8395efdc1  test1/6
84bc3da1b3e33a18e8d5e1bdd7a18d7a  test1/7
c30f7472766d25af1dc80b3ffc9a58c7  test1/8
9e6b1b425e8c68d99517d849d020c8b7  test1/9
d8016131a2724252b2419bf645aab221  test1/qwer
b026324c6904b2a9cb4b88d6d61c81d1  test2/1
31d30eea8d0968d6458e0ad0027c9f80  test2/10
26ab0db90d72e28ad0ba1e22ee510510  test2/2
6d7fce9fee471194aa8b5b6e47267f03  test2/3
e760668b6273d38c832c153fde5725da  test2/4
1dcca23355272056f04fe8bf20edfce0  test2/5
9ae0ea9e3c9c6e1b9b6252c8395efdc1  test2/6
84bc3da1b3e33a18e8d5e1bdd7a18d7a  test2/7
c30f7472766d25af1dc80b3ffc9a58c7  test2/8
9e6b1b425e8c68d99517d849d020c8b7  test2/9
2b00042f7481c7b056c4b410d28f33cf  test2/qwer

$ fdupes -ri1 ./test1 ./test2
test2/7 test1/7
test2/9 test1/9
test2/3 test1/3
test2/1 test1/1
test2/2 test1/2
test2/4 test1/4
test2/10 test1/10
test2/6 test1/6
test2/8 test1/8
test2/5 test1/5

ffmpeg

general notes on ffmpeg

loops

ffmpeg consumes stdin and acts weird if you loop for example over a list of files provided by stdin.

solution:

while read -r file; do
    cat /dev/null | ffmpeg -i "${file}" "${file}.new"
done <./toEncode.lst

Convert commands with ffmpeg

Amazone offers the files in default as m4a files

$ ffmpeg -i Weihnachtsklingel-6G8_sdQ26MY.m4a -vn -ab 320k -ar 44100 Weihnachtsklingel.mp3

Converts all flac files to mp3 and keeps metat data (cover art not working)

$ for f in $(ls -1 *.flac) ; do ffmpeg -i "${f}" -vn -map_metadata 0 -id3v2_version 3 -ab 320k -ar 44100 "$(sed -E 's/\.flac/mp3/g' <<<"${f}")" ; done

Converts all mp3 files to flac

$ while read line ; do yes | ffmpeg -i "$line" "$(echo "$line" | sed -e 's/mp3/flac/g')" -vsync 2 ; done<<<$(find ./ -name "*mp3")

Converts $file into avi thats working on bones Kid Car TVs

$ ffmpeg -i "${file}" -vf scale=720:-1 -b:v 1.2M -vcodec mpeg4 -acodec mp3 "${destination:-.}/${file/%${file##*\.}/avi}"

Changing video rotation or flip it

flip video vertically

$ ffmpeg -i inputfile -vf vflip -c:a copy outputfile

flip video horizontally

$ ffmpeg -i inputfile -vf hflip -c:a copy outputfile

rotate 90 degrees clockwise

$ ffmpeg -i inputfile -vf transpose=1 -c:a copy outputfile

rotate 90 degrees counterclockwise

$ ffmpeg -i inputfile -vf transpose=2 -c:a copy outputfile

Docu review done: Wed 31 Jul 2024 02:13:10 PM CEST

fio

Table of content

• Installation
• Testing IOPS
  • RW performance
  • Random read performance
  • Random write performance
  • Benchmark disk sample
  • Benchmark test db storage

Installation

$ apt install fio

Testing IOPS

RW performance

The first test is for measuring random read/write performances. In a terminal, execute the following command:

$ fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

During the test, the terminal window will display an output like the following one:

test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 4096MB)
Jobs: 1 (f=1): [m(1)] [0.1% done] [447KB/131KB/0KB /s] [111/32/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [383KB/147KB/0KB /s] [95/36/0 iops] [eta 01h:4Jobs: 1 (f=1): [m(1)] [0.1% done] [456KB/184KB/0KB /s] [114/46/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [624KB/188KB/0KB /s] [156/47/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [443KB/115KB/0KB /s] [110/28/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [515KB/95KB/0KB /s] [128/23/0 iops] [eta 01h:4Jobs: 1 (f=1): [m(1)] [0.1% done] [475KB/163KB/0KB /s] [118/40/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.2% done] [451KB/127KB/0KB /s] [112/31/0 iops]

So, the program will create a 4GB file (--size=4G), and perform 4KB reads and writes using three reads for every write ratio (75%/25%, as specified with option --rwmixread=75), split within the file, with 64 operations running at a time. The RW ratio can be adjusted for simulating various usage scenarios. At the end, it will display the final results:

test: (groupid=0, jobs=1): err= 0: pid=4760: Thu Mar  2 13:23:28 2017
  read : io=7884.0KB, bw=864925B/s, iops=211, runt=  9334msec
  write: io=2356.0KB, bw=258468B/s, iops=63, runt=  9334msec
  cpu          : usr=0.46%, sys=2.35%, ctx=2289, majf=0, minf=29
  IO depths    : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=0.6%, 32=1.2%, >=64=97.5%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=1971/w=589/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=7884KB, aggrb=844KB/s, minb=844KB/s, maxb=844KB/s, mint=9334msec, maxt=9334msec
  WRITE: io=2356KB, aggrb=252KB/s, minb=252KB/s, maxb=252KB/s, mint=9334msec, maxt=9334msec

Disk stats (read/write):
    dm-2: ios=1971/589, merge=0/0, ticks=454568/120101, in_queue=581406, util=98.44%, aggrios=1788/574, aggrmerge=182/15, aggrticks=425947/119120, aggrin_queue=545252, aggrutil=98.48%
  sda: ios=1788/574, merge=182/15, ticks=425947/119120, in_queue=545252, util=98.48%

Note from the author: I ran fio on my laptop, so the last output was obtained running the test with a 10MB file; as can be seen above, the 4GB option would have taken more than 1 hour.

Random read performance

In this case, the command is:

$ fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read.fio --bs=4k --iodepth=64 --size=4G --readwrite=randread

The output will be similar to the RW case, just specialized in the read case.

Random write performance

$ fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randwrite

As above, in random write case.

Write test sample

$ fio --time_based --filename=./fio-bench-file --direct=1 --rw=write --refill_buffers --ioengine=libaio --bs=128k --size=4G --filesize=4G --iodepth=16  --numjobs=24  --runtime=600 --group_reporting --name=fio_secwrite

Benchmark disk sample

#!/bin/bash
#
# - Required Debian packages: bonnie++ fio libaio1
#

CPUS=$(cat /proc/cpuinfo | grep processor | wc -l)
RAM=$(free -m | grep 'Mem:' | perl -ne '/Mem:\s+(\d+)/; print $1')
BLOCK_COUNT=$(echo "scale=0;(${RAM} * 1024) * 4.25 / 8"|bc -l) # (RAM = in MB, so * 1024) * 4.25 / 8k blocksize
RAM_BYTES=$(echo "scale=0; $RAM * 1024 * 1024" | bc -l)
FIO_BENCH_SIZE=$(echo "scale=0; $RAM_BYTES * 4.25" | bc -l)
FIO_CPUS=$(echo "scale=0; $CPUS * 2" | bc -l)

logfile="bench_disk_$$.log"

openlog() {
    exec 6>&1
    exec 7>&2
    exec >> $logfile 2>&1
}
closelog() {
    # Close the file descriptors
    exec 1>&6 6>&-
    exec 2>&7 7>&-
}

# disable oom killer for this process
echo -17 > /proc/$$/oom_adj

openlog
echo "Number of CPUs: $CPUS"
echo "Available RAM : $RAM"
echo "Available RAM : $RAM_BYTES bytes"
echo "Block Size    : $BLOCK_COUNT"
closelog

TEST_DIR=${1:-/data/postgres/testdir}

BENCH1="bench01"
BENCH2="bench02"
BENCH3="bench03"
BENCH4="bench04"
BENCH5="bench05"

openlog
echo "Test directory: $TEST_DIR"
closelog

mkdir -p "${TEST_DIR}"
mkdir -p "logs/$date"
cd       "logs/$date"

# Test 0: fio benchtest
echo "test 0: FIO benchtests"
openlog
echo "test 0.1: FIO seq write benchtest"
date
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=write --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --refill_buffers --end_fsync=1 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_seqwrite
echo "test 0.2: FIO random write benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=randwrite --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_randomwrite
echo "test 0.3: FIO seq read benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=read --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_seqread
echo "test 0.4: FIO random read benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=randread --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_randomread
echo "test 0.5: FIO random read/write benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --rwmixread=60 --iodepth=16 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_randomreadwrite
date
closelog

echo "finished bench_disk"

Benchmark test db storage

#!/bin/bash
#
# - Required Debian packages: bonnie++ fio libaio1
# - Postgresql must be installed, tuned by Ansible playbook and started
#

running_user=$(id | egrep -E -o "uid=[0-9]+\([a-z]+\)" | cut -d'(' -f2 | cut -d ')' -f1)
if [ $running_user != "postgres" ]; then
    echo "You must run this script as 'postgres' user"
    exit 1
fi

CPUS=$(cat /proc/cpuinfo | grep processor | wc -l)
RAM=$(free -m | grep 'Mem:' | perl -ne '/Mem:\s+(\d+)/; print $1')
BLOCK_COUNT=$(echo "scale=0;(${RAM} * 1024) * 4.25 / 8"|bc -l) # (RAM = in MB, so * 1024) * 4.25 / 8k blocksize
RAM_BYTES=$(echo "scale=0; $RAM * 1024 * 1024" | bc -l)
FIO_BENCH_SIZE=$(echo "scale=0; $RAM_BYTES * 4.25" | bc -l)
FIO_CPUS=$(echo "scale=0; $CPUS * 2" | bc -l)

logfile="bench_db_$$.log"

openlog() {
    exec 6>&1
    exec 7>&2
    exec >> $logfile 2>&1
}
closelog() {
    # Close the file descriptors
    exec 1>&6 6>&-
    exec 2>&7 7>&-
}

# disable oom killer for this process
echo -17 > /proc/$$/oom_adj

openlog
echo "Number of CPUs: $CPUS"
echo "Available RAM : $RAM"
echo "Available RAM : $RAM_BYTES bytes"
echo "Block Size    : $BLOCK_COUNT"
closelog

DB_DIR=${1:-/data/postgres}
TEST_DIR=${1:-/data/postgres/testdir}
PG_DIR="/opt/postgresql"
pgbench="$PG_DIR/bin/pgbench "
createdb="$PG_DIR/bin/createdb"
dropdb="$PG_DIR/bin/dropdb"
initdb="$PG_DIR/bin/initdb"

BENCH1="bench01"
BENCH2="bench02"
BENCH3="bench03"
BENCH4="bench04"
BENCH5="bench05"

postgres_uid=$(id postgres|sed -e 's/uid=\([0-9]*\).*/\1/')
postgres_gid=$(id postgres|sed -e 's/.*gid=\([0-9]*\).*/\1/')

openlog
echo "Test directory: $DB_DIR"
echo "Postgres UID  : $postgres_uid"
echo "Postgres GID  : $postgres_gid"
closelog

mkdir -p "${TEST_DIR}"
mkdir -p "logs/$date"
cd       "logs/$date"

# Test 0: fio benchtest
echo "test 0: FIO benchtests"
openlog
echo "test 0.1: FIO seq write benchtest"
date
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=write --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --refill_buffers --end_fsync=1 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_seqwrite
echo "test 0.2: FIO random write benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=randwrite --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_randomwrite
echo "test 0.3: FIO seq read benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=read --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_seqread
echo "test 0.4: FIO random read benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=randread --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --filesize=$FIO_BENCH_SIZE --iodepth=256 --sync=0 --randrepeat=0 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_randomread
echo "test 0.5: FIO random read/write benchtest"
fio --time_based --filename=$TEST_DIR/fio-bench-file2 --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=8k --size=$FIO_BENCH_SIZE --rwmixread=60 --iodepth=16 --numjobs=$FIO_CPUS --runtime=600 --group_reporting --name=fio_randomreadwrite
date
closelog

#echo "test 1: dd write tests"
# Test 1.: dd write test
#openlog
#echo "test 1.1: dd write test oflag=direct"
#date
#time bash -c "dd if=/dev/zero of=${TEST_DIR}/dd_benchmark_file bs=8k count=$BLOCK_COUNT conv=fdatasync oflag=direct && sync" 2>&1
#date
#echo "test 1.2: dd write test oflag=dsync"
#time bash -c "dd if=/dev/zero of=${TEST_DIR}/dd_benchmark_file bs=8k count=$BLOCK_COUNT conv=fdatasync oflag=dsync && sync" 2>&1
#date
#echo "test 1.3: dd write test no oflag"
#time bash -c "dd if=/dev/zero of=${TEST_DIR}/dd_benchmark_file bs=8k count=$BLOCK_COUNT conv=fdatasync && sync" 2>&1
#date
#echo
#closelog

#echo "test 2: dd read test"
# Test 2.: dd read test
# Redirect output
#openlog
#echo "test 2: dd read test"
#date
#time dd if=${TEST_DIR}/dd_benchmark_file of=/dev/null bs=8k 2>&1
#date
#rm ${TEST_DIR}/dd_benchmark_file
#closelog

# Test 3: bonnie++
#echo "test 3: bonnie test"
#openlog
#echo "test 3: bonnie test"
#date
#/usr/sbin/bonnie++ -n 0 -u ${postgres_uid}:${postgres_gid} -r $(free -m | grep 'Mem:' | awk '{print $2}') -s $(echo "scale=0;`free -m | grep 'Mem:' | awk '{print $2}'`*4.25" | bc -l) -f -b -d ${TEST_DIR}
#date
#closelog

# Test 4: pgbench buffer test
echo "test 4: pgbench buffer test"
openlog
echo "test 4: pgbench buffer test"
$dropdb $BENCH1
$createdb $BENCH1
$pgbench -i -s 15 $BENCH1
date
$pgbench -c 24 -j 12 -T 600 $BENCH1
date
closelog

# Test 5: pgbench mostly cache test
echo "test 5: pgbench mostly cache test"
openlog
echo "test 5: pgbench mostly cache test"
$dropdb $BENCH2
$createdb $BENCH2
$pgbench -i -s 70 $BENCH2
date
$pgbench -c 24 -j 12 -T 600 $BENCH2
date
closelog

# Test 6: pgbench on-disk test
echo "test 6: pgbench on-disk test"
openlog
echo "test 6: pgbench on-disk test"
$dropdb $BENCH3
$createdb $BENCH3
$pgbench -i -s 600 $BENCH3
date
$pgbench -c 24 -j 12 -T 600 $BENCH3
date
closelog

# Test 7: pgbench Read-Only Test
echo "test 7: pgbench read-only test"
openlog
echo "test 7: pgbench read-only test"
date
$pgbench -c 24 -j 12 -T 600 -S $BENCH2
date
closelog

# Test 8: pgbench simple write test
echo "test 8: pgbench simple write test"
openlog
echo "test 8: pgbench simple write test"
date
$pgbench -c 24 -j 12 -T 600 -N $BENCH2
date
closelog

# Test 9: pgbench prepared read-write
echo "test 9: pgbench prepared read-write"
openlog
echo "test 9: pgbench prepared read-write"
date
$pgbench -c 24 -j 12 -T 600 -M prepared $BENCH2
date
closelog

# Test 10: pgbench prepared read-only
echo "test 10: pgbench prepared read-only"
openlog
echo "test 10: pgbench prepared read-only"
date
$pgbench -c 24 -j 12 -T 600 -M prepared -S $BENCH2
date
closelog

# Test 11: connection test
echo "test 11: pgbench connection test"
openlog
echo "test 11: pgbench connection test"
echo " - fill up database (+-73GB)"
$dropdb $BENCH4
$createdb $BENCH4
$pgbench -i -s 5000 $BENCH4
echo " - fill up filesystem caches"
tar cvf - ${DB_DIR} > /dev/null
echo " - warmup postgres cache"
$pgbench -j 6 -c 6 -T 1800 -S $BENCH4
#for clients in 1 5 10 20 30 40 50 60 80 100 150 200 250 300 350 400 450 500
for clients in 1 5 10 20 30 40 50 60 80 100 150 200
do
    THREADS=${clients}
    if [ $clients > $CPUS ]
    then
        THREADS=10
    else
        THREADS=${clients}
    fi
    echo " -- Number of Clients: ${clients} | THREADS: ${THREADS}"
    $pgbench -j ${THREADS} -c ${clients} -T 180 -S $BENCH4
done
closelog

# cleanup

#pg_ctl -D ${DB_DIR} -m fast stop
#rm -fr ${DB_DIR}

Docu review done: Wed 31 Jul 2024 02:14:23 PM CEST

firejail

Table of content

• Description
  • Security Profiles
  • Filesystem
• Installation
• Commands
• Network
  • Networkfilter Default
  • Networkfilter Specific
  • Network stats
• Samples
  • Firefox

Description

Firejail is a SUID sandbox program that reduces the risk of security breaches by restricting the running environment of untrusted applications using Linux namespaces, seccomp-bpf and Linux capabilities. It allows a process and all its descendants to have their own private view of the globally shared kernel resources, such as the network stack, process table, mount table. Firejail can work in a SELinux or AppArmor environment, and it is integrated with Linux Control Groups.

Firejail allows the user to manage application security using security profiles. Each profile defines a set of permissions for a specific application or group of applications. The software includes security profiles for a number of more common Linux programs, such as Mozilla Firefox, Chromium, VLC, Transmission etc.

Security Profiles

Several command line options can be passed to the program using profile files. Firejail chooses the profile file as follows:

1. If a profile file is provided by the user with --profile=FILE option, the profile FILE is loaded. If a profile name is given, it is searched for first in the ~/.config/firejail directory and if not found then in /etc/firejail directory. Profile names do not include the .profile suffix. If there is a file with the same name as the given profile name, it will be used instead of doing the profile search. To force a profile search, prefix the profile name with a colon (:), eg. --profile=:PROFILE_NAME. Example:

$ firejail --profile=/home/netblue/icecat.profile icecat
Reading profile /home/netblue/icecat.profile
[...]

$ firejail --profile=icecat icecat-wrapper.sh
Reading profile /etc/firejail/icecat.profile
[...]

2. If a profile file with the same name as the application is present in ~/.config/firejail directory or in /etc/firejail, the profile is loaded. ~/.config/firejail takes precedence over /etc/firejail. Example:

$ firejail icecat
Command name #icecat#
Found icecat profile in /home/netblue/.config/firejail directory
Reading profile /home/netblue/.config/firejail/icecat.profile
[...]

3. Use default.profile file if the sandbox is started by a regular user, or server.profile file if the sandbox is started by root. Firejail looks for these files in ~/.config/firejail directory, followed by /etc/firejail directory. To disable default profile loading, use –noprofile command option. Example:

$ firejail
Reading profile /etc/firejail/default.profile
Parent pid 8553, child pid 8554
Child process initialized
[...]

$ firejail --noprofile
Parent pid 8553, child pid 8554
Child process initialized
[...]

Filesystem

Insatllation

$ apt install firejail

Commands

Network

Networkfilter Default

The default firewall is optimized for regular desktop applications. No incoming connections are accepted:

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# allow ping
-A INPUT -p icmp --icmp-type destination-unreachable -j ACCEPT
-A INPUT -p icmp --icmp-type time-exceeded -j ACCEPT
-A INPUT -p icmp --icmp-type echo-request -j ACCEPT
# drop STUN (WebRTC) requests
-A OUTPUT -p udp --dport 3478 -j DROP
-A OUTPUT -p udp --dport 3479 -j DROP
-A OUTPUT -p tcp --dport 3478 -j DROP
-A OUTPUT -p tcp --dport 3479 -j DROP
COMMIT

Networkfilter Specific

Please use the regular iptables-save/iptables-restore format for the filter file. The following examples are available in /etc/firejail directory.

webserver.net is a webserver firewall that allows access only to TCP ports 80 and 443. Example:

$ firejail --netfilter=/etc/firejail/webserver.net --net=eth0 /etc/init.d/apache2 start

nolocal.net is a desktop client firewall that disable access to local network. Example:

$ firejail --netfilter=/etc/firejail/nolocal.net --net=eth0 firefox

Network stats

Monitor network namespace statistics, see MONITORING section for more details.

$ firejail --netstats
PID  User    RX(KB/s) TX(KB/s) Command
1294 netblue 53.355   1.473    firejail --net=eth0 firefox
7383 netblue 9.045    0.112    firejail --net=eth0 transmission

Samples

Firefox

by default, a single Firefox process instance handles multiple browser windows. If you already have Firefox running, you would need to use -no-remote command line option, otherwise you end up with a new tab or a new window attached to the existing Firefox process

$ firejail firefox -no-remote

To assign an IP address, Firejail ARP-scans the network and picks up a random address not already in use. Of course, we can be as explicit as we need to be:

$ firejail --net=eth0 --ip=192.168.1.207 firefox

Note: Ubuntu runs a local DNS server in the host network namespace. The server is not visible inside the sandbox. Use --dns option to configure an external DNS server:

$ firejail --net=eth0 --dns=9.9.9.9 firefox

By default, if a network namespace is requested, Firejail installs a network filter customized for regular Internet browsing. It is a regular iptable filter. This is a setup example, where no access to the local network is allowed:

$ firejail --net=eth0 --netfilter=/etc/firejail/nolocal.net firefox

On top of that, you can even add a hosts file implementing an adblocker:

$ firejail --net=eth0 --netfilter=/etc/firejail/nolocal.net --hosts-file=~/adblock firefox

In this setup we use /home/username/work directory for work, email and related Internet browsing. This is how we start all up:

$ firejail --private=/home/username/work thunderbird &
$ firejail --private=/home/username/work firefox -no-remote &

Both Mozilla Thunderbird and Firefox think ~/work is the user home directory. The configuration is preserved when the sandbox is closed.

firewalld

Table of Content

• Installation
• Default Zones
• Commands and Descriptions
• Adding firewalld zone to NetworkManager connection
• Behind the sceens

Installation

On Debian (and Debian based systems) you can install it with apt/apt-get/nala/…

$ apt install firewalld

Default Zones

Commands and Descriptions

If you don’t specify --zone=[zone_name] the default zone will be used (which is per default public)

Adding firewalld zone to NetworkManager connection

Why would you add a zone from firewalld to a NetworkManager connection. This makes a lot of sense if you are for example doing that on clients where they switch between connections (e.g. a laptop)

First we have to check if there is alrey a zone defined for the connection where we want to apply a zone to.

If it looks like the below sample, it will use the default zone as no specific one got assigned.

$ nmcli -p connection show [connection_name] | grep connection.zone
connection.zone:                        --

Have a look at the zone which are provided by firewalld and decide which one is best fitting.

If you want to create a new one, e.g. based on the zone public, just copy the file /usr/lib/filewalld/zones/public.xml to /etc/firewalld/zones with a new name and reload firewalld. After you are done modifying it (dont forget to use --permanent or --runtime-to-permanent) you can add the zone to the connection like so:

$ nmcli connection modify [connection_name] connection.zone [zone_name]

And if you rerun the command from above (nmcli -p connection show...) you will get this:

$ nmcli -p connection show [connection_name] | grep connection.zone
connection.zone:                        [zone_name]

Behind the sceens

To see what is going on in nftables you can use the commands:

$ nft list ruleset inet

Of course you could change in the config that it should uses iptables by adding FirewallBackend and set it to iptables, but keep in mind, iptables is deprecated and will be removed in future releases.

If you still use it, with iptables -nvL you could see the applied rules.

Docu review done: Wed 31 Jul 2024 02:15:20 PM CEST

Fun with Linux

Table of content

• Cowsay
• sl
• figlet or toilet
• cmatix
• rev
• apt moo
• aptitude moo
• Starwars in telnet
• factor
• pi
• xcowsay
• xeyes
• rig
• aafire
• lolcat
• asciiviewe

cowsay

cowsay is a talking cow that will speak out anything you want it to

Installation

$ apt install cowsay

Usage

$ cowsay "Mooooooooo"
 ____________
< Mooooooooo >
 ------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Additional themes

$ cowsay -l
Cow files in /usr/share/cowsay/cows:
apt beavis.zen bong bud-frogs bunny calvin cheese cock cower daemon default
dragon dragon-and-cow duck elephant elephant-in-snake eyes flaming-sheep
ghostbusters gnu head-in hellokitty kiss kitty koala kosh luke-koala
mech-and-cow meow milk moofasa moose mutilated pony pony-smaller ren sheep
skeleton snowman sodomized-sheep stegosaurus stimpy suse three-eyes turkey
turtle tux unipony unipony-smaller vader vader-koala www

Useage different theme

$ cowsay -f ghostbusters "Who you Gonna Call"
 ____________________
< Who you Gonna Call >
 --------------------
          \
           \
            \          __---__
                    _-       /--______
               __--( /     \ )XXXXXXXXXXX\v.
             .-XXX(   O   O  )XXXXXXXXXXXXXXX-
            /XXX(       U     )        XXXXXXX\
          /XXXXX(              )--_  XXXXXXXXXXX\
         /XXXXX/ (      O     )   XXXXXX   \XXXXX\
         XXXXX/   /            XXXXXX   \__ \XXXXX
         XXXXXX__/          XXXXXX         \__---->
 ---___  XXX__/          XXXXXX      \__         /
   \-  --__/   ___/\  XXXXXX            /  ___--/=
    \-\    ___/    XXXXXX              '--- XXXXXX
       \-\/XXX\ XXXXXX                      /XXXXX
         \XXXXXXXXX   \                    /XXXXX/
          \XXXXXX      >                 _/XXXXX/
            \XXXXX--__/              __-- XXXX/
             -XXXXXXXX---------------  XXXXXX-
                \XXXXXXXXXXXXXXXXXXXXXXXXXX/
                  ""VXXXXXXXXXXXXXXXXXXV""

Alternative, let the cow think

It comes also by installing cowsay

$ cowthink "Mooooooo?"
 ___________
( Mooooooo? )
 -----------
        o   ^__^
         o  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

sl

With the sl command, a steam locomotive will run across your terminal from right to left.

Installation

$ apt install sl

Usage

$ sl
                       (@@) (  ) (@)  ( )  @@    ()    @     O     @
                     (   )
                 (@@@@)
              (    )
            (@@@)
         ====        ________                ___________
     _D _|  |_______/        \__I_I_____===__|_________|
      |(_)---  |   H\________/ |   |        =|___ ___|      ________________
      /     |  |   H  |  |     |   |         ||_| |_||     _|
     |      |  |   H  |__--------------------| [___] |   =|
     | ________|___H__/__|_____/[][]~\_______|       |   -|
     |/ |   |-----------I_____I [][] []  D   |=======|____|_________________
   __/ =| o |=-O=====O=====O=====O \ ____Y___________|__|___________________
    |/-=|___|=    ||    ||    ||    |_____/~\___/          |_D__D__D_|  |_D_
     \_/      \__/  \__/  \__/  \__/      \_/               \_/   \_/    \_/

figlet or toilet

Can be used to draw large sized text banners

Installation

$ apt install figlet
$ apt install toilet

Usage

$ figlet LINUX

 m      mmmmm  mm   m m    m m    m
 #        #    #"m  # #    #  #  #
 #        #    # #m # #    #   ##
 #        #    #  # # #    #  m""m
 #mmmmm mm#mm  #   ## "mmmm" m"  "m

$ toilet LINUX

 m      mmmmm  mm   m m    m m    m
 #        #    #"m  # #    #  #  #
 #        #    # #m # #    #   ##
 #        #    #  # # #    #  m""m
 #mmmmm mm#mm  #   ## "mmmm" m"  "m

cmatix

You should know whta it does, if not read again the name

Installation

$ apt install cmatrix

rev

The rev command will print the reverse of whatever you type in. First run rev, then start typing one sentence at a time

Installation

$ apt install util-linux

Usage

$ rev
This is reversed
desrever si sihT

apt moo

The apt-get command has this easter egg where the cow does a moo

Usage

$ apt-get moo
         (__)
         (oo)
   /------\/
  / |    ||
 *  /\---/\
    ~~   ~~
...."Have you mooed today?"...

aptitude moo

The aptitude command moos a bit reluctantly and here is how to make it do s

Usage

$ aptitude moo
There are no Easter Eggs in this program.
$ aptitude -v moo
There really are no Easter Eggs in this program.
$ aptitude -vv moo
Didn't I already tell you that there are no Easter Eggs in this program?
enlightened@enlightened:~$ aptitude -vvv moo
Stop it!
$ aptitude -vvvv moo
Okay, okay, if I give you an Easter Egg, will you go away?
$ aptitude -vvvvv moo
All right, you win.

                               /----\
                       -------/      \
                      /               \
                     /                |
   -----------------/                  --------\
   ----------------------------------------------
$ aptitude -vvvvvv moo
What is it?  It's an elephant being eaten by a snake, of course.

Starwars in telnet

This is not actually a command, but a text animation broadcasted at towel.blinkenlights.nl and can be played inside the terminal by telnetting to the server.

Usage

$ telnet towel.blinkenlights.nl

factor

This command would print out all the lowest common multiple (LCM) factors of any given number.

Instalation

$ apt install coreutils

Usage

$ factor 1337
1337: 7 191

pi

The pi command prints the mathematical constant PI to any number of decimal figures. So lets print it to the first 500 figures after decimal.

Installation

$ apt install pi

Usage

$ pi 50
3.141592653589793238462643383279502884197169399375

xcowsay

xcowsay is the gui version of the cowsay command, and you need a running desktop (X display) to use it. It cannot work solely from a terminal.

Installation

$ apt install xcowsay

Usage

$ xcowsay test

xeyes

xeyes is also a gui program that draws a pair of eyes on the desktop which follow the mouse cursor. The eyes would look where ever the mouse cursor goes.

Installation

$ apt install x11-apps

Usage

$ xeyes

rig

The rig command generates random and possibly fake identities.

Installation

$ apt install rig

Usage

$ rig
Solomon Ortiz
557 East Parson St
Alton, IL  62002
(708) xxx-xxxx

aafire

The next command is aafire and it too is an asciiart animation that renders a burning fire on the terminal.

Installation

$ apt install libaa-bin

Usage

$ aafire

lolcat

lolcat concatenates files like the UNIX cat program, but colors it for the lulz in a rainbow animation. Terminals with 256 colors and animations are supported.

Installation

$ apt install lolcat

Usage

$ apt show lolcat | lolcat
# or
$ ls -la | lolcat

asciiviewer

Since the terminal is limited to only text, tools like asciiviewer are often useful to generate images out of pure text. And to do this you need the tool called asciiviewer

Installation

$ apt install aview

Usage

$ asciiview ./mypicture.png -driver curses
                                          _ _w_,
                                    .,vaauXua%_)q_,
                                  __auZZ#Z#Zwd####oa_,
                                .jaZXZZZZZZ#Z##ZZ#ZZ#haaw_
                             ._{uZZXZZ#Z#Z#Z##Z#ZZ#ZXXX##mzp
                            _qdZXZXZZZZ#Z#Z##Z#Z#ZZZXX#mmmmWa/
                           _sXZXXZZZ#Z#2"@^`     ~"?S#mmWmWWWmp
                          guXZXXZZZZ#Y^              ^VmWmWBWmWc
                         ddXXXXZZZX!^^                 "$WWWBWWWc
                        ,dXXXZZZZe                      )VWmWWmWBc
                       _vXXXXZZZ('                        3WWBWBWm,
                    _, ]dXXXZZ"                           )VWBWmWBk
                    vf,nXXXZ?`                             )$mWmW#Wp
                     ]uZXZZ!                                )WmWk"$f
                     QXXZZ(                 ___g__          )3Wmma)f
                  _v]dXZZ(               .)aWmW#VAwc,        ]BWmm)(
                  jrsXZZZ(              .sdmT""   ^?s/       ]WmBDf{
                   <]ZZZZ              .smP`         "       jdWmQ`jf
                   i#Z#Z'             .vm!                    3mBmf
                   vZZZr              gyZ                     3WmBg
                   v#Z#[             .j#'              /      3mmWZ
                   vZ#Zf             ]mk              )v      3mW#[
                   v#Z#f             iW[               ]     jjWmEf
                   v#Z#'             iBL                     j3mm1{
                   v#ZZ              iWm             gi`     ]mmmiI
                   v##k              ]WWf           j)u     _nmmf<7
                   3Z#k              )$mk            {`     jmm#'g`
                   v#ZX              jJWQ/     )uwu        pdm#C.I
                   I#Z#/            .w4dmm/      ~`       JwmmL j'
                   4#ZZ[            )3g/4mmg           ._sm##1`
                   jZZZ[             )1{34#mp/        ,nwmm#?
                   )3XXo              )"{n3$Whw,_w__nammm#!`
                    3Xm#/                "SuVWmWWBWmBmm#!
                    ]mZmc_                 "VaI$??YY?!^
                    )3mBWmc                   ""<ivF
                     <WmWmWf
                     )$mWWm1
                      )WBWmc
                       3WWWWf
                       j$WmWa/
                        <WWBm,
                         ]BWWL
                          ]WmWp,
                           ?mWmh,
                            ?mWmc,
                             ]$Wmmp
                              )$#mBa,
                                )Wmmc//
                                 )?mmmwc
                                   )?###o,
                                      "4Xmap__,

Docu review done: Wed 31 Jul 2024 02:15:32 PM CEST

getopt

General

This script show the usage of getopt to handle arguments

und=$(tput sgr 0 1)             # Underline
bld=$(tput bold)                # Bold
err=${txtbld}$(tput setaf 1)    # red
ok=${txtbld}$(tput setaf 2)     # green
info=${txtbld}$(tput setaf 4)   # blue
bldwht=${txtbld}$(tput setaf 7) # white
rst=$(tput sgr0)                # Reset

usage() {
cat << EOF
Usage: $0 OPTIONS srcFile dstFile

OPTIONS:

--help      | -h    Display this message
--first     | -f    First argument, need a value
--second    | -s    Second argument, does not need a value (OPTIONAL)

${info}Example: $0 --first potatoe --second${rst}

EOF

exit 1
}

initII() {
    TEMP=$(getopt -s bash -o h,f:,s --longoptions help,first:,second -n 'template.sh' -- "$@")
    eval set -- "$TEMP"
    while true
    do
        case "${1}" in
            -f | --first)
                shift
                PROJECT="${1}"
                shift
                ;;
            -s | --second)
                shift
                ;;
            -h | --help)
                usage
                ;;
            --)
                shift
                break
                ;;
            *)
                echo "Incorrect parameter: $1"
                usage
                ;;
        esac
    done
}

init() {
    TEMP=`getopt -o h,f:,s --longoptions help,first:,second -n 'template.sh' -- "$@"`
    eval set -- "$TEMP"

    while true
    do
            case "$1" in
            -h | --help     )       usage;;
            -f | --first    )       FIRST="$2"; shift 2;;
            -s | --second   )       SECOND="1"; shift ;;
            --              )       shift; break;;
            *               )       echo "Incorrect parameter: $1"; usage;;
            esac
    done

    if [[ -z $FIRST ]]
    then
        echo "${err}The argument first is requierd${rst}"
        usage
    fi
}

action() {
    if [ "$SECOND" == "1" ]
    then
        options="second is set"
    else
        options="second is not set"
    fi

    echo "Performing action on ${info}${FIRST}${rst} ($options) ..."

    if [ $? == 0 ]
    then
        echo "${ok}Success.${rst}"
    else
        echo "${err}Fail.${rst}"
    fi
}

init $@
action

Docu review done: Wed 31 Jul 2024 02:16:24 PM CEST

gif

General

How to cervert videos/pictures/screenthots to gif files

Convert video to gif

uses mplayer

$ mplayer -ao null <video file name> -vo jpeg:outdir=output

Convert screenshots/files to gif

uses ImageMagick

$ convert output/* output.gif
$ convert output.gif -fuzz 10% -layers Optimize optimised.gif

Docu review done: Wed 31 Jul 2024 02:16:20 PM CEST

glances

Table of content

• Description
• Installation
• Parametes
• Standalone Mode
• Server Mode
• Web Server Mode
• Central client

Description

Glances is a cross-platform monitoring tool which aims to present a maximum of information in a minimum of space through a curses or Web based interface. It can adapt dynamically the displayed information depending on the terminal size.

It can also work in client/server mode. Remote monitoring could be done via terminal, Web interface or API (XMLRPC and RESTful).

Glances is written in Python and uses the psutil library to get information from your system.

Installation

$ apt install glances

Parametes

Standalone Mode

If you want to monitor your local machine, open a console/terminal and simply run:

$ glances

Glances should start (press ‘q’ or ‘ESC’ to exit):

It is also possible to display RAW JSON stats directly to stdout using:

$ glances --stdout cpu.user,mem.used,load
cpu.user: 30.7
mem.used: 3278204928
load: {'cpucore': 4, 'min1': 0.21, 'min5': 0.4, 'min15': 0.27}
cpu.user: 3.4
mem.used: 3275251712
load: {'cpucore': 4, 'min1': 0.19, 'min5': 0.39, 'min15': 0.27}
...

or in a CSV format thanks to the stdout-csv option:

$ glances --stdout-csv now,cpu.user,mem.used,load
now,cpu.user,mem.used,load.cpucore,load.min1,load.min5,load.min15
2018-12-08 22:04:20 CEST,7.3,5948149760,4,1.04,0.99,1.04
2018-12-08 22:04:23 CEST,5.4,5949136896,4,1.04,0.99,1.04
...

Note: It will display one line per stat per refresh.

Server Mode

While running the server mode you can specify with the parameters -B ADDRESS and -p PORT where it should listen on

$ glances -s -B 10.0.0.19 -p 10012

To limit the access you can use the parameter --password PWD and specify one. If you want, the SHA password will be stored in username.pwd file. Next time your run the server/client, password will not be asked. The defualt user name is glances which can be overwritten with the parameter --username USERNAME

Web Server Mode

If you want to remotely monitor a machine, called server, from any device with a web browser, just run the server with the -w option:

$ glances -w

then on the client enter the following URL in your favorite web browser:

http://@server:61208

where @server is the IP address or hostname of the server.

To change the refresh rate of the page, just add the period in seconds at the end of the URL. For example, to refresh the page every 10 seconds:

http://@server:61208/10

The Glances web interface follows responsive web design principles.

Central client

Glances can centralize available Glances servers using the --browser option. The server list can be statically defined via the configuration file (section [serverlist]).

Example

[serverlist]
# Define the static servers list
server_1_name=xps
server_1_alias=xps
server_1_port=61209
server_2_name=win
server_2_port=61235

Glances can also detect and display all Glances servers available on your network via the zeroconf protocol (not available on Windows):

To start the central client, use the following option:

$ glances --browser

Use –disable-autodiscover to disable the auto discovery mode.

When the list is displayed, you can navigate through the Glances servers with up/down keys. It is also possible to sort the server using: - ‘1’ is normal (do not sort) - ‘2’ is using sorting with ascending order (ONLINE > SNMP > PROTECTED > OFFLINE > UNKNOWN) - ‘3’ is using sorting with descending order (UNKNOW > OFFLINE > PROTECTED > SNMP > ONLINE)

Docu review done: Wed 31 Jul 2024 02:16:47 PM CEST

googler

General

googler allows you to perform google search directly from your terminal

Installation

$ apt install googler

samples

$ googler test

 1.  Speedtest by Ookla - The Global Broadband Speed Test
     https://www.speedtest.net/
     Test your Internet connection bandwidth to locations around the world with this
     interactive broadband speed test from Ookla.

 2.  Fast.com: Internet Speed Test
     https://fast.com/
     How fast is your download speed? In seconds, FAST.com's simple Internet speed
     test will estimate your ISP speed.
  .
  .
  .

Docu review done: Wed 31 Jul 2024 02:17:36 PM CEST

GoPro

Table of Content

• GoPro Linux
  • Installation
  • Requirements
  • GoPro Installation
  • Often used commands
    • Usage
    • Sample

GoPro Linux

Linux Bash scripts and command line interface for processing media filmed on GoPro HERO 3, 4, 5, 6, and 7 cameras Github GoPro-Linux

Installation

Requirements

• ffmpeg
• imagemagick
• mencoder

GoPro Installation

You can download the file directly from github:

$ curl https://raw.githubusercontent.com/KonradIT/gopro-linux/master/gopro -o /opt/bin/gopro
$ chmod +x /opt/bin/gopro

Often used commands

###Removing finshey

Usage

$ gopro fisheye ./file.to.apply.filter.jpg
GoPro Tool for Linux
To see a list of commands and syntax available run: gopro help
Checking dependencies...
Resolution:
-[0] 4:3 Wide FOV
-[1] 4:3 Medium FOV
-[2] 4:3 Narrow FOV
Photo resolution:

Sample

With this it will ran through all files and will use the medium filter (my personal default)

$ for f in *.JPG ; do yes 1 | gopro fisheye ${f} ; done

Docu review done: Wed 31 Jul 2024 02:17:46 PM CEST

groff

General

groff is used to create man pages

Cheatsheet

The helpful GNU troff cheatsheet along with examples. groff-cheatsheet

Docu review done: Wed 31 Jul 2024 02:18:02 PM CEST

h2db

Connect to h2db

$ find / -name "*h2*.jar"                                     # to get the h2 jar file
$ java -cp /path/h2-1.3.176.jar  org.h2.tools.Shell           # run h2 jar file
Welcome to H2 Shell 1.3.176 (2014-04-05)                    # welcome msg
Exit with Ctrl+C                                            # how to exit
[Enter]   jdbc:h2:~/test                                    # default by pressing enter
URL       jdbc:h2:/path/to/file/file.h2.db                  # specify your path
[Enter]   org.h2.Driver                                     # default by pressing enter
Driver                                                      # i pressed enter ;)
[Enter]                                                     # no user if press enter
User                                                        # no user if press enter
[Enter]   Hide                                              # no pwd if press enter
Password                                                    # no pwd if press enter
Password                                                    # no pwd if press enter
Connected                                                   # connected msg
Commands are case insensitive; SQL statements end with ';'  #
help or ?      Display this help                            #
list           Toggle result list mode                      #
maxwidth       Set maximum column width (default is 100)    # Shows help
show           List all tables                              #
describe       Describe a table                             #
quit or exit   Close the connection and exit                #
sql>                                                        # start with queries

h2db-Commands

Docu review done: Wed 31 Jul 2024 02:18:15 PM CEST

holiday

Table of Content

• Description
• Installation
• Usage
• Data
• Output Formats
• Limitations

Description

holidata is a utility for algorithmically producing holiday data. Its purpose is mainly for holidata.net. Holiday data can be produced for a given year in a supported locale and output format.

Installation

First of all, clone the git repo

$ git clone https://github.com/GothenburgBitFactory/holidata.git
$ cd holidata

Inside of the repo you will find the setup.py file, run this with the parameter build and install

$ python3 ./setup.py build
$ sudo python3 ./setup.py install

After you finished the installation of holidata, you need to fullfill the requirements Therefor you can just install the following package with apt

$ sudo apt install python3-arrow python3-dateutil

Optional you can install the package python3-pytest as well if you want to run the test python script

$ sudo apt install python3-pytest

One more thing is missing, the holidata/holidays python file. Just run holidata the first time and you will get an error like that:

$ holidata --help
Traceback (most recent call last):
  File "/usr/local/bin/holidata", line 22, in <module>
    from holidata import Emitter
  File "/usr/local/lib/python3.8/dist-packages/holidata/__init__.py", line 1, in <module>
    from .holidays import *

There you will find the path where to store the missing python fines and you are good to go (e.g. /usr/local/lib/python3.8/dist-packages/holidata/)

# from the repo root
$ sudo cp -rf holidata/holidays /usr/local/lib/python3.8/dist-packages/holidata/
$ sudo chown -R root:staff /usr/local/lib/python3.8/dist-packages/holidata/holidays

Congratulations, now you should be able to run it without issues

$ holidata --help
Holidata - generate holidata files.

Usage:
  holidata (--year=<value>) (--locale=<value>) [--output=<value>]

Options:
    --year=<value>       Specify which year to generate data for.
    --locale=<value>     Specify the locale for which data should be generated.
    --output=(csv|json)  Specify the output format [default: csv].

Dependencies:
    pip3 install arrow docopt

Usage

Call holidata providing the necessary data, e.g.

$ holidata --year=2020 --locale=de-DE

Call holidata with the --usage or --help option to get usage info or help respectively.

Data

For each holiday the following data is provided:

• locale - language and country the holiday is defined for
• region - region code of the given subdivision the holiday is defined for
• date - actual date the holiday takes place
• description - name of the holiday in the given language
• type - holiday type flags
• notes - additional information

Output Formats

Holidata supports different output formats, currently csv, json, yaml, and xml.

If you think an output format is missing, open a feature request on github.

Limitations

Holidata focuses on holidays which are defined by law on which business or work are suspended or reduced (there may be some exceptions to that rule).

Holidata only provides data for countries and their principal subdivisions (both as they are defined in ISO 3166). Holidays for other subdivisions are either merged or ignored. There is also no explicit representation of partial holidays.

Docu review done: Wed 31 Jul 2024 02:33:51 PM CEST

ioping

Table of content

• Installation
• Measure latency
  • RAW Statistics
  • Samples

General

Latency measures with IOPing

Installation

$ apt install ioping

Measure latency

RAW Statistics

$ ioping -p 100 -c 200 -i 0 -q .

will output RAW data:

99 10970974 9024 36961531 90437 110818 358872 30756 100 12516420
100 9573265 10446 42785821 86849 95733 154609 10548 100 10649035
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Samples

Measure latency on . using 100 requests

$ ioping -c 100 .
4 KiB <<< . (xfs /dev/dm-2): request=1 time=16.3 ms (warmup)
4 KiB <<< . (xfs /dev/dm-2): request=2 time=253.3 us
4 KiB <<< . (xfs /dev/dm-2): request=3 time=284.0 ms
...
4 KiB <<< . (xfs /dev/dm-2): request=96 time=175.6 us (fast)
4 KiB <<< . (xfs /dev/dm-2): request=97 time=258.7 us (fast)
4 KiB <<< . (xfs /dev/dm-2): request=98 time=277.6 us (fast)
4 KiB <<< . (xfs /dev/dm-2): request=99 time=242.3 us (fast)
4 KiB <<< . (xfs /dev/dm-2): request=100 time=36.1 ms (fast)

--- . (xfs /dev/dm-2) ioping statistics ---
99 requests completed in 3.99 s, 396 KiB read, 24 iops, 99.3 KiB/s
generated 100 requests in 1.65 min, 400 KiB, 1 iops, 4.04 KiB/s
min/avg/max/mdev = 163.5 us / 40.3 ms / 760.0 ms / 118.5 ms

Last line shows the latency measures of the disk.

Measure latency on /tmp/using 10 requests of 1MB each

$ ioping -c 10 -s 1M /tmp

Measure disk seek rate

$ ioping -R /dev/sda

Measure disk sequential speed

$ ioping -RL /dev/sda

Get disk sequential speed in bytes per second

$ ioping -RLB . | awk '{print $4}'

iperf/iperf3

Table of content

• Commands
• Difference iperf and iperf3

Commands

Difference iperf and iperf3

iperf (iperf2) is the original tool where iperf3 comes from. The difference is that iperf3 is a complete re-write with a smaller and simpler code base and comes with more features, including a library version of iperf3 to be able to use it from another source code as well. iperf (iperf2) and iperf3 are not compactible, so don’t mix them up for testing, it will not work.

Docu review done: Wed 31 Jul 2024 02:34:19 PM CEST

iptables-persistent

General

iptables-persistent allows you to keep your currnet iptables rules after a reboot

Commands

Configfiles

Docu review done: Wed 31 Jul 2024 02:34:40 PM CEST

keepalived

Table of Content

• Template
• Sample configs master template
• Sample configs slave template

Template

• peempt_delay: this means that the server will not be master after a reboot for the defined time (sec)
• grap_master_delay: this speeds up the fail over and raises or lowes the ping time (default 5)
• track_script: track script influences the priority, if the checked is success full it returns the wight which summs up on the prio

so than the real prio is prio+weigth.

e.g.:

• Node1 has priority 101, application runs with weight 2, means Node1 has real priority 103
• Node2 has priority 100, application runs with weight 2, means Node2 has real priority 102.
• Node 1 will be master. If the application will be disabled on Node1 (priority lowers because of weight 2 to 101)
• Node 2 will become the new one. If now the application will be disbaled on Node2, Node1 will be the master again

global_defs {
    router_id <hostname>
}

vrrp_script chk_<application> {
    script "killall -0 <application>"  # cheaper than pidof
    interval 2                 # check every 2 seconds
    weight 2                 # add 2 points of prio if OK
}

# instance for LVSlan to use as gateway
vrrp_instance <Internal keepalived interface name> {
    interface <NIC>
    state <BACKUP/MASTER - if nopreemtp than only BACKUP>
    nopreempt
    virtual_router_id <INT virtual router id>
    priority <PRIO-INT value>
    advert_int 1
    peempt_delay 300
    garp_master_delay 1
    track_script {
        chk_<application>
    }

    authentication {
        auth_type PASS
        auth_pass <PWD - length 8>
    }
    virtual_ipaddress {
        <VIP>
    }
}

Sample configs master template

global_defs {
    router_id <hostname>
    notification_email {
        <destination@mail.com>
    }
    notification_email_from <keepalived@localdomainofserver.com>
    smtp_server localhost
    smtp_connect_timeout 30
    lvs_id LVS01
}

vrrp_script chk_<application> {
    script "killall -0 <application>"  # cheaper than pidof
    interval 2                 # check every 2 seconds
    weight 2                 # add 2 points of prio if OK
}

# instance for LVSlan to use as gateway
vrrp_instance <Internal keepalived interface name> {
    interface <NIC>
    lvs_sync_daemon_interface <NIC>
    state MASTER
    virtual_router_id <INT virtual router>
    priority <PRIO-INT vlaue>
    advert_int 1
    smtp_alert
    peempt_delay 300
    garp_master_delay 1
    track_script {
        chk_<application>
    }

    authentication {
        auth_type PASS
        auth_pass <PWD - length 8>
    }
    virtual_ipaddress {
        <VIM>
    }
}

Sample configs slave template

global_defs {
    router_id <hostname>
    notification_email {
        <destination@mail.com>
    }
    notification_email_from <keepalived@localdomainofserver.com>
    smtp_server localhost
    smtp_connect_timeout 30
    lvs_id LVS01
}

vrrp_script chk_<application> {
    script "killall -0 <application>"  # cheaper than pidof
    interval 2                 # check every 2 seconds
    weight 2                 # add 2 points of prio if OK
}

# instance for LVSlan to use as gateway
vrrp_instance <Internal keepalived interface name> {
    interface <NIC>
    lvs_sync_daemon_interface <NIC>
    state BACKUP
    virtual_router_id <INT virtual router>
    priority <PRIO-INT vlaue>
    advert_int 1
    smtp_alert
    peempt_delay 300
    garp_master_delay 1
    track_script {
        chk_<application>
    }

    authentication {
        auth_type PASS
        auth_pass <PWD - length 8>
    }
    virtual_ipaddress {
        <VIM>
    }
}

Docu review done: Mon 03 Jul 2023 16:39:52 CEST

kid3

General

kid3 is as well as easytag an audio tag manipulation tool and its based on kde and comes which quite some dependencies.

pros: way better userinterface as easytag and there is a kid3-cli package to have a command line interface as well.

kid3-cli

For example you can set with this command the file name as the title tag:

works for the following file naming schema:

[0-9]_<pert>_<of>_<title>.mp3

$ for f in *mp3 ; do kid3-cli -c "set Title \"$(sed -E 's/^[0-9]+_//g;s/_/ /g;s/.mp3//g' <<<"${f}")\"" ./$f ; done

It will replace all underscores with spaces and removes the track number at the front

Docu review done: Wed 31 Jul 2024 02:34:58 PM CEST

luminance

Table of Content

• Description
• Installation
• Samples
  • Convert RAW to PNG
• Grafical UI

Description

graphical user interface providing a workflow for HDR imaging

Installation

To installit on debian e.g. you can install it from the upstream debian repos like this: $ apt install luminance-hdr

Samples

Convert RAW to PNG

$ luminance-hdr-cli -b -o ./DSC_0450.png ./DSC_0450.NEF

Grafical UI

luminance-hdr comes also with an UI which allows you to perform actions also with the grafical interface.

Docu review done: Wed 31 Jul 2024 02:36:13 PM CEST

markdown

Table of content

• Internal header links
• Headers
  • Headers Best Practice
• Horizontal Rule
• Emphasis
• Lists
  • Unordered
  • Ordered
  • Mixed
• Images
  • Inline style
  • Reference style
• Links
  • Link Best Practices
  • Inline html based links
  • Inline custom IDs for headings
  • Index based links
• Footnotes
• Definition Lists
• Blockquotes
  • Nested blockquoates
  • Blockquotes Best Practice
• Inline Code
• Task Lists
• Tables
• Username AT mentions
• Automatic linking for URLs
• Strike through
• Inline HTML
• YouTube Videos
• Syntax highlighting
• Emojis
• Escaping Characters
  • Characters you can escape
  • Escaping backticks
  • Escaping Pipe in Tables
• Comments in Markdown
  • Adding HTML Comments in Markdown

General

Because no one can read your mind (yet)

Online documentation

Another online documentation

Internal header links

You can use the direct header name or you use the id from an html a tag

And depending on the markdown processors also custom heading ids

1. [Table of Contents](#table-of-contens)
2. [Headers](#headers)
...

Headers

also called headings

# This is an <h1> tag
## This is an <h2> tag
###### This is an <h6> tag

With HTML tag<a id="IDNAME"></a> or inline custom IDs #+ <HEEADER_NAME> {#<INLINE_ID>}, you can to have special characters in the header and still create a link

HTML tag sample:

<a id="headertag1"></a>
# This-is_an(<h1>) tag
<a id="headertag2"></a>
## This-is_an(<h2>) tag
<a id="headertag6"></a>
###### This-is_an(<h6>) tag

inline custom IDs sample:

# This-is_an(<h1>) tag {#headertag1}
## This-is_an(<h2>) tag {#headertag2}
###### This-is_an(<h6>) tag {#headertag6}

So if you have HTML tag/inline custom ID in place, you can use that one as a pointer like this:

[This-is_an(<h1>) tag](#headertag1)
[This-is_an(<h2>) tag](#headertag2)
[This-is_an(<h6>) tag](#headertag6)

Headers Best Practice

For compactibility, put blank lines before and after headers

Section above

## Headers Best Practice

For compactibility,....

Horizontal Rule

---

Emphasis

*This text will be italic*
_This will also be italic_

**This text will be bold**
__This will also be bold__

_You **can** combine them_

This text will be italic This will also be italic

This text will be bold This will also be bold

You can combine them

Lists

Unordered

* Item 1
* Item 2
    * Item 2a
    * Item 2b

• Item 1
• Item 2
  • Item 2a
  • Item 2b

Ordered

1. Item 1
2. Item 2
3. Item 3
    1. Item 3a
    2. Item 3b

1. Item 1
2. Item 2
3. Item 3
   1. Item 3a
   2. Item 3b

Mixed

1. Item 1
1. Item 2
1. Item 3
    * Item 3a
    * Item 3b

1. Item 1
2. Item 2
3. Item 3
   • Item 3a
   • Item 3b

Images

Inline style

Remote file: ![RemoteFile](https://gitea.sons-of-sparda.at/assets/img/logo.svg)

Locale file![LocaleFile](/images/mdadm_raid6.png)

Remote file:

Locale file:

The local file will only work if the path what you have specified exists in the same directory where the markdown file is stored Other wise it will look like above, where you just get the link shown but not he image it self

Reference style

Reference style remote file: ![RemoteFile][refstlogo]

[refstlogo]: https://gitea.sons-of-sparda.at/assets/img/logo.svg "Reference style logo"

Reference style remote file:

Links

Autodetect of url:                 https://github.com
Inline style:                      [Inlinestype-Github](https://github.com)
Inline style of url with title:    [Inlinestype-Github with title](https://github.com "GitHub Link")
External link mapping:             [GitHub](http://github.com)
Internal link in md:               [InternalMDLink](#links)
Internal link in md with spaces:   [InternalMDLinkWithSpaces](#inline-code)
Internal link in md with index:    [InternalMDLinkWithIndex](#pro-1)
Internal link in md with html tag: [InternalMDLinkWithHtmlTag](#imdlwht)
Reference link style:              [Arbitrary case-insensitive reference text] or [1]
Email link:                        <fancy-mail-link@sons-of-sparda.at>
Link on images:                    [![gitea svg](https://gitea.sons-of-sparda.at/assets/img/logo.svg "Link to gitea home")](https://gitea.sons-of-sparda.at)
[arbitrary case-insensitive reference text]: https://github.com
[1]: https://github.com

Link type	Url
Autodetect of url	https://github.com
Inline style	Inlinestype-Github
Inline style of url with title	Inlinestype-Github with title
External link mapping	GitHub
Internal link in md	InternalMDLink
Internal link in md with spaces	InternalMDLinkWithSpaces
Internal link in md with index	InternalMDLinkWithIndex
Internal link in md with html tag	InternalMDLinkWithHtmlTag
Reference link style	Arbitrary case-insensitive reference text or 1
Email link	fancy-mail-link@sons-of-sparda.at
Link on images

You can also point to inline html IDs. You create the inline link as above

[My first linline link](#mfll)

Link Best Practices

Markdown applications don’t agree on how to handle spaces in the middle of a URL. For compatibility, try to URL encode any spaces with %20.

[link](https://sons-of-sparda.at/site%20does%20not%20exist)

link

Inline html based links

Then you create somewhere your ID tag inside the md file:

<a id="mfll"></a>

Links to IDs can be very helpful when:

• your header is very long
• your header contains special characters
• your header exists multiple times (different solution for that below)

Inline custom IDs for headings

Many Markdown processors support custom IDs for headings — some Markdown processors automatically add them. Adding custom IDs allows you to link directly to headings and modify them with CSS. To add a custom heading ID, enclose the custom ID in curly braces on the same line as the heading.

### Inline custom IDs for headings {#icifh}

To link to a heading id, use the id as you would do normaly with a heading

Index based links

If you have multiple times the same header, you can also create links based on index number of same header.

- [Option1](#option1)
  - [Pro](#pro)
  - [Con](#con)
- [Option2](#option2)
  - [Pro](#pro-1)
  - [Con](#con-1)
- [Option3](#option3)
  - [Pro](#pro-2)
  - [Con](#con-2)

# Option1
## Pro
## Con

# Option2
## Pro
## Con

# Option3
## Pro
## Con

• Option1
  • Pro
  • Con
• Option2
  • Pro
  • Con
• Option3
  • Pro
  • Con

Option1

Pro

Con

Option2

Pro

Con

Option3

Pro

Con

Footnotes

Footnotes allow you to add notes and references without cluttering the body of the document. When you create a footnote, a superscript number with a link appears where you added the footnote reference. Readers can click the link to jump to the content of the footnote at the bottom of the page.

To create a footnote reference, add a caret and an identifier inside brackets [^1]. Identifiers can be numbers or words, but they can’t contain spaces or tabs. Identifiers only correlate the footnote reference with the footnote itself — in the output, footnotes are numbered sequentially.

Add the footnote using another caret and number inside brackets with a colon and text [^1]: My footnote.. You don’t have to put footnotes at the end of the document. You can put them anywhere except inside other elements like lists, block quotes, and tables.

Here's a simple footnote,[^1] and here's a longer one.[^bignote]

[^1]: This is the first footnote.

[^bignote]: Here's one with multiple paragraphs and code.

    Indent paragraphs to include them in the footnote.

    `{ my code }`

    Add as many paragraphs as you like.

Here’s a simple footnote,¹ and here’s a longer one.²

Definition Lists

Some Markdown processors allow you to create definition lists of terms and their corresponding definitions. To create a definition list, type the term on the first line. On the next line, type a colon followed by a space and the definition.

First Term
: This is the definition of the first term.

Second Term
: This is one definition of the second term.
: This is another definition of the second term.

First Term : This is the definition of the first term.

Second Term : This is one definition of the second term. : This is another definition of the second term.

Blockquotes

As Kanye West said:

> We're living the future so
> the present is our past.

As Kanye West said:

We’re living the future so the present is our past.

Nested blockquoates

> Blockquoats 1
>
>> Nested Blockquats 2

Blockquoats 1

Nested Blockquats 2

Blockquotes Best Practice

For compactibility, put blank lines before and after blockquotes

Line 1

> my blockquote

Line 3

Inline code

I think you should use an
`<addr>` element here instead.

I think you should use an <addr> element here instead.

Task Lists

- [x] @mentions, #refs, [links](markdown.md#task-lists), **formatting**, and <del>tags</del> supported
- [x] list syntax required (any unordered or ordered list supported)
- [x] this is a complete item
- [ ] this is an incomplete item

• @mentions, #refs, links, formatting, and tags supported
• list syntax required (any unordered or ordered list supported)
• this is a complete item
• this is an incomplete item

Tables

First Header | Second Header
------------ | -------------
Content from cell 1 | Content from cell 2 | Content from cell 3
Content in the first column | Content in the second column |  Content in the third column
*Still* | `renders` | **nicely**

Colons can be used to align columns

| Tables        | Are           | Cool  |
| ------------- |:-------------:| -----:|
| col 3 is      | right-aligned | $1600 |
| col 2 is      | centered      |   $12 |
| zebra stripes | are neat      |    $1 |

To create a new line sinside of a table, you can use the <br> tag like that:

Col1 | Col2 | Col3
---- | ---- | ----
one line | one line | first line <br> second line <br> third line
first line <br> second line | one line | one line
one line | first line <br> second line | one line
one line | one line | one very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very long line with no new line

Username AT mentions

Typing an @ symbol, followed by a username, will notify that person to come and view the comment. This is called an @mention, because you’re mentioning the individual. You can also @mention teams within an organization.

Automatic linking for URLs

Any URL (like http://www.github.com/) will be automatically converted into a clickable link.

Strike through

Any word wrapped with two tildes (like ~~this~~) will appear crossed out.

Any word wrapped with two tildes (like this) will appear crossed out.

Inline HTML

You can use raw HTML in your Markdown and it will mostly work pretty well.

<dl>
  <dt>Definition list</dt>
  <dd>Is something people use sometimes.</dd>

  <dt>Markdown in HTML</dt>
  <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd>
</dl>

Definition list

Is something people use sometimes.

Markdown in HTML

Does *not* work **very** well. Use HTML tags.

YouTube Videos

They can’t be added directly but you can add an image with a link to the video like this:

<a href="http://www.youtube.com/watch?feature=player_embedded&v=YOUTUBE_VIDEO_ID_HERE " target="_blank">
<img src="http://img.youtube.com/vi/YOUTUBE_VIDEO_ID_HERE/0.jpg"
alt="IMAGE ALT TEXT HERE" width="240" height="180" border="10" />
</a>

Or, in pure Markdown, but losing the image sizing and border:

[![IMAGE ALT TEXT HERE](http://img.youtube.com/vi/YOUTUBE_VIDEO_ID_HERE/0.jpg)](http://www.youtube.com/watch?v=YOUTUBE_VIDEO_ID_HERE)

TeX Mathematical Formulae

A full description of TeX math symbols is beyond the scope of this cheat sheet. Here’s a good reference, and you can try stuff out on CodeCogs. You can also play with formulae in the Markdown Here options page.

Here are some examples to try out:

Syntax highlighting

```markdown
> Use markdown highlighting
[because we can](#syntax-highlighting)
``` # no text, only close the section

Supported syntax highlighting languages:

Emojis

Emojis are common, in chats/messengers, mails and so on and of course, you can have them in markdown as well.

It depends on where you markdown file will be placed, based on that, different emojis are available

For gitea you can have a look at this list: https://gitea.com/gitea/gitea.com/issues/8 For github, you can have a look there: https://github.com/StylishThemes/GitHub-Dark/wiki/Emoji

In general the syntax looks like this:

:EMOJINAME:

It can be place everywhere :yellow_heart:

So have fun and enjoy them :wink:

Escaping Characters

To display a literal character that would otherwise be used to format text in a Markdown document, add a backslash \ in front of the character

\* Without the backslash, this would be a bullet in an unordered list.

* Without the backslash, this would be a bullet in an unordered list.

Characters you can escape

You can use a backslash to escape the following characters.

Escaping backticks

If the word or phrase you want to denote as code includes one or more backticks, you can escape it by enclosing the word or phrase in double backticks ``

``Use `code` in your Markdown file.``

Use `code` in your Markdown file.

Escaping Pipe in Tables

You can display a pipe | character in a table by using its HTML character code | also putting it into a code section does not help you there.

Comments in Markdown

Markdown doesn’t include specific syntax for comments, but there is a workaround using the reference style links syntax. Using this syntax, the comments will not be output to the resulting HTML.

[]: # (This is a comment)
[]: # "And this is a comment"
[]: # 'Also this is a comment'
[//]: # (Yet another comment)
[comment]: # (Still another comment)

[]: # (This is a comment) []: # “And this is a comment” []: # ‘Also this is a comment’ [//]: # (Yet another comment) [comment]: # (Still another comment)

Each of these lines works the same way:

• [...]: identifies a reference link (that won’t be used in the article)
• # defines the destination, in this case # is the shortest valid value for a URL
• (...), "...", and '...' define the reference title, which we repurpose to make a comment

Adding HTML Comments in Markdown

If you’d like for your comments to show up in the HTML output, a simple modified HTML comment syntax will work:

<!--- This is an HTML comment in Markdown -->

Unlike a “normal” HTML comment which opens with <!-- (two dashes), an HTML comment in Markdown opens with <!--- (three dashes). Some Markdown parsers support two-dash HTML comments, but the three-dash version is more universally compatible.

Docu review done: Thu 29 Jun 2023 12:33:11 CEST

Table of content

• Commands and Descriptions
• Installation
• Raid Levels
• Create array
  • Parameter
  • Sample Create RAID 0
  • Sample Create RAID 1
• Delete array
  • Sample Delete array
• List arrays and partitions
• Hotspare
• Rebuild
• Checking array
• md device vanished and mdadm sefault

Commands and Descriptions

-n [0-9]+ is equivalent to --raid-devices=[0-9]+

-l[0-9]+ is equivalent to --level [0-9]+

To boot a machine even with a degraded array, modify /etc/initramfs-tools/conf.d/mdadm and run update-initramfs -c -kall (Use with caution!)

Installation

$ apt install mdadm

Raid Levels

To get an over view of RAID have a look at the RAID documentation

Create array

$ mdadm --create /dev/md/<lable> --level=<RAID-Level> --raid-devices=<sum of physical partitions in array> /dev/<device1> /dev/<device2> /dev/<deviceX>

Parameter

• --create : for optional labble on raid devie, e.g. /dev/md/md_test
• --level= : defines raid level. Allowed values are: linear, raid0, 0, stripe, raid1, 1, mirror, raid4, 4, raid5, 5, raid6, 6, raid10, 10, multipath, mp, faulty, container
• --raid-devies= : specifies the phyical partion inside the newly generated sofware raid, e.g. --raid-devices=2 /dev/sdb /dev/sdc3

Sample Create RAID 0

Creating a RAID 0 (block level striping) with two partitions

$ mdadm --create /dev/md/md_test --level=0 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md_test started.

Sample Create RAID 1

Createing a RAID 1

$ mdadm --create /dev/md/md_test --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md_test started.

Delete array

To be able to remoe an array it needs to be unmounted and you have to run

$ mdadm --stop /dev/md/<raid name>

To remove the array you need to set the super block to null on each disk

$ mdadm --zero-superblock /dev/sdX

Sample Delete array

$ umount -l /mnt/test
$ mdadm --stop /dev/md/md_test
mdadm: stopped /dev/md/md_test
$ mdadm --zero-superblock /dev/sdb1
$ mdadm --zero-superblock /dev/sdc1

List arrays and partitions

RAID-Arrays can be lsited with two commands

• --detail : is showing the full acvie array
• --examine : shows details about individual physical devices inside the raid

$ mdadm --examine --brief --scan  --config=partitions
ARRAY /dev/md/md_test metadata=1.2 UUID=81c1d8e5:27f6f8b9:9cdc99e6:9d92a1cf name=swraid:md_test

This command can be shorted with -Ebsc partitions

$mdadm --detail /dev/md/md_test
/dev/md/md_test:
        Version : 1.2
  Creation Time : Fri Jul  5 09:14:36 2013
     Raid Level : raid0
     Array Size : 16776192 (16.00 GiB 17.18 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Fri Jul  5 09:14:36 2013
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

     Chunk Size : 512K

           Name : swraid:md_test  (local to host swraid)
           UUID : 81c1d8e5:27f6f8b9:9cdc99e6:9d92a1cf
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       8       33        1      active sync   /dev/sdc1

Hotspare

A hotspare disc/partition are devices which are not used in emergency cases. They will be used if an active disc/partition from a RAID has issues or is broken. If there is not hotspare disk defined inside of a RAID, you need to perform the rebuild manually. If there is one defined the rebuild will happen automatically. To add a disc/partition as hotspare, you can run the command:

mdadm --add /dev/md/<RAID Name> /dev/sdX

If you wana remove a hotspare disc/partition from an existing raid, you can use the command:

mdadm --remove /dev/md/<RAID Name> /dev/sdX

Rebuild

If there are issues on a disc/partition inside of the RAID, you need to trigger the rebuild. To do that, first you need to remove broken disc/drive form the RAID which will be done with the command

$ mdadm --mange /dev/md/<RAID Name> -r /dev/sdX

Important for the new disc/partition is that the size is the same as the broken one. There are several tools which help you on performing partitioning of drives e.g fdisk /dev/sdX, cfdisk /dev/sdX, parted /dev/sdX.

Is the size of the new disc/partition matching the size from the broken one it can be added to the RAID. For adding it, run the command:

$ mdadm --manage /dev/md/<RAID Name> -a /dev/sdX

If there are no isses during the adding of the disc/partition you can start the rebuild. To achieve that you need to set the new disk/partition to the state “faulty” by running:

$ mdadm --manage --set-faulty /dev/md/<RAID Name> /dev/sdX

By doing that, the rebuild will be triggered and you can watch the status of it with the command cat /proc/mdstat.

Every 2.0s: cat /proc/mdstat                                                         Fri Jul  5 09:59:16 2013


root@swraid:/dev# watch cat /proc/mdstat
Personalities : [raid0] [raid1]
md127 : active raid1 sdc1[1] sdb1[0]
      8384448 blocks super 1.2 [2/2] [UU]
      [==============>......]  check = 74.7% (6267520/8384448) finish=0.1min speed=202178K/sec

unused devices: <none>

After the rebuild process finished, you need to remove and add the disk/partition again to the RAID to remove the “faulty” state. Just run the commands like this for removing/adding the disk/partition:

$ mdadm --manage /dev/md/<RAID Name> -r /dev/sdX && mdadm --manage /dev/md/<RAID Name> -a /dev/sdX

To verify that the state is good again of the RAID you can use the command mdadm --detail /dev/md/<RAID Name> and this should show now State: clean.

Checking array

To make use of permanent monitoring the tool checkarray needs to be available and can be combined within cron.

/usr/share/mdadm/checkarray --cron --all --quiet

md device vanished and mdadm sefault

If cat /proc/mdstat does not return any output and/or hangs and you see segfault in dmesg about mdadm like this:

mdadm[42]: segfault at ffffffff0000009c ip 00000000004211fe sp 000050bbaf1211fb error 5 in mdadm[400000+78000]

You have to enforce the assemble of the md device and re-grow it

$ mdadm --assemble --foce /dev/md0 /dev/sda1 /dev/sda2 /dev/sda3 /dev/sda4 /dev/sda5
$ mdadm --grow /dev/md0 -n 6 -l 1

Now it should be back in an healthy state.

Docu review done: Mon 06 May 2024 09:56:44 AM CEST

mdbook

Table of Content

• General
• Concept for building this wiki
  • Setup and Requirements
  • Repo structure
    • mdbook
    • docu
  • Update and build script sample

General

mdbook is able to create out of markdown files, SUMMARY.md (as controler file) and book.toml (config for wiki) files html files.

This wiki is build fully automated out of a existing git repository which holds the .md file.

The original mdBook documentation

Concept for building this wiki

Setup and Requirements

• mdbook is available in your $PATH, (installation documentation)
• Repository holding the doumentaion files .md
• Repository hodlign configuration for mdbook
• Trigger/hook for executing update script (gitea webhook used in this case)
  • for catching the push based gitea-webhook we use in this case webhookey
• A running webserver for ofthering the files

Inside of the mdbook config repository, we have specified the documention repository as a submodule on the direcotry src.

Repo structure

mdbook

/data/mdbook
├── .gitignore
├── .gitmodules
├── book
├── book.toml
├── src
└── update_deploy.sh

docu

/data/mdbook/src
├── SUMMARY.md
├── dir1
│   ├── dokufile1_1.md
│   └── dokufile1_2.md
└── dir2
    ├── dokufile2_1.md
    └── dokufile2_2.md

Update and build script sample

This is a sample who you can automatically build the book and copy the new data to your html path:

#!/bin/bash
mdbook_source="/data/mdbook"
mdbook_source_src="${mdbook_source}/src"
mdbook="/opt/mdbook/mdbook"

echo "1: Updateing mdbook repo"
git -C "${mdbook_source}" pull -q origin master

echo "2: Removing 'old' src dir"
rm -rf "${mdbook_source_src}"/* "${mdbook_source_src}"/.g* >/dev/null

echo "3: Regenerating src dir"
git -C "${mdbook_source}" submodule -q update --init --recursive

echo "4: Updating submodules"
git -C "${mdbook_source}" submodule -q update --remote

echo "5: Rebuilding mdbook"
mdbook build "${mdbook_source}" >/dev/null

echo "6: Updating external directory"
rsync -aP "${mdbook_source}"/book/* /var/www/wiki_mdbook >/dev/null

Docu review done: Wed 31 Jul 2024 02:36:27 PM CEST

mdless

General

mdless is a terminal based markdown reader

Installation

$ snap install mdless

Requirements

Snapd needs to be installed on the debian system as long mdless is not available on the debian package repo

URLs

https://snapcraft.io/install/mdless/debian

Docu review done: Wed 31 Jul 2024 02:36:37 PM CEST

mp3info

General

With mp3info you can set and list all ID3 tags on mp3s

Commands

$ mp3info <audo_file.mp3>     # list mp3 ID3 tags

Docu review done: Mon 06 May 2024 09:59:54 AM CEST

Network tools

Table of Content

• iptraf-ng
• tcpdump
  • Commands
    • netstat-nat
    • netstat-nat snat
    • netstat-nat dnat

iptraf-ng

showing actuall network trafic with nice ui

tcpdump

shoing actuall network trafic tcpdump doku

Commands

$ tcpdump -n -i anz host 10.10.10.10 and port 1234 or port 6789

netstat-nat

Show the natted connections on a linux iptable firewall

netstat-nat snat

$ netstat-nat -S
Proto NATed Address                  Destination Address            State
tcp   10.13.37.35:40818              orwell.freenod:afs3-fileserver ESTABLISHED
tcp   10.13.37.35:45422              refraction.oftc.net:ircs-u     ESTABLISHED
tcp   10.13.37.35:57510              jmt1.darkfasel.net:9999        ESTABLISHED
tcp   10.84.42.3:58288               104.22.27.164:https            TIME_WAIT
tcp   10.84.42.3:46266               104.22.23.187:https            ESTABLISHED
udp   10.13.37.2:52543               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.2:50158               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.2:43517               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.2:41412               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.64:8303               master.status.tw:8283          ASSURED
udp   10.13.37.64:8303               twmaster2.teecloud.eu:8283     ASSURED
udp   10.13.37.64:8303               twmaster3.teecloud.eu:8283     ASSURED
udp   10.13.37.64:8303               ddnet.tw:8283                  ASSURED
udp   10.84.42.3:57388               185.69.161.157:9987            ASSURED

# with filter on source
$ netstat-nat -S -s 10.13.37.2
Proto NATed Address                  Destination Address            State
udp   10.13.37.2:52543               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.2:50158               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.2:43517               dns9.quad9.net:domain          UNREPLIED
udp   10.13.37.2:41412               dns9.quad9.net:domain          UNREPLIED

netstat-nat dnat

$ netstat-nat -D
Proto NATed Address                  Destination Address            State

# with filter on testination
$ netstat-nat -D -d 9.9.9.9
Proto NATed Address                  Destination Address            State

nmap

Table of Content

• [Scan OpenSSH server for Algorythims](#scan openssh server for algorythims)
• [Scan Ports for Ciphers TLS Protokolls](#scan ports for ciphers tls protokolls)
• [Scan Webserver for accessable files and directories](#scan webserver for accessable files and directories)
• [Other usefull scanns](#other usefull scanns)

Scan OpenSSH server for Algorythims

To see what an OpenSSH server offers for algorythms you can use the following command:

$ nmap --script ssh2-enum-algos -sV -p <PORT> <IP -n/FQDN> -P
Nmap scan report for <FQDN> (<IP>)
Host is up (0.042s latency).

PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH <VERSION> <OS VERSION> (protocol <VERSION>)
| ssh2-enum-algos:
|   kex_algorithms: (4)
|       curve25519-sha256@libssh.org
|       diffie-hellman-group-exchange-sha256
|       diffie-hellman-group14-sha1
|       kex-strict-s-v00@openssh.com
|   server_host_key_algorithms: (5)
|       rsa-sha2-512
|       rsa-sha2-256
|       ssh-rsa
|       ssh-ed25519
|       ssh-ed25519-cert-v01@openssh.com
|   encryption_algorithms: (5)
|       chacha20-poly1305@openssh.com
|       aes256-gcm@openssh.com
|       aes128-gcm@openssh.com
|       aes256-ctr
|       aes128-ctr
|   mac_algorithms: (5)
|       hmac-sha2-512-etm@openssh.com
|       hmac-sha2-256-etm@openssh.com
|       umac-128-etm@openssh.com
|       hmac-sha2-512
|       hmac-sha2-256
|   compression_algorithms: (2)
|       none
|_      zlib@openssh.com
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Scan Ports for Ciphers TLS Protokolls

To see which Ciphers and TLS versions are supported by an application you can use ssl-enum-ciphers:

$ nmap --script ssl-enum-ciphers -sV -p <PORT> <IP -n/FQDN> -P
Nmap scan report for <FQDN> (<IP>)
Host is up (0.042s latency).

PORT    STATE SERVICE  VERSION
<PORT>/tcp open  <SERVICE/PROTOKOLL> <APPLICATION VERSION>
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_DHE_RSA_WITH_AES_128_CCM (dh 4096) - A
|       TLS_DHE_RSA_WITH_AES_128_CCM_8 (dh 4096) - A
|       TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (dh 4096) - A
|       TLS_DHE_RSA_WITH_AES_256_CCM (dh 4096) - A
|       TLS_DHE_RSA_WITH_AES_256_CCM_8 (dh 4096) - A
|       TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (dh 4096) - A
|       TLS_DHE_RSA_WITH_ARIA_256_GCM_SHA384 (dh 4096) - A
|       TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (dh 4096) - A
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
|     compressors:
|       NULL
|     cipher preference: client
|     warnings:
|       Key exchange (secp256r1) of lower strength than certificate key
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (ecdh_x25519) - A
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (ecdh_x25519) - A
|     cipher preference: client
|_  least strength: A
|_http-server-header: Apache

Scan Webserver for accessable files and directories

$ nmap --script http-enum -sV -p <PORT> <IP -n/FQDN> -P
Nmap scan report for <FQDN> (<IP>)
Host is up (0.0021s latency).

PORT    STATE SERVICE VERSION
<PORT>/tcp open  <SERVICE/PROTOKOLL> <APPLICATION VERSION>
|_http-server-header: <APPLICATION HEADER NAME>
| http-enum:
|_  /<DIR>/: Potentially interesting folder
|_  /<FILE>: Potentially interesting file

Other usefull scanns

Docu review done: Tue 17 Oct 2023 10:50:14 AM CEST

Table of content

• General
• Installation
• Preperation
• Configuration
  • OpenDKIM Config
  • Defaults for opendkim service
  • Excludes for Signing
  • Key pairing
  • Kreating key pair
  • Placing the DNS record
  • Restart systemd service
  • Testing your setup
  • Postfix setup
  • Mailtest
• Issues
  • Double sigend mail

General

In this documentation, we assue that you have already a running and working postgrey instance on your server.

Installation

First you need fo course the packages installed, for Debian, you can use the command below.

$ apt install opendkim opendkim-tools

Preperation

Next is to create the dircetories where you want to store your configurtion files and keys and set the correct owner and group + permissions.

mkdir -p /etc/opendkim/keys
chown -R opendkim:opendkim /etc/opendkim
chmod go-rw /etc/opendkim/keys

Assuming you have the above mentined packages already installed and running Debian (or a Debian based system), you should add the user postgres to the group opendkim.

Configuration

OpenDKIM Config

Now you need to configure your /etc/opendkim.conf file.

This is how a configuration could look like:

# This is a basic configuration for signing and verifying. It can easily be
# adapted to suit a basic installation. See opendkim.conf(5) and
# /usr/share/doc/opendkim/examples/opendkim.conf.sample for complete
# documentation of available configuration parameters.

Syslog              yes
SyslogSuccess       yes

# Common signing and verification parameters. In Debian, the "From" header is
# oversigned, because it is often the identity key used by reputation systems
# and thus somewhat security sensitive.
Canonicalization    relaxed/simple
Mode                sv
SubDomains          no
AutoRestart         yes
AutoRestartRate     10/1M
Background          yes
DNSTimeout          5
SignatureAlgorithm  rsa-sha256
OversignHeaders     From

# In Debian, opendkim runs as user "opendkim". A umask of 007 is required when
# using a local socket with MTAs that access the socket as a non-privileged
# user (for example, Postfix). You may need to add user "postfix" to group
# "opendkim" in that case.
UserID          opendkim
UMask           007

# Socket for the MTA connection (required). If the MTA is inside a chroot jail,
# it must be ensured that the socket is accessible. In Debian, Postfix runs in
# a chroot in /var/spool/postfix, therefore a Unix socket would have to be
# configured as shown on the last line below.
Socket          local:/var/spool/postfix/opendkim/opendkim.sock

PidFile         /run/opendkim/opendkim.pid

# The trust anchor enables DNSSEC. In Debian, the trust anchor file is provided
# by the package dns-root-data.
TrustAnchorFile     /usr/share/dns/root.key

# Map domains in From addresses to keys used to sign messages
KeyTable           refile:/etc/opendkim/key.table
SigningTable       refile:/etc/opendkim/signing.table

# Hosts to ignore when verifying signatures
ExternalIgnoreList  /etc/opendkim/trusted.hosts

# A set of internal hosts whose mail should be signed
InternalHosts       /etc/opendkim/trusted.hosts

Defaults for opendkim service

For Debian and Debian based systems, you should also configure the file /etc/defaults/opendkim. Sample:

# NOTE: This is a legacy configuration file. It is not used by the opendkim
# systemd service. Please use the corresponding configuration parameters in
# /etc/opendkim.conf instead.
#
# Previously, one would edit the default settings here, and then execute
# /lib/opendkim/opendkim.service.generate to generate systemd override files at
# /etc/systemd/system/opendkim.service.d/override.conf and
# /etc/tmpfiles.d/opendkim.conf. While this is still possible, it is now
# recommended to adjust the settings directly in /etc/opendkim.conf.
#
#DAEMON_OPTS=""
# Change to /var/spool/postfix/run/opendkim to use a Unix socket with
# postfix in a chroot:
RUNDIR=/var/spool/postfix/opendkim
#RUNDIR=/run/opendkim
#
# Uncomment to specify an alternate socket
# Note that setting this will override any Socket value in opendkim.conf
# default:
SOCKET=local:$RUNDIR/opendkim.sock
USER=opendkim
GROUP=opendkim
PIDFILE=$RUNDIR/$NAME.pid
EXTRAAFTER=

Excludes for Signing

Sometimes you don’t want to sign a mail, e.g. if you are sending the mail only internal. To do this, you have to add them into the file /etc/opendkim/trusted.hosts or better say, in the file which you have configured within the option InternalHosts.

10.0.0.0/8
127.0.0.1
::1
localhost
itgui.de
*.itgui.de

Key pairing

To configure which key should be use for which (sub)domain, you need to add this information into the file /etc/opendkim/signing.table or better say, in the file which you have configured within the option SigningTable like this:

*@itgui.de    default._domainkey.itgui.de

If you have more keys for other (sub)domains, just add a new line and specify there the mapping.

Just make sure that you have only one mapping per line.

Next is to link the link now virtual name with the real flie. This is done in the file /etc/opendkim/key.table or better say, in the file which you have configured within the option KeyTable like this:

default._domainkey.itgui.de     itgui.de:default:/etc/opendkim/keys/itgui.de/default.privat

Kreating key pair

To create the key pair, you can use the command opendkim-genkey, like this (+ ensure the permissions):

$ opendkim-genkey -b 2048 -d "itgui.de" -D "/etc/opendkim/keys/itgui.de" -s default
$ chown -R opendkim. /etc/opendkim/keys/itgui.de
$ chmod -R o-rw /etc/opendkim/keys/itgui.de

In general it is recommended to renew the keys every 4 months. But if you also look at the big players, they also keep sometimes the for years ;)

If you are rotating keys, keep in mind, mails can keep alive for very long times, so ensure that you have both keys (old and new one) available for quite some time in DNS records to ensure, that mails signed with old keys, can be still validated.

Placing the DNS record

Now you need to place your public key into your DNS as a TXT record.

First you need the data what you want to place, basically it is everything which is between the breaks ( ), but here is a oneline for it (just a sample output) :

$ echo "v=DKIM1; k=rsa; s=email; $(sed -E '/default._domain.*IN.*TXT.*DKIM/d;s/\t*  *"//g;s/".*//g' "/etc/opendkim/keys/itgui.de/default.txt")"
v=DKIM1; k=rsa; s=email; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCA...73Qzl5Czna797955/zX7Bp10e/lATZbVtP6Qu6eC2TMpWx06bEDRZ...oAtNNuhQIDAQAB

If you are able to talk to your DNS via API, you can easily put that into an automation, like this for netcup.de DNS:

#!/bin/bash
dkim_domain="${1}"
dkim_pre="v=DKIM1; h=sha256; k=rsa;"

dkim_dns_record_id=$(/usr/local/scripts/ncdapi.sh -g "${dkim_domain}" | jq -r '.[] |  select ( .destination | contains("DKIM")) | .id')
if grep -qE '[0-9]{8,}' <<<"${dkim_dns_record_id}" ; then
    echo "Detected existing DKIM record (id: ${dkim_dns_record_id}), skipping process..."
    exit 0
fi

dkim_key=$(sed -E '/default._domain.*IN.*TXT.*DKIM/d;s/\t*  *"//g;s/".*//g' "/etc/opendkim/keys/${dkim_domain}/default.txt" | tr -d '\n')

if [ -z "${dkim_key}" ] || grep -vqE "^p=" <<<"${dkim_key}"; then
    echo "Failed during parsing, skipping process..."
    exit 1
fi

if /usr/local/scripts/ncdapi.sh -N default._domainkey "${dkim_domain}" TXT "${dkim_pre} ${dkim_key}" ; then
    echo "DNS record added"
else
    echo "Failed to insert DNS record"
    exit 1
fi

If not, then create it with the following specs (adopted to your data of course):

• Host: default._domainkey
• Domain: itgui.de
• Type: TXT
• Destination: v=DKIM1; k=rsa; s=email; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCA...73Qzl5Czna797955/zX7Bp10e/lATZbVtP6Qu6eC2TMpWx06bEDRZ...oAtNNuhQIDAQAB

Restart systemd service

Now we have everything on the server in place and can finaly restart the opendkim serivce

$ systemctl restart opendkim.serivce

Testing your setup

Of course you want to konw if it is working, and with the command opendkim-testkey you can do that.

opendkim-testkey -d itgui.de -s default -vvv
  opendkim-testkey: using default configfile /etc/opendkim.conf
  opendkim-testkey: checking key 'default._domainkey.itgui.de'
  opendkim-testkey: key not secure
  opendkim-testkey: key OK

Dont worry about the key not secure this is just that DNSSEC is not in use right now and you can continue to work with it.

Postfix setup

Now we want to enable it in postfix.

Before you enable it, set the option soft_bounce=yes in your /etc/postfix/main.cf.

By setting this option, you don’t loose any mails if something goes wrong in your setup and postfix will just respond with a 4xx error to the sender mail server, instead of 5xx error, which leads to, that the sender mail server will retry it in a couple of minutes again.

Inside the /etc/postfix/main.cf you have to add the following to enable the opendkim:

# Milter configuration
milter_default_action = accept
milter_protocol = 6
smtpd_milters = local:/opendkim/opendkim.sock
non_smtpd_milters = $smtpd_milters

If you have already something in your milter (e.g. spamassassin), then just add it like this:

smtpd_milters     = unix:spamass/spamass.sock, local:/opendkim/opendkim.sock
non_smtpd_milters = local:/opendkim/opendkim.sock

Now you have to restart the postfix service like this for example systemctl restart postfix.serivce.

Mailtest

Now you can try to send your self a mail, after that have a look at the mail headers and you should find something like this:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=itgui.de;
    s=default; t=1479644038;
    bh=g3zLYH4xKxcPrHOD18z9YfpQcnk/GaJedfustWU5uGs=;
    h=To:From:Subject:Date:From;
    b=VI6TIDLrzG8nAUYWwt5QasKJkkgU+Sv8sPGC1fynSEGo0GSULGgCjVN6KXPfx1rgm
     1uX2sWET/oMCpxjBFBVUbM7yHGdllhbADa2SarzYhkoEuNhmo+yxGpXkuh0ttn4z7n
     OmkLwjrdLafMOaqSBrXcTMg/593b/EQXtouEvHqOG59d0fubIGCJBF6DG5gcITS9CP
     q9tiVwIDVuQqynL9Crodl+2IObcvl15MeK2ej322qrjrTij6vZOru9SeVno4LNTkQ7
     tOT4s14BGLx8aRe5YXZj38AWsR6DxkT6OzM+3TnOhIfX3ZdkufMz8AUnTNuLhViZ1M
     betE0x1iOi/HQ==

A nice test page for this is isnotspam, before testing, you should wait a bit till the DNS changes has propagaged throught the DNS servers over the wild.

Issues

Double sigend mail

If you are running postfix, spamassassin and opendkim for the same mail instance, then it can happen that you see that your outgoing mails are swinged twice by opendkim.

The reason for that is, that they get signed before spamassassins takes care about the mail and after it. In the following logsample, you see the double signing.

May  4 13:37:42 my_mail_server postfix/smtpd[2240071]: connect from myhost[10.0.0.2]
May  4 13:37:42 my_mail_server postfix/smtpd[2240071]: Anonymous TLS connection established from myhost[10.0.0.2]: TLSv...
May  4 13:37:42 my_mail_server postfix/smtpd[2240071]: E242018B0000: client=myhost[10.0.0.2], sasl_method=xxxxx, sasl_username=myuser@itgui.de
May  4 13:37:42 my_mail_server postfix/cleanup[2240086]: E242018B0000: message-id=<a3cdd989-e554-8cc5-1c8d-1e1c5697ae9e@itgui.de>
May  4 13:37:42 my_mail_server opendkim[2240050]: E242018B0000: DKIM-Signature field added (s=default, d=itgui.de)
May  4 13:37:42 my_mail_server postfix/qmgr[3846771]: E242018B0000: from=<myuser@itgui.de>, size=1712, nrcpt=1 (queue active)
May  4 13:37:42 my_mail_server spamd[2644747]: spamd: connection from ::1 [::1]:57232 to port 783, fd 5
May  4 13:37:42 my_mail_server spamd[2644747]: spamd: setuid to spamassassin succeeded
May  4 13:37:42 my_mail_server spamd[2644747]: spamd: processing message <a3cdd989-e554-8cc5-1c8d-1e1c5697ae9e@itgui.de> for spamassassin:5000
May  4 13:37:42 my_mail_server postfix/smtpd[2240071]: disconnect from myhost[10.0.0.2] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
May  4 13:37:42 my_mail_server spamd[2644747]: spamd: clean message (-0.8/5.0) for spamassassin:5000 in 0.2 seconds, 2173 bytes.
May  4 13:37:42 my_mail_server spamd[2644747]: spamd: result: . 0 - ALL_TRUSTED,DKIM_INVALID,DKIM_SIGNED,URIBL_BLOCKED scantime=0.2,size=2173,user=spambro,uid=0000,required_score=5.0,rhost=::1,raddr=::1,rport=57232,mid=<a3cdd989-e554-8cc5-1c8d-1e1c5697ae9e@tgui.de>,autolearn=no autolearn_force=no
May  4 13:37:42 my_mail_server postfix/pickup[2021986]: 274A118B0005: uid=0000 from=<myuser@itgui.de>
May  4 13:37:42 my_mail_server postfix/pipe[2240087]: E242018B0000: to=<myuser@itgui.de>, orig_to=<my_second_user@itgui.de>, relay=spamassassin, delay=0.26, delays=0.08/0.01/0/0.18, dsn=2.0.0, status=sent (delivered via spamassassin service)
May  4 13:37:42 my_mail_server postfix/qmgr[3846771]: E242018B0000: removed
May  4 13:37:42 my_mail_server postfix/cleanup[2240086]: 274A118B0005: message-id=<a3cdd989-e554-8cc5-1c8d-1e1c5697ae9e@itgui.de>
May  4 13:37:42 my_mail_server opendkim[2240050]: 274A118B0005: DKIM-Signature field added (s=default, d=itgui.de)
May  4 13:37:42 my_mail_server postfix/qmgr[3846771]: 274A118B0005: from=<myuser@itgui.de>, size=2639, nrcpt=1 (queue active)
May  4 13:37:42 my_mail_server postfix/pipe[2240092]: 274A118B0005: to=<myuser@itgui.de>, relay=dovecot, delay=0.03, delays=0.01/0.01/0/0.02, dsn=2.0.0, status=sent (delivered via dovecot service)

To get rid of the issue, you have to perform a small change in the master.cf of postfix.

Search in the master.cf the line where you have spamassassin configured for the smtp service and add -o receive_override_options=no_milters like this:

# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
  -o content_filter=spamassassin -o receive_override_options=no_milters

Now all you have to do is restarting postgres and your issue should be solved.

Docu review done: Wed 31 Jul 2024 02:37:05 PM CEST

pastebinit

General

pastebinit is an interact with online pastbin instances

Installation

$ apt install pastebinit

Comamnds

Sampes

$ pastebinit -i hello_unixnlinux.txt -b http://pastebin.com               # sending file to url
http://pastebin.com/d6uXieZj                                            # result will be a url which you can share:

$ pastebinit -i sample.bash -f bash -a ex_bash_1 -b http://pastebin.com   # sending file and set syncax for bash
http://pastebin.com/jGvyysQ9

pee

Table of Content

• Description
• Installation
• Examples

Description

pee is like tee but for pipes. Each command is run and fed a copy of the standard input. The output of all commands is sent to stdout.

Note that while this is similar to tee, a copy of the input is not sent to stdout, like tee does. If that is desired, use pee cat …

Installation

In Debian and for Debian based systems you will find it in the package moreutils, which can be installed with apt.

$ apt install moreutils

Examples

$ ~/ ll | pee '! /bin/grep Jan | /bin/grep cache' '! /bin/grep Feb | /bin/grep config'
drwxr-xr-x 39 user user 4.0K Jan 27 16:27 .cache/
lrwxrwxrwx  1 user user   61 Feb  9  2016 .config -> /home/myuser/git/configs
lrwxrwxrwx  1 user user   45 Feb  1  2017 .ssh -> .configs/gitdir/.ssh/
drwxr-xr-x 39 user user 4.0K Jan 10 09:01 CACHEdir/
drwxr-xr-x 38 user user 4.0K Feb 13 09:50 configdirectory/

Docu review done: Mon 03 Jul 2023 16:50:59 CEST

Table of Content

• Commands and Descriptions
• Backup DBs
• Restore a DB
• Upgrade from Postgresversion X to N

Commands and Descriptions

Backup DBs

For dumping a DB you can use the pg_dump Either use dump a dedicated DB

$ /usr/bin/pg_dump -h host -p port dbname --clean > /packup/path/backupfilename.db.dump

or all DBs

$ /usr/bin/pg_dumpall -h host -p port --clean > /packup/path/backupfilename.db.dump

Restore a DB

Restores can just run a psql command with -f parameter (if it got dumped with pg_dumpall)

# backup generated like this: pg_dumpall dbname --clean > /packup/path/backupfilename.db.dump
$ psql -f /pacup/path/backupfilename.db.dump

Normal dumps are restored like with redirects:

# backup generated like this: pg_dump dbname > /packup/path/backupfilename.db.dump
$ createdb -T template0 mynewDB
$ psql mynewDB < /packup/path/backupfilename.db.dump

If you have problems while applying the dump, you can enable stop on error:

$ psql --set ON_ERROR_STOP=on myneDB < /packup/path/backupfilename.db.dump

OR a better way is to use the parameter ‘-1’ or ‘–single-transaction’

$ psql -1 ON_ERROR_STOP=on myneDB < /packup/path/backupfilename.db.dump

Upgrade from Postgresversion X to N

upgrade from e.g. 9.6 to 11

installed postgres versions 9.6, 10, 11

disable monitoring (e.g. for monit)

$ monit unmonitor postgres

stop postgres services first

$ systemctl stop postgresql.service

verify that all postgres services are down

$ ps afxj | grep postgres

drop default installed data in new DBs

$ pg_dropcluster --stop 10 main
$ pg_dropcluster --stop 11 main

start migration from 9.6 to newest installed version

$ pg_upgradecluster -m upgrade 9.6 main

output of the migration, wait till it says its done

Disabling connections to the old cluster during upgrade...
Restarting old cluster with restricted connections...
Stopping old cluster...
Creating new PostgreSQL cluster 11/main ...
.
.
.
Disabling automatic startup of old cluster...
Configuring old cluster to use a different port (5433)...
Running finish phase upgrade hook scripts ...
vacuumdb: processing database "<DBNAME>": Generating minimal optimizer statistics (1 target)
.
.
.
vacuumdb: processing database "<DBNAME>": Generating default (full) optimizer statistics

Success. Please check that the upgraded cluster works. If it does,
you can remove the old cluster with
    pg_dropcluster 9.6 main

Ver Cluster Port Status Owner    Data directory               Log file
9.6 main    5433 down   postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log
Ver Cluster Port Status Owner    Data directory              Log file
11  main    5432 online postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log

after the migration yill get the message that you can now drop the old main data

$ pg_dropcluster 9.6 main

after this is done, you can safely remove the old packages

$ apt purge postgresql-9.6 postgresql-10

change configuration link in etc

$ rm /etc/postgres/main
$ ln -s /etc/postgres/11 /etc/postgres/main

stop the new cluster if running and restart it using the general postgresql service

$ pg_ctlcluster 11 main stop
$ systemctl start postgresql.service

verify that postgres is up and running

$ systemctl status postgresql@11-main.service
● postgresql@11-main.service - PostgreSQL Cluster 11-main
   Loaded: loaded (/lib/systemd/system/postgresql@.service; enabled-runtime; vendor preset: ena
   Active: active (running) since Thu 2018-12-13 00:03:10 CET; 12min ago
  Process: 13327 ExecStart=/usr/bin/pg_ctlcluster --skip-systemctl-redirect 11-main start (code
 Main PID: 13332 (postgres)
    Tasks: 8 (limit: 4915)
   Memory: 30.8M
   CGroup: /system.slice/system-postgresql.slice/postgresql@11-main.service

Needed for older postgres versions (<15) after that, it is part of the upgrade hook

after postgres started, recreate the optimizer statistics

$ /usr/lib/postgresql/11/bin/vacuumdb --all --analyze-in-stages

enable monitoring again (e.g. for monit)

$ monit unmonitor postgres

Docu review done: Wed 31 Jul 2024 02:37:20 PM CEST

qrencode

Table of Content

• General
• Install
• Samples
  • Output to in terminal
  • Output to file

General

Generates QR-Codes from with terminal with qrencode and outputs it into the term, or saves it toa file

Install

To install qrencode with apt, you can just run:

$ apt install qrencode

Samples

Output to in terminal

$ qrencode -l H -t UTF8 "STRING"

Output to file

$ qrencode -t EPS -o qrcode.eps 'Hello World!'

Docu review done: Tue 17 Oct 2023 10:55:07 AM CEST

Table of Content

• General
• Installation
• Aliases
• Userscripts
  • Small helper scripts
    • showSSL

General

qutebrowser is a keyboard-focused browser with a minimal GUI. It’s based on Python and Qt5 and is free software, licensed under the GPL. It was inspired by other browsers/addons like dwb and Vimperator/Pentadactyl.

Installation

If you are running debian, you can easily install it with apt like:

$ apt install qutebrowser

Aliases

Inside of qutebrowser you can create your own aliases which can help you in performing long commands way faster and easier.

You can configure your aliases in two different files:

• qutebrowser/config.py
• qutebrowser/config/autoconfig.yml

As I configure new changes of my qutebrowser always inside of it self, so get it stored in both ;) and it works like perfect which then goes into my automation to deploy it on all other clients.

Inside of qutebrowser, you would do it like this (sample with default aliases):

:set aliases '{"q": "close", "qa": "quit", "w": "session-save", "wq": "quit --save", "wqa": "quit --save", "my_new_alias": "qute_command with out starting double_dots"}'

Lets see how it is done in the files directly:

For qutebrowser/config.py add the following line :

c.aliases = {'my_new_alias': 'qute_command with out starting double_dots', 'q': 'close', 'qa': 'quit', 'w': 'session-save', 'wq': 'quit --save', 'wqa': 'quit --save'}

For qutebrowser/config/autoconfig.yml add a new item to the settings.alias nested hash:

settings:
  aliases:
    global:
      my_new_alias: qute_command with out starting double_dots
      q: close
      qa: quit
      w: session-save
      wq: quit --save
      wqa: quit --save

Userscripts

Userscripts are executables files (executable permission required) which can be triggered by qutebrowser. These allow you to run external commands/scripts on your local system and interact with your system and/or with qutebrowser. Userscripts are loaced in your config/data directory, for example beneath ~/.local/share/qutebrowser/userscripts or ~/.confg/qutebrowser/userscripts. If you want to define a global one, you can place it beneath /usr/share/qutebroswer/userscripts. For more details, please have a look in qute://help/userscripts.html (inside of qutebrowser).

Small helper scripts

showSSL

As qutebrowser is not able to display the certificates of the current homepage, we have created a small and simple script which can help out there.

This script will display you the certificate chain and also the certificate which got fetched from the destination.

One thing to mention: the script will create a separate connection to the domain, so it can be that the scripts connects to a different server as it could be that there is a round robin behind the domain or something else which would let you target a different TLS termination destination.

This is the content of the script ~/.config/qutebrowser/userscripts/showSSL :

#!/bin/bash
tg_url=$(sed -E 's@https://([a-z0-9._-]+)/.*@\1@g' <<<"${QUTE_URL}")
ssl_data=$(echo | openssl s_client -connect ${tg_url}:443 -showcerts 2>/dev/null)
echo "${ssl_data}"
echo "$(openssl x509 -text <<<"${ssl_data}")"
echo "open -r -b qute://process/$$" >> "${QUTE_FIFO}"

To get the latest version of the script please have a look at https://gitea.sons-of-sparda.at/outerscripts/qutebrowser_scripts

As you can see, the script uses openssl, so please make sure you have it installed on your local client.

To make easy use of it, you can create an alias inside of qutebrowser, e.g: "showssl": "spawn --userscript showSSL"

Docu review done: Wed 31 Jul 2024 02:37:37 PM CEST

recordmydesktop

General

Notice also, that if any option is entered you have to specify the output file with the -o switch.

If you try to save under a filename that already exists, the name will be post-fixed with a number (incremented if that name exists already)

Commands

Record a section

$ recordmydesktop --no-sound -x X_pos -y Y_pos --width WIDTH --height HEIGHT -o foo.ogv
# where X_pos and Y_pos specify the offset in pixels from the upper left
# corner of your screen and WIDTH and HEIGHT the size of the window to be recorded(again in pixels).
# If the area extends beyond your current resolution, you will be notified appropriately and nothing will happen.

Docu review done: Wed 31 Jul 2024 02:37:54 PM CEST

rawtherapee

Table of content

• General
• Installation
• HDR convert over CLI

General

RawTherapee is an advanced program for developing raw photos and for processing non-raw photos. It is non-destructive, makes use of OpenMP, supports all the cameras supported by dcraw and more, and carries out its calculations in a high precision 32-bit floating point engine.

• Documentation: http://rawpedia.rawtherapee.com/
• Forum: https://discuss.pixls.us/c/software/rawtherapee
• Code and bug reports: https://github.com/Beep6581/RawTherapee

It can be also used to generate HDR images out of RAW files. This can be done inside the graphical interface and also with the command line interface.

Installation

If you are running Debian for example, you can just use apt to install it.

$ apt install rawtherapee

This will also install you the CLI binary rawtherapee-cli

If you are more into AppImage installation, you can go to there download page https://rawtherapee.com/downloads/ and download it there. They also offer download links for setup files used within Windows and dmg files for macOS.

HDR convert over CLI

To convert RAW (e.g. .nef) files into HDR .png files, without any special settings, you could use the rawtherapee-cli. This is quite useful, to perform such actions on a lot of pictures and only manually adopt where it is needed.

In the below sample, we have a ~800 .nef files, which we want to convert to .png files with best quality, a less compression and 16bit depth per channel and the results get stored in the sub directory ./HDR_pngs.

Keep in mind, this uses quite some disc space

$ rawtherapee-cli -o ./HDR_pngs -n -js3 -b16 -q -c ./DSC_*

To do the same conversion, just from .nef to .jpg, you need to remove some parameters as jpg is the default output format.

$ rawtherapee-cli -o ./HDR_pngs -js3 -q -c ./DSC_*

The default compression for .jpg is set to 92 ( = -js92 ) and bit depth fixed to 8bit, does not matter if you would set the parameter -b16.

Docu review done: Wed 31 Jul 2024 02:38:08 PM CEST

ripperx

General

ripperx small ripping tool from CD to mp3/waf with flac encoding + cddb to fetsh metadata for disc (like title,author,…)

Docu review done: Mon 20 Feb 2023 11:04:03 CET

samba

Table of Content

• Config test
• audit logging
• Enable smb1

Config test

To run a config syntax check you can execute testparm without any parameter and you will getsomething like this:

$ testparm
Registered MSG_REQ_POOL_USAGE
Registered MSG_REQ_DMALLOC_MARK and LOG_CHANGED
Load smb config files from /etc/samba/smb.conf
Processing section "[myshare1]"
Processing section "[myshare2]"
Processing section "[myshare3]"
Processing section "[myshare4]"
.
.
.
Loaded services file OK.

Server role: ROLE_STANDALONE

Press enter to see a dump of your service definitions

[myshare1]
        browseable = No
        comment = my share 1
        include = /etc/samba/include/myshare1.conf
        path = /data/myshare1
        read only = No
        valid users = share

[myshare2]
        browseable = No
        comment = my share 2
        include = /etc/samba/include/myshare2.conf
        path = /data/myshare2
        read only = No
        valid users = share
.
.
.

vfs audit logging

vfs audit or vfs full_audit allows you to track down who is doing what on your shares in a very simple way.

To enable it on your share, make sure you have it installed, e.g. for debian you will see the package samba-vfs-modules installed and if you can execute man vfs_full_audit you are on a good position ;)

To enable it in samba, you have to create a small configuration inside the [global] section, which could look like this for example:

vfs objects         = full_audit
full_audit:facility = local7
full_audit:priority = debug
full_audit:prefix   = %u|%I
full_audit:success  = chmod chown link lock open rename rmdir unlink write
full_audit:failure  = chmod chown link lock open rename rmdir unlink write chdir fchmod fchown fsync ftruncate getlock kernel_flock readdir

What is the config above doing:

• objects : specifies the object of vfs you can also use audit if you really need a very small set of information
• facility : specifies the logfacility where to send the logs
• priority : as it says the log priority
• prefix: it allows you to add some prefix to the log original (%u for user and %I for client IP), these variables can be looked up at man smb.conf
• success: filters to the given types of loges, it allso allowes !<type> to disable a specific one + to log everything you can just define all
• failure: same as success, it is a filter for failed logs

After you have configured it based on your needs, you have to restart the samba service.

A reload on its own is to less.

And dont forget to create the log configuration e.g. in your rsyslog or syslog service. You can place a filter, the application/program name is smbd_audit

Whe you have done all the things, you will get something like this:

Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|chdir|ok|chdir|testing/1
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|chdir|ok|chdir|/data/myshare1/testdir1
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|rmdir|ok|testing/1
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|unlink|ok|/data/myshare1/testdir1/testing/t1
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|chdir|ok|chdir|testing
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|open|ok|r|/data/myshare1/testdir1/testing
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|chdir|ok|chdir|/data/myshare1/testdir1
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|chdir|ok|chdir|testing
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|chdir|ok|chdir|/data/myshare1/testdir1
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|rmdir|ok|testing
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|readdir|ok|
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|open|ok|w|/data/myshare1/testdir1/asdf
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|kernel_flock|ok|/data/myshare1/testdir1/asdf
Jan 01 13:37:42 my_samba_server01 smbd_audit[3362]: share|13.37.42.69|kernel_flock|ok|/data/myshare1/testdir1/asdf

Lets have a look at the structure

Enable smb1

Add the following line to the smb.conf

ntlm auth = ntlmv1-permitted

After you have done that, restart the samba service

$ systemctl restart smbd.service

Docu review done: Wed 31 Jul 2024 02:38:19 PM CEST

screen

General

shell multiplexer in one session

Commands

Multiusermode inside of screen

$ screen -R [name]
Ctrl+a :multiuser on                        # to enable the multiuser mode
Ctrl+a :acladd [username]                   # permissts second user
Ctrl+a :layout save default                 # saves layout of screen to get it back after rejoining the session

secodnuser > screen -x firstuser/[name]     # to attach the screen session of firstuser

Docu review done: Wed 31 Jul 2024 02:38:34 PM CEST

smartctl

Table of Content

• Table of Contents
• Installation of Smartmontools
• Available Tests
  • ATA SCSI Tests
    • Short Test
    • Long Test
  • ATA specified Tests
    • Conveyance Test
    • Select Tests
• Test procedure with smartctl
• Viewing the Test Results
• References

General

All modern hard drives offer the possibility to monitor its current state via SMART attributes. These values provide information about various parameters of the hard disk and can provide information on the disk’s remaining lifespan or on any possible errors. In addition, various SMART tests can be performed to determine any hardware problems on the disk. This article describes how such tests can be performed for Linux using smartctl (Smartmontools).

Installation of Smartmontools

The Smartmontools can be installed on Ubuntu using the package sources:

$ sudo apt-get install smartmontools

To ensure the hard disk supports SMART and is enabled, use the following command (in this example for the hard disk /dev/sdc):

$ sudo smartctl -i /dev/sdc

Example Output:

smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.5.0-39-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital RE4 Serial ATA
Device Model:     WDC WD5003ABYX-01WERA1
Serial Number:    WD-WMAYP5453158
LU WWN Device Id: 5 0014ee 00385d526
Firmware Version: 01.01S02
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   8
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Mon Sep  2 14:06:57 2013 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

The last two lines are the most important as these indicate whether SMART support is available and enabled.

Available Tests

SMART offers two different tests, according to specification type, for and SCSI devices.[1] Each of these tests can be performed in two modes:

• Foreground Mode
• Background Mode

In Background Mode the priority of the test is low, which means the normal instructions continue to be processed by the hard disk. If the hard drive is busy, the test is paused and then continues at a lower load speed, so there is no interruption of the operation. In Foreground Mode all commands will be answered during the test with a “CHECK CONDITION” status. Therefore, this mode is only recommended when the hard disk is not used. In principle, the background mode is the preferred mode.

ATA SCSI Tests

Short Test

The goal of the short test is the rapid identification of a defective hard drive. Therefore, a maximum run time for the short test is 2 min. The test checks the disk by dividing it into three different segments. The following areas are tested:

• Electrical Properties: The controller tests its own electronics, and since this is specific to each manufacturer, it cannot be explained exactly what is being tested. It is conceivable, for example, to test the internal RAM, the read/write circuits or the head electronics.
• Mechanical Properties: The exact sequence of the servos and the positioning mechanism to be tested is also specific to each manufacturer.
• Read/Verify: It will read a certain area of the disk and verify certain data, the size and position of the region that is read is also specific to each manufacturer.

Long Test

The long test was designed as the final test in production and is the same as the short test with two differences. The first: there is no time restriction and in the Read/Verify segment the entire disk is checked and not just a section. The Long test can, for example, be used to confirm the results of the short tests.

ATA specified Tests

All tests listed here are only available for ATA hard drives.

Conveyance Test

This test can be performed to determine damage during transport of the hard disk within just a few minutes.

Select Tests

During selected tests the specified range of LBAs is checked. The LBAs to be scanned are specified in the following formats:

$ sudo smartctl -t select,10-20 /dev/sdc #LBA 10 to LBA 20 (incl.)
$ sudo smartctl -t select,10+11 /dev/sdc #LBA 10 to LBA 20 (incl.)

It is also possible to have multiple ranges, (up to 5), to scan:

$ sudo smartctl -t select,0-10 -t select,5-15 -t select,10-20 /dev/sdc

Test procedure with smartctl

Before performing a test, an approximate indication of the time duration of the various tests are displayed using the following command:

sudo smartctl -c /dev/sdc

Example output:

[...]
Short self-test routine
recommended polling time:    (   2) minutes.
Extended self-test routine
recommended polling time:    (  83) minutes.
Conveyance self-test routine
recommended polling time:    (   5) minutes.
[...]

The following command starts the desired test (in Background Mode):

$ sudo smartctl -t <short|long|conveyance|select> /dev/sdc

It is also possible to perform an “offline” test.[2] However, only of the standard self test (Short Test) is performed.

Example output:

smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.5.0-39-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: "Execute SMART Short self-test routine immediately in off-line mode".
Drive command "Execute SMART Short self-test routine immediately in off-line mode" successful.
Testing has begun.
Please wait 2 minutes for test to complete.
Test will complete after Mon Sep  2 15:32:30 2013

Use smartctl -X to abort test.

To perform the tests in Foreground Mode a “-C” must be added to the command.

$ sudo smartctl -t <short|long|conveyance|select> -C /dev/sdc

Viewing the Test Results

In general, the test results are included in the output of the following commands:

sudo smartctl -a /dev/sdc

Example:

[...]
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      2089         -
# 2  Extended offline    Completed without error       00%      2087         -
# 3  Short offline       Completed without error       00%      2084         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
[...]

The following command can also be used, if only the test results should are displayed:

$ sudo smartctl -l selftest /dev/sdc

Example output:

smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.5.0-39-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      2089         -
# 2  Extended offline    Completed without error       00%      2087         -
# 3  Short offline       Completed without error       00%      2084         -

In the english Wikipedia article on SMART, a list of known attributes SMART Attributen including a short description is given.

References

1. Hard Drive Self-tests
2. Smartmontools Wiki

Docu review done: Wed 31 Jul 2024 02:39:18 PM CEST

snapd

Table of Content

• General
• Installation
• Aftermath
• Search for snaps
  • Usage
    • Sample
• Offline package install
• List intsalled packges with snap
• List updates for packages
• Upgrade snap packages
  • Usage
    • Sample
• Upgrade snap packages from different channel
• Downgrade snap packages
• List changes made by snap on your system
• Remove snap package

General

Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully.

Snaps are discoverable and installable from the Snap Store, an app store with an audience of millions.

Installation

$ apt install snapd

Aftermath

To use the installed binaries from snap you have to add the installation path to the variable PATH either to .profile,.bashrc,.zshr,..

$ export PATH="/snap/bin:${PATH}"

Search for snaps

Usage

$ snap find searchstring

Sample

$ snap find mdless
Name    Version       Publisher      Notes  Summary
mdless  1.0.10+snap7  arub-islander  -      View Markdown comfortably in the terminal

Install package

$ snap install packagename

Offline package install

Download first the assert and snap file

$ snap download packagename

Install downloaded files manually

$ snap ack packagename.assert
$ snap install packagename.snap

List intsalled packges with snap

$ snap list
Name    Version       Rev   Tracking       Publisher      Notes
core18  20200427      1754  latest/stable  canonical✓     base
mdless  1.0.10+snap7  146   latest/stable  arub-islander  -
snapd   2.44.3        7264  latest/stable  canonical✓     snapd

List updates for packages

$ snap refresh --list

Upgrade snap packages

Snap packages are upgraded automatically For manual upgrade run snap refresh packagename

Usage

$ snap refresh packagename

Sample

$ snap refresh mdless
$ snap "mdless" has no updates available

Upgrade snap packages from different channel

$ snap refresh packagename --channel=channelname

Downgrade snap packages

$ snap revert packagename

List changes made by snap on your system

$ snap changes

Remove snap package

$ snap remove packagename

Docu review done: Tue 17 Oct 2023 10:50:21 AM CEST

Table of Content

• General
• Installation
• Setup
  • DNS config
  • Postfix config

General

Sender Policy Framework (SPF) is an email authentication method designed to detect forging sender addresses during the delivery of the email. SPF alone, though, is limited to detecting a forged sender claim in the envelope of the email, which is used when the mail gets bounced. Only in combination with DMARC can it be used to detect the forging of the visible sender in emails (email spoofing), a technique often used in phishing and email spam.

SPF allows the receiving mail server to check during mail delivery that a mail claiming to come from a specific domain is submitted by an IP address authorized by that domain’s administrators. The list of authorized sending hosts and IP addresses for a domain is published in the DNS records for that domain.

Sender Policy Framework is defined in RFC 7208 dated April 2014 as a “proposed standard”.

Installation

If you are running Debian, you can install the needed packages like this:

$ apt install postfix-policyd-spf-python

Setup

DNS config

To get validate from others, you will need a TXT DNS record on your side, which can look like this:

TXT  @    v=spf1 mx ~all

• TXT indicates this is a TXT record.
• Enter @ in the name field.
• v=spf1 indicates this is an SPF record and the SPF record version is SPF1.
• mx means all hosts listed in the MX records are allowed to send emails for your domain and all other hosts are disallowed.
• ~all indicates that emails from your domain should only come from hosts specified in the SPF record. Emails that are from other hosts will be flagged as untrustworthy. Possible alternatives are +all, -all, ?all, but they are rarely used.

Postfix config

First what you need is to add the SPF policy to the master.cf of postfix.

Add now the line shown below into your master.cf (normally added at the bottom of the file).

policyd-spf  unix  -       n       n       -       0       spawn
    user=policyd-spf argv=/usr/bin/policyd-spf

Next setup is to modify the main.cf of postfix.

Here you have to add two new configuration parameters.

smtpd_recipient_restrictions will alrady have some entries in there, just add there check_policy_service unix:private/policyd-spf

policyd-spf_time_limit = 3600
smtpd_recipient_restrictions =
   permit_mynetworks,
   permit_sasl_authenticated,
   reject_unauth_destination,
   check_policy_service unix:private/policyd-spf

With the first parameter (policyd-spf_time_limit) you specify the timeout setting of the sfp agent and with the second one check_policy_service unix:private/policyd-spf you enable the validation of incoming mails and rejection of unauthorized ones with validation the SPF record

Now it is time to restart the postfix service.

Docu review done: Wed 31 Jul 2024 02:39:49 PM CEST

sqlite

Table of Content

• Open local files

Open local files

Open the file with sqllite direte

$ sqlite3 -column -header ./path/to/sqlitefile

Or, you can first run sqlite without defining the database file and load then the db file

$ sqlite3 -column -header
SQLite version 3.34.1 2021-01-20 14:10:07
Enter ".help" for usage hints.
sqlite>

sqlite> .open ./path/to/sqlitefile

If you have now a look into the loaded databases with by using .database you will see that your newly added one is showing up

sqlite> .databases
main: /home/myawesomeuser/sqlitedbs/path/to/sqlitefile r/w

main indicates that this is the default database

Now lets see what tables are exiting in the database

sqlite> .tables
CHNL     SRV      SRV_FAV  SRV_IP

The rest is standard sql ;)

Docu review done: Wed 31 Jul 2024 02:40:03 PM CEST

strace

Table of Content

• General
• Commands
• Filter by type of syscall
• Examples

General

strace traces system calls and signals and is an invaluable tool for gathering context when debugging.

Commands

Filter by type of syscall

Used by parameter -e

Examples

$ strace -Tfe trace=open,read,write ./my_script.sh
$ strace -fp 1337 -e trace=open,read,write
$ strace -fp 1337 -e trace=file

$ strace -c ls > /dev/null
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 89.76    0.008016           4      1912           getdents
  8.71    0.000778           0     11778           lstat
  0.81    0.000072           0      8894           write
  0.60    0.000054           0       943           open
  0.11    0.000010           0       942           close
  0.00    0.000000           0         1           read
  0.00    0.000000           0       944           fstat
  0.00    0.000000           0         8           mmap
  0.00    0.000000           0         4           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         7           brk
  0.00    0.000000           0         3         3 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           sysinfo
  0.00    0.000000           0         1           arch_prctl
------ ----------- ----------- --------- --------- ----------------
100.00    0.008930                 25440         3 total

Network sample

$ strace -f -e trace=network curl http://google.at

The network trace allows you to get more information about the network traffic of a serivce, but you should think of piping it to a grep or similar commands, as it can be very verbose.

For example, if you are only interested on the IPs which are source/dest, add 2>&1 | grep sin_addr:

strace -f -e trace=network curl http://google.at 2>&1 | grep sin_addr
[pid 780784] connect(7, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("142.250.186.99")}, 16 ) = 0
connect(5, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("142.250.186.99")}, 16) = -1 EINPROGRESS (Operation now in progress)
getsockname(5, {sa_family=AF_INET, sin_port=htons(39392), sin_addr=inet_addr("10.0.1.2")}, [128 => 16]) = 0
getsockname(5, {sa_family=AF_INET, sin_port=htons(39392), sin_addr=inet_addr("10.0.1.2")}, [128 => 16]) = 0
getsockname(5, {sa_family=AF_INET, sin_port=htons(39392), sin_addr=inet_addr("10.0.1.2")}, [128 => 16]) = 0

tang

Table of Content

• General
• Commands
• Installation
• Setup
• Key files
• Rekey tang

General

Tang is a service for binding cryptographic keys to network presence. It offers a secure, stateless, anonymous alternative to key escrow services.

Commands

Installation

$ apt install tang

If you wana monitor it, there is a package called tang-nagios which gives you the script check_tang and allows you to query a running tang server and returns the health state.

Setup

After installing the tang package, you are already able to start the socket tangd.socket.

Keep in mind, that tang listens perdefault on port 80

To adjust the port, have a look in the unit file, there you will find the attribute ListenStream, which controuls the listening port.

If you don’t want to change it directly in the package managed file and use an overwrite instead (would recommend that) you have explicit clrea ListenStream first and add then the replacement.

[Socket]
ListenStream=
ListenStream=8080

To enable the socket and start it right away, run the following command:

$ systemctl enable --now tangd.socket

If you check now with ss, you should see the following:

$ ss -tulpen | grep 8080
tcp   LISTEN     0      4096   *:9090  *:*     users:(("systemd",pid=1,fd=102))         ino:199422916 sk:5d8 cgroup:/system.slice/tangd.socket v6only:0 <->

Key files

The keys files for tang are (at least for Debian) stored beneath /var/lib/tang.

These files can be backuped, in case something breaks and you need to start it up again with the old keys.

Rekey tang

Sometimes it is needed to perform a rekey which you can do in two different ways:

So first, stop tang

$ systemctl stop tangd.socket

Next (first method), lets remove the key files and perform a keygen:

$ rm /var/lib/tang/*jwk
$ /usr/libexec/tangd-keygen /var/lib/tang

If you are not removing the “old” keys before running tangd-keygen you will keep them and tang will also load them.

As second method, we run the rotate keys

$ /usr/libexec/tangd-rotate-keys
Disabled advertisement of key gSFSpnZmWnHcLTAhViARtIWYdw30DtIbTmWqJ24bh3Y.jwk -> .gSFSpnZmWnHcLTAhViARtIWYdw30DtIbTmWqJ24bh3Y.jwk
Disabled advertisement of key zXnckFX8OehQ6-GiQh7nQo7x4jefwlsWvuFbODRfaYA.jwk -> .zXnckFX8OehQ6-GiQh7nQo7x4jefwlsWvuFbODRfaYA.jwk
Created new key RL9twdG6EE4lbHDDCuI2XqlD3iZp57qG9I49flhCpBo.jwk
Created new key XKYTFAqwGyMD9c-kU3XSTGRoFjG9Xv9tEIdSPs-I0nA.jwk
Keys rotated successfully

This also takes care about the old keys.

Still have a look at your socket file, as it might be that tang also loads .*.jwk files.

This allows you to still use the old keys with your client, but new actions will only be taken with the new generated keys.

After all clients using the old keys, you can safely remove the .*.jwk files. But do never remove files while the socket is running and serving data to clients, this can lead to dataloss.

And at last step, we start it again.

$ systemctl start tangd.socket

Docu review done: Wed 31 Jul 2024 02:40:17 PM CEST