Docu review done: Fri 26 Jan 2024 04:39:08 PM CET

Table of content

Builtins Commands
- Special Builtins
- COPROC
Network Integration
POSIX Mode
leading zeros and calculations
Custom Tab Completion
Exclamation Mark in commands

Builtin Commands

URL: https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html

Commands which are build in into bash

. : [ alias bg bind break builtin caller cd command compgen complete compopt continue declare dirs disown echo enable eval exec exit export false fc fg getopts hash help history jobs kill let local logout mapfile popd printf pushd pwd read readarray readonly return set shift shopt source suspend test times trap true type typeset ulimit umask unalias unset wait

Each of them can be disabled by running the command

$ enable -n [builtincommand]

To enable it again, use:

$ enable [builtincommand]

For listing all enabled and disable builtin commands you can use enable -a which will give you something like that:

$ enable -a
enable .
enable :
enable [
enable alias
enable bg
enable bind
enable break
enable builtin
enable caller
enable cd
...

Special Builtins

URL: https://www.gnu.org/software/bash/manual/html_node/Special-Builtins.html#Special-Builtins

For historical reasons, the POSIX standard has classified several builtin commands as special. When Bash is executing in POSIX mode, the special builtins differ from other builtin commands in three respects:

Special builtins are found before shell functions during command lookup.
If a special builtin returns an error status, a non-interactive shell exits.
Assignment statements preceding the command stay in effect in the shell environment after the command completes.

When Bash is not executing in POSIX mode, these builtins behave no differently than the rest of the Bash builtin commands. The Bash POSIX mode is described in Bash POSIX Mode.

These are the POSIX special builtins:

break : . continue eval exec exit export readonly return set shift trap unset

COPROC

COPROC is a bash builtin since v4.0. Useful to send commands and get stdout from background commands. Invoke coproc using coproc NAME COMMAND

coproc NAME (sleep 1; echo "foo")

When the coprocess is executed, the shell creates an array variable named NAME in the context of the executing shell. The standard output of command is connected via a pipe to a file descriptor in the executing shell, and that file descriptor is assigned to NAME[0]. The standard input of command is connected via a pipe to a file descriptor in the executing shell, and that file descriptor is assigned to NAME[1]. NAME_PID contains the PID. Send commands to a running process

echo "command" >&"${NAME[1]}"

Read output of a running process

read -r output <&"${NAME[0]}"

All variables created by coproc only exist as long as the process is running, in order to avoid race conditions with whatever is running in the foreground script duplicate them using file descriptors 5 and 6 for example. (of course both commands must be executed as long as the process is still running) The first exec opens the fd, the second one closes it again

coproc NAME (sleep 1; echo "foo")
pid_coproc="${NAME_PID}"
exec 5<&${NAME[0]}
sleep 2
read -ru 5 output
echo "output was: ${output}"
exec 5<&-
wait "${pid_coproc}" || echo "coproc failed with exit code $?"

Network Integration

Bash also allows you to transfer data directly to a target over the network. Just use your destination address as a device, and bash will do the rest for you.

Syntax:

$ <cour_command_which_outputs something> > /dev/<protocol>/<dest-[IP|FQDN]>/<dest-PORT>

TCP

$ echo "asdf" > /dev/tcp/8.8.8.8/53

UDP

$ echo "asdf" > /dev/udp/8.8.8.8/53

Compare between bash and nc

No tweeks or something like that have been done. Testfile has ~10G

$ ls -lah | grep transfer
-rw-r--r--  1 root   root   9.7G Mar 30 15:27 bash.transfer
-rw-r--r--  1 root   root   9.7G Mar 30 15:19 sizefile.transfer

$ time nc -q 0 127.0.0.1 1234321 < ./sizefile

real    0m17.516s
user    0m0.220s
sys     0m12.977s

$ time cat ./sizefile > /dev/tcp/127.0.0.1/1234321

real    0m16.578s
user    0m0.080s
sys     0m11.032s

And what you can see there is, that bash si there alrady a second faster then nc.

POSIX Mode

URL: https://www.gnu.org/software/bash/manual/html_node/Bash-POSIX-Mode.html#Bash-POSIX-Mode

Starting Bash with the --posix command-line option or executing set -o posix while Bash is running will cause Bash to conform more closely to the POSIX standard by changing the behavior to match that specified by POSIX in areas where the Bash default differs.

When invoked as sh, Bash enters POSIX mode after reading the startup files.

The following list is what’s changed when ‘POSIX mode’ is in effect:

Bash ensures that the POSIXLY_CORRECT variable is set.
When a command in the hash table no longer exists, Bash will re-search $PATH to find the new location. This is also available with shopt -s checkhash.
The message printed by the job control code and builtins when a job exits with a non-zero status is ‘Done(status)’.
The message printed by the job control code and builtins when a job is stopped is ‘Stopped(signame)’, where signame is, for example, SIGTSTP.
Alias expansion is always enabled, even in non-interactive shells.
Reserved words appearing in a context where reserved words are recognized do not undergo alias expansion.
The POSIX PS1 and PS2 expansions of ! to the history number and !! to ! are enabled, and parameter expansion is performed on the values of PS1 and PS2 regardless of the setting of the promptvars option.
The POSIX startup files are executed ($ENV) rather than the normal Bash files.
Tilde expansion is only performed on assignments preceding a command name, rather than on all assignment statements on the line.
The default history file is `~/.sh_history`` (this is the default value of $HISTFILE).
Redirection operators do not perform filename expansion on the word in the redirection unless the shell is interactive.
Redirection operators do not perform word splitting on the word in the redirection.
Function names must be valid shell names. That is, they may not contain characters other than letters, digits, and underscores, and may not start with a digit. Declaring a function with an invalid name causes a fatal syntax error in non-interactive shells.
Function names may not be the same as one of the POSIX special builtins.
POSIX special builtins are found before shell functions during command lookup.
When printing shell function definitions (e.g., by type), Bash does not print the function keyword.
Literal tildes that appear as the first character in elements of the PATH variable are not expanded as described above under Tilde Expansion.
The time reserved word may be used by itself as a command. When used in this way, it displays timing statistics for the shell and its completed children. The TIMEFORMAT variable controls the format of the timing information.
When parsing and expanding a ${…} expansion that appears within double quotes, single quotes are no longer special and cannot be used to quote a closing brace or other special character, unless the operator is one of those defined to perform pattern removal. In this case, they do not have to appear as matched pairs.
The parser does not recognize time as a reserved word if the next token begins with a -.
The ! character does not introduce history expansion within a double-quoted string, even if the histexpand option is enabled.
If a POSIX special builtin returns an error status, a non-interactive shell exits. The fatal errors are those listed in the POSIX standard, and include things like passing incorrect options, redirection errors, variable assignment errors for assignments preceding the command name, and so on.
A non-interactive shell exits with an error status if a variable assignment error occurs when no command name follows the assignment statements. A variable assignment error occurs, for example, when trying to assign a value to a readonly variable.
A non-interactive shell exits with an error status if a variable assignment error occurs in an assignment statement preceding a special builtin, but not with any other simple command.
A non-interactive shell exits with an error status if the iteration variable in a for statement or the selection variable in a select statement is a readonly variable.
Non-interactive shells exit if filename in . filename is not found.
Non-interactive shells exit if a syntax error in an arithmetic expansion results in an invalid expression.
Non-interactive shells exit if a parameter expansion error occurs.
Non-interactive shells exit if there is a syntax error in a script read with the . or source builtins, or in a string processed by the eval builtin.
Process substitution is not available.
While variable indirection is available, it may not be applied to the # and ? special parameters.
When expanding the * special parameter in a pattern context where the expansion is double-quoted does not treat the $* as if it were double-quoted.
Assignment statements preceding POSIX special builtins persist in the shell environment after the builtin completes.
Assignment statements preceding shell function calls persist in the shell environment after the function returns, as if a POSIX special builtin command had been executed.
The command builtin does not prevent builtins that take assignment statements as arguments from expanding them as assignment statements; when not in POSIX mode, assignment builtins lose their assignment statement expansion properties when preceded by command.
The bg builtin uses the required format to describe each job placed in the background, which does not include an indication of whether the job is the current or previous job.
The output of kill -l prints all the signal names on a single line, separated by spaces, without the SIG prefix.
The kill builtin does not accept signal names with a SIG prefix.
The export and readonly builtin commands display their output in the format required by POSIX.
The trap builtin displays signal names without the leading SIG.
The trap builtin doesn’t check the first argument for a possible signal specification and revert the signal handling to the original disposition if it is, unless that argument consists solely of digits and is a valid signal number. If users want to reset the handler for a given signal to the original disposition, they should use - as the first argument.
The . and source builtins do not search the current directory for the filename argument if it is not found by searching PATH.
Enabling POSIX mode has the effect of setting the inherit_errexit option, so subshells spawned to execute command substitutions inherit the value of the -e option from the parent shell. When the inherit_errexit option is not enabled, Bash clears the -e option in such subshells.
Enabling POSIX mode has the effect of setting the shift_verbose option, so numeric arguments to shift that exceed the number of positional parameters will result in an error message.
When the alias builtin displays alias definitions, it does not display them with a leading alias unless the -p option is supplied.
When the set builtin is invoked without options, it does not display shell function names and definitions.
When the set builtin is invoked without options, it displays variable values without quotes, unless they contain shell metacharacters, even if the result contains nonprinting characters.
When the cd builtin is invoked in logical mode, and the pathname constructed from $PWD and the directory name supplied as an argument does not refer to an existing directory, cd will fail instead of falling back to physical mode.
When the cd builtin cannot change a directory because the length of the pathname constructed from $PWD and the directory name supplied as an argument exceeds PATH_MAX when all symbolic links are expanded, cd will fail instead of attempting to use only the supplied directory name.
The pwd builtin verifies that the value it prints is the same as the current directory, even if it is not asked to check the file system with the -P option.
When listing the history, the fc builtin does not include an indication of whether or not a history entry has been modified.
The default editor used by fc is ed.
The type and command builtins will not report a non-executable file as having been found, though the shell will attempt to execute such a file if it is the only so-named file found in $PATH.
The vi editing mode will invoke the vi editor directly when the v command is run, instead of checking $VISUAL and $EDITOR.
When the xpg_echo option is enabled, Bash does not attempt to interpret any arguments to echo as options. Each argument is displayed, after escape characters are converted.
The ulimit builtin uses a block size of 512 bytes for the -c and -f options.
The arrival of SIGCHLD when a trap is set on SIGCHLD does not interrupt the wait builtin and cause it to return immediately. The trap command is run once for each child that exits.
The read builtin may be interrupted by a signal for which a trap has been set. If Bash receives a trapped signal while executing read, the trap handler executes and read returns an exit status greater than 128.
Bash removes an exited background process’s status from the list of such statuses after the wait builtin is used to obtain it.

There is other POSIX behavior that Bash does not implement by default even when in POSIX mode. Specifically:

The fc builtin checks $EDITOR as a program to edit history entries if FCEDIT is unset, rather than defaulting directly to ed. fc uses ed if EDITOR is unset.
As noted above, Bash requires the xpg_echo option to be enabled for the echo builtin to be fully conformant.

Bash can be configured to be POSIX-conformant by default, by specifying the --enable-strict-posix-default to configure when building (see Optional Features).

leading zeros and calculations

a=09; ((b=a-3)); echo $b

does not work as bash treats 09 as octal. Use this instead:

a=09; ((b=10#$a-3)); echo $b

note the “10#” before $a

Custom Tab Completion

To create your own tab completion for a script you can do the following thing Create the function which fetches the data like the sample below

This will fetch the host entries from your local .ssh/config

function _ssht_compl_bash() {
    # ensures only 1 time tab completion
    if [ "${#COMP_WORDS[@]}" != "2" ]; then
        return
    fi

    # fill the variable suggestions with the data you want to have for tab completion
    local IFS=$'\n'
    local suggestions=($(compgen -W "$(sed -E '/^Host +[a-zA-Z0-9]/!d;s/Host //g' ~/.ssh/config | sort -u)" -- "${COMP_WORDS[1]}"))

    if [ "${#suggestions[@]}" == "1" ]; then
        # if there's only one match, we remove the command literal
        # to proceed with the automatic completion of the data
        local onlyonesuggestion="${suggestions[0]/%\ */}"
        COMPREPLY=("${onlyonesuggestion}")
    else
        # more than one suggestions resolved,
        # respond with the suggestions intact
        for i in "${!suggestions[@]}"; do
            suggestions[$i]="$(printf '%*s' "-$COLUMNS"  "${suggestions[$i]}")"
        done

        COMPREPLY=("${suggestions[@]}")
    fi
}

Now that we have our function, you just need to attach it to the command which is done with complete -F <functionname> <executeable/alias> For our sample above, it would look like this:

complete -F _ssht_compl_bash ssht

Next setp is to just source the file what you have written in your bash.rc or where ever your bash sources files and start a shell

source ~/.config/bash/ssht_completion.bash

Now you can tab it, have fun

$ ssht<tab><tab>
Display all 1337 possibilities? (y or n)
server1
server2
server3
...

$ ssht server1<tab><tab>
Display all 490 possibilities? (y or n)
server1
server10
server11
...

$ ssht server100<tab>
$ ssht server1001

Exclamation Mark in commands

You for sure had executed some commands which contained an exclamation mark !, like ouput of messages and so on. What can happen is that you got then an error, as the script did something different then what you expected. It can mean, that it reexecuted the last command from the history, which can lead to unwanted behaviour of your script. To get sort out that issue, you just need to add one of the follogwing lines to your script:

#!/bin/bash
set +H
set +o histexpand

This will turn off the history expansion and will work as you intended to.

Sharing is caring