Keyboard shortcuts
Command Line Editing
Commands For Moving
Mastering The Linux Shell – Bash Shortcuts Explained (Now With Cheat Sheets)
Bash Keyboard Shortcuts
[CTRL + P] Print previous command (Up arrow) [CTRL + N] Print next command (Down arrow) [CTRL + R] History search [ALT + R] Cancel changes, put back like in the history [! + !] Repeat last command [! + <cmd>] Run last command starting with <cmd> [! + *] Run all argument of previous command [! + $] Run last argument of previous command [ALT + .] Print last argument of previous command [^<p1> + ^<p2>] Run last command, replace <p1> with <p2>
[CTRL + F] Move forward one character (Right arrow) [CTRL + B] Move backward one character (Left arrow) [ALT + F] Move forward one word [ALT + B] Move backward one word [CTRL + A] Move to the start of the line [CTRL + E] Move to the end of the line [CTRL + X + X] Toggle between start of word and cursor
[CTRL + H] Delete previous character from the cursor (= Rubout, like Backspace) [CTRL + D] Delete current character from the cursor (Delete)
[CTRL + K] Cut from the cursor to the end of the line [CTRL + U] Cut from the cursor to the start of the line (like [CTRL + X + BACKSPACE] [ALT + D] Cut from the cursor to the end of the current word [CTRL + W] Cut from the cursor to the previous whitespace [CTRL + Y] Paste
[ALT + T] Swap current word with previous (or [ESC + T]) [CTRL + T] Swap the last two characters before the cursor (typo) [ALT + U] Upper-case word and move to the end of the word [ALT + L] Lower-case word and move to the end of the word [ALT + C] Capitalize word and move to the end of the word
Job ID / Percent sign (%)
Percent sign (%) in front of shell command
$ tail -f /var/log/messages [CTRL + Z] [1]+ Stopped tail -f /var/log/messages $ tail -f /var/log/maillog [CTRL + Z] [2]+ Stopped tail -f /var/log/maillog $ jobs [1]- Stopped tail -f /var/log/messages [2]+ Stopped tail -f /var/log/maillog $ fg %2 tail -f /var/log/maillog [CTRL + Z] [2]+ Stopped tail -f /var/log/maillog $ fg %1 tail -f /var/log/messages [CTRL + Z] [1]+ Stopped tail -f /var/log/messages $ kill %1 %2 [1]- Terminated tail -f /var/log/messages [2]+ Terminated tail -f /var/log/maillog
Simple commands vs Compound commands
What is the essential difference between compound command and normal command in bash?
four types of compound commands:
Funktion | Erklärung |
Group: {...;} |
can be used to group simple commands together to form a compound command. |
Subshell: (...) |
is similar to a group except that the commands are run in subshell environment. |
Arithmetic Expression: ((..)) |
a series of comma-separated arithmetic calculations may be performed. |
Test Command: [[...]] |
advanced form of the test command. |
Bash Brackets Quick Reference
Bash Brackets Quick Reference
How to use double or single brackets, parentheses, curly braces
When do we need curly braces around shell variables?
Differences Between Single and Double Brackets in Bash
( Single Parentheses ) (( Double Parentheses )) <( Angle Parentheses ) $( Dollar Single Parentheses ) $( Dollar Single Parentheses Dollar Q )$? $(( Dollar Double Parentheses )) [ Single Square Brackets ] [[ Double Square Brackets ]] { Single Curly Braces } ${Dollar Braces} <<Double Angle Heredocs
if [ ]
File based
Primary | Meaning |
---|---|
[ -a FILE ] | True if FILE exists. |
[ -b FILE ] | True if FILE exists and is a block-special file. |
[ -c FILE ] | True if FILE exists and is a character-special file. |
[ -d FILE ] | True if FILE exists and is a directory. |
[ -e FILE ] | True if FILE exists. |
[ -f FILE ] | True if FILE exists and is a regular file. |
[ -g FILE ] | True if FILE exists and its SGID bit is set. |
[ -h FILE ] | True if FILE exists and is a symbolic link. |
[ -k FILE ] | True if FILE exists and its sticky bit is set. |
[ -p FILE ] | True if FILE exists and is a named pipe (FIFO). |
[ -r FILE ] | True if FILE exists and is readable. |
[ -s FILE ] | True if FILE exists and has a size greater than zero. |
[ -t FD ] | True if file descriptor FD is open and refers to a terminal. |
[ -u FILE ] | True if FILE exists and its SUID (set user ID) bit is set. |
[ -w FILE ] | True if FILE exists and is writable. |
[ -x FILE ] | True if FILE exists and is executable. |
[ -O FILE ] | True if FILE exists and is owned by the effective user ID. |
[ -G FILE ] | True if FILE exists and is owned by the effective group ID. |
[ -L FILE ] | True if FILE exists and is a symbolic link. |
[ -N FILE ] | True if FILE exists and has been modified since it was last read. |
[ -S FILE ] | True if FILE exists and is a socket. |
[ FILE1 -nt FILE2 ] | True if FILE1 has been changed more recently than FILE2, or if FILE1 exists and FILE2 does not. |
[ FILE1 -ot FILE2 ] | True if FILE1 is older than FILE2, or is FILE2 exists and FILE1 does not. |
[ FILE1 -ef FILE2 ] | True if FILE1 and FILE2 refer to the same device and inode numbers. |
String based
Primary | Meaning |
---|---|
[ -o OPTIONNAME ] | True if shell option “OPTIONNAME” is enabled. |
[ -z STRING ] | True of the length if “STRING” is zero. |
[ -n STRING ] or [ STRING ] | True if the length of “STRING” is non-zero. |
[ STRING1 == STRING2 ] | True if the strings are equal. “=” may be used instead of “==” for strict POSIX compliance. |
[ STRING1 != STRING2 ] | True if the strings are not equal. |
[ STRING1 < STRING2 ] | True if “STRING1” sorts before “STRING2” lexicographically in the current locale. |
[ STRING1 > STRING2 ] | True if “STRING1” sorts after “STRING2” lexicographically in the current locale. |
[ ARG1 OP ARG2 ] | “OP” is one of -eq, -ne, -lt, -le, -gt or -ge. These arithmetic binary operators return true if “ARG1” is equal to, not equal to, less than, less than or equal to, greater than, or greater than or equal to “ARG2”, respectively. “ARG1” and “ARG2” are integers. |
Process List (ps)
$ ps auxf
$ ps auxd
#!/usr/bin/env
Howto Make Script More Portable With #!/usr/bin/env As a Shebang
#!/usr/bin/env bash rather than #!/bin/bash
Bash Conditional Expressions
The Set Builtin
Glob and Wildcard
What if a Wildcard Doesn’t Match?
Why does using double brackets “[[” to check if wildcard matched files exists fail, while single brackets “[” work?
Test if there are files matching a pattern in order to execute a script
Test whether a glob has any matches in Bash
Think You Understand Wildcards? Think Again.
how to handle bash * matching when there are no matches?
shopt -s nullglob set -- *.txt if [ "$#" -gt 0 ]; then ./script "$@" # call script with that list of files. fi # Or with bash arrays so you can keep the arguments: files=( *.txt ) # apply C-style boolean on member count (( ${#files[@]} )) && ./script "${files[@]}" # Or with Bash built-in function compgen -G "<glob-pattern>" if compgen -G "/tmp/someFiles*" > /dev/null; then echo "Some files exist." fi
Arrays
6.7 Arrays
You don’t know Bash: An introduction to Bash arrays
- Any element of an array may be referenced using
${name[subscript]}
. - The braces are required to avoid conflicts with the shell’s filename expansion operators.
- If the subscript is
‘@’
or‘*’
, the word expands to all members of the array name
Funktion | Erklärung |
arr=() |
Create an empty array |
arr=(1 2 3) |
Initialize array |
${arr[2]} |
Retrieve third element |
${arr[@]} |
Retrieve all elements |
${!arr[@]} |
Retrieve array indices |
${#arr[@]} |
Calculate array size |
arr[0]=3 |
Overwrite 1st element |
arr+=(4) |
Append value(s) |
arr=( $(ls) ) |
Save ls output as an array of files |
${arr[@]:s:n} |
Retrieve n elements starting at index s |
Control Structures
if
Bash if elif else Statement: A Comprehensive Tutorial
Syntax | What it is | When to use |
Subshell executed in a subprocess | When the commands affect the current shell or environment. The changes do not remain when the subshell completes. | |
Bash extension | Use for arithmetic operations and C-style variable manipulation. | |
POSIX builtin | Comparing numbers and testing whether a file exists. | |
Bash extension | String matching a wildcard pattern. |
while
while [ ]; do command done while [[ ]]; do command done while (( )); do command done
until
until [condition] do block of code done
for
Parameter
Parameter | Purpose |
$0, $1, $2, etc. |
Positional parameters, passed from command line to script, passed to a function, or set to a variable. |
$# |
Number of command-line arguments or positional parameters |
$* |
All of the positional parameters, seen as a single word. |
$@ |
Same as $*, but each parameter is a quoted string, that is, the parameters are passed on intact, without interpretation or expansion. This means, among other things, that each parameter in the argument list is seen as a separate word. |
Parameter | Purpose |
$0, $1, $2, ... |
The positional parameters starting from parameter 0. Parameter 0 refers to the name of the program that started bash, or the name of the shell script if the function is running within a shell script. See the bash man pages for information on other possibilities, such as when bash is started with the -c parameter. A string enclosed in single or double quotes will be passed as a single parameter, and the quotes will be stripped. In the case of double quotes, any shell variables such as $HOME will be expanded before the function is called. You will need to use single or double quotes to pass parameters that contain embedded blanks or other characters that might have special meaning to the shell. |
$* |
The positional parameters starting from parameter 1. If the expansion is done within double quotes, then the expansion is a single word with the first character of the IFS special variable separating the parameters, or no intervening space if IFS is null. The default IFS value is a blank, tab, and newline. If IFS is unset, then the separator used is a blank, just as for the default IFS. |
$@ |
The positional parameters starting from parameter 1. If the expansion is done within double quotes, then each parameter becomes a single word, so that “$@” is equivalent to “$1” “$2” … If your parameters are likely to contain embedded blanks, you will want to use this form. |
$# |
The number of parameters, not including parameter 0. |
Internal Variables
Advanced Bash-Scripting Guide: 9.1. Internal Variables
Variable | Purpose |
$BASHPID |
Process ID of the current instance of Bash. This is not the same as the $$ variable, but it often gives the same result. |
$PS1 |
This is the main prompt, seen at the command-line. |
$PS2 |
The secondary prompt, seen when additional input is expected. It displays as “>”. |
$PS3 |
The tertiary prompt, displayed in a select loop. |
$PS4 |
The quartenary prompt, shown at the beginning of each line of output when invoking a script with the -x [verbose trace] option. |
$PWD |
Working directory (directory you are in at the time). |
$OLDPWD |
Old working directory. |
$HOME |
PHome directory of the user. |
$- |
Flags passed to script. |
$! |
PID (process ID) of last job run in background. |
$_ |
Special variable set to final argument of previous command executed. |
$? |
Exit status of a command, function, or the script itself. |
$$ |
Process ID (PID) of the script itself. The $$ variable often finds use in scripts to construct “unique” temp file names. |
$ ( echo $$; echo $BASHPID ) 11436 8408
Default values
Expansion | Purpose |
${PARAMETER:-WORD} |
If PARAMETER is unset or null, the shell expands WORD and substitutes the result.The value of PARAMETER is not changed. |
${PARAMETER:=WORD} |
If PARAMETER is unset or null, the shell expands WORD and assigns the result to PARAMETER. This value is then substituted. You cannot assign values to positional parameters or special parameters this way. |
${PARAMETER:?WORD} |
If PARAMETER is unset or null, the shell expands WORD and writes the result to standard error. If WORD is not present a message is written instead. If the shell is not interactive, it exits. |
${PARAMETER:+WORD} |
If PARAMETER is unset or null, nothing is substituted. Otherwise the shell expands WORD and substitutes the result. |
APP_NAME=${APP_NAME:-"default_name"} $ export FOO=first $ echo "The ${FOO:-second} choice" The first choice $ unset FOO $ echo "The ${FOO:-second} choice" The second choice $ export FOO= $ echo "The ${FOO:-second} choice" The second choice
Default values
bash assign default value
Field splitting / Word splitting / String splitting
What is word splitting? Why is it important in shell programming?
- Early shells had only a single data type: strings.
- To store a list of file names in a variable, you would put spaces between them./li>
- At the time, spaces in file names were either forbidden or widely considered Not Done.
list="a b c" for entry in list; do echo $entry done
- These days, spaces in file names are something you need to cope with
- Always use double quotes, i.e. write “$foo”, unless you understand why you need word splitting
- The term word splitting also called field splitting, because what constitutes a word (also called field) can be configured by setting the
IFS
variable - By default,
IFS
contains basic whitespace characters (ASCII space, tab and newline)
How to Split a String Into an Array in Bash
- A for loop construct in
Bash
can split a string value and iterate over the tokens. - The for loop performs string splitting based on the characters defined in the IFS shell variable.
- Splitting a string into an array is a common task.
- The sentence was split into words and the words were added to the words array correctly
sentence='Hello World of Linux' words=() for i in $sentence; do words+=($i) ; done for word in ${words[@]}; do echo $word ; done # Result: Hello World of Linux
- The for loop uses the characters in the
IFS
shell variable to split strings. IFS
stands for Internal Field Separator andBash
uses it to recognize fields.- Its value defaults to
" \t\n"
, meaning that the shell uses the space, the tab, or the newline characters to split a string value into fields or words.
IFS=$IFS# printf "%q" "$IFS" # Result ' \t\n#' sentence='Hello#World#of#Linux' words=() for i in $sentence; do words+=($i) ; done for word in ${words[@]}; do echo $word ; done # Result: Hello World of Linux
- Finally, we can reset IFS to its initial value by removing the # character:
- We extract a three-character length substring of
IFS
, starting from position 0, and assigned it back to theIFS
variable
IFS="${IFS:0:3}" $ printf '%q' "${IFS}" # Result: ' \t\n'
Single (‘) and Double (“) Quotes
Difference between single and double quotes in Bash
Parameter Expansion
Parameter expansion
Parameter Expansion
- Simple usage
- Indirection
- Case modification
- Variable name expansion
- Substring removal
- Search and replace
- String length
- Substring expansion
- Use a default value
- Assign a default value
- Use an alternate value
- Display error if null or unset
Manipulating Strings
Bash : removing part of a string (= Parameter Expansion)
Substring Removal (Abschneiden von Mustern)
Eine gewöhnungsbedürftige, aber doch sehr nette Funktion ist das Herausschneiden bestimmter Muster aus der Zeichenkette einer Variablen.
Funktion | Erklärung |
${variable%muster} |
Entfernt rechts das kleinste passende Stück. |
${variable%%muster} |
Entfernt rechts das größte passende Stück. |
${variable#muster} |
Entfernt links das kleinste passende Stück. |
${variable##muster} |
Entfernt links das größte passende Stück. |
From the beginning: path1="/usr/local/bin/bash" ${PARAMETER#PATTERN} => shortest matching: ${path1#/*} => usr/local/bin/bash ${PARAMETER##PATTERN} => longest matching: ${path1##*/} => bash From the end: path2="x/usr/local/bin/bash" ${PARAMETER%PATTERN} => shortest matching: ${path2%/*} => x/usr/local/bin ${PARAMETER%%PATTERN} => longest matching: ${path2%%/*} => x var="Memory Used: 19.54M" var=${var#*: } # Remove everything up to a colon and space var=${var%M} # Remove the M at the end
Substring Replacement
Funktion | Erklärung |
${string/substring/replacement} |
Replace first match of $substring with $replacement. |
${string//substring/replacement} |
Replace all matches of $substring with $replacement. |
${string/#substring/replacement} |
If $substring matches front end of $string, substitute $replacement for $substring. |
${string/%substring/replacement} |
If $substring matches back end of $string, substitute $replacement for $substring. |
stringZ=abcABC123ABCabc echo ${stringZ/#abc/XYZ} # XYZABC123ABCabc # Replaces front-end match of 'abc' with 'XYZ'. echo ${stringZ/%abc/XYZ} # abcABC123ABCXYZ # Replaces back-end match of 'abc' with 'XYZ'.
Lists of Commands, Command Sequences
Lists of Commands
Writing Better Shell Scripts – Part 2
Meaning of colon in Bash after a double pipe
Bourne Shell Builtins
Is there a difference between how two ampersands and a semi-colon operate in bash?
- Two logical short-circuits are the double ampersand (&&) and double pipe (||) operators.
- The && only allows the command that comes after it in the series to be executed if the previous command exited with a status of 0.
- The || operator does the opposite by only allowing the next command to be executed if the previous one returned a non-zero exit status.
- The ; just separates one command from another.
- The : is a null statement, so it does nothing.
- The . executes a script in the current shell, not starting a new shell
$ true; echo $? 0 $ false; echo $? 1 === && === $ true && echo "hello" hello $ false && echo "hello" <no output> === || === $ true || echo "hello" <no output> $ false || echo "hello" hello === ; === $ true; echo "hallo" hallo $ false; echo "hallo" hallo ======================= [ -n STRING ] => True if the length of "STRING" is non-zero. === return value === $ [ -n "hallo" ]; echo $? 0 $ [ -n "" ]; echo $? 1 === if then === $ if [ -n "hallo" ]; then echo "welt"; fi welt $ if [ -n "" ]; then echo "welt"; fi <no output> === && === $ [ -n "hallo" ] && echo "welt" welt $ [ -n "" ] && echo "welt" <no output> === || === $ [ -n "hallo" ] || echo "welt" <no output> $ [ -n "" ] || echo "welt" welt
testfunc() { return $1 } { echo "success" testfunc 0 echo "success" testfunc 0 echo "failure" testfunc 4 } && { echo "good 1" } || { echo "exception 1" } # Result: exception 1 { echo "success" testfunc 0 echo "success" testfunc 0 echo "failure" testfunc 4 echo "success" testfunc 0 } && { echo "good 2" } || { echo "exception 2" } # Result good 2 # Just test the last function!
Bang dollar-sign
$ ./app a b c d e f $ !* a b c d e f -bash: a: command not found $ !$ f -bash: f: command not found
Brace Expansion
$ cp filename filename-old $ cp filename-old filename
$ cp filename{,-old} $ cp filename{-old,} $ cp filename{-v1,-v2}
Arithmetic Expansion
Bash Math Operations (Bash Arithmetic) Explained
- No
$
in Arithmetic Expansion… - … except that you want to assign/output/store it in the script
- Compound notation
(())
which evaluates the expression - The variable operator
$
to store the result
$((expression)) echo $((2+3)) # Result 5 echo $((x=2, y=3, x+y)) # Result 5 ((x=2, y=3, a=x+y, b=x*y, c=x**y)); echo $a, $b, $c # Result 5, 6, 7
number=1 echo $((++number)) echo $number # Result 2 2 number=1 echo $((number++)) echo $number # Result 1 2 number=1 ((number=number+2)) echo $number # Result 3
number=1 if ((number > 0)); then echo "hurra!" fi idx=0 count=3 while ((idx < count)); do echo $idx ((idx++)) done
Process Substitution
- pipe. Passes the output (stdout) of a previous command to the input (
stdin
) of the next one, or to the shell. This is a method of chaining commands together. - redirection.
scriptname >filename
redirects the output ofscriptname
to filefilename
- process substitution.
Chapter 23. Process Substitution
A Comprehensive Guide to Process Substitution in Bash
Exit code
Exit and Exit Status
Exit Codes With Special Meanings
Understanding Exit Codes and how to use them in bash scripts
$ bla -bash: bla: command not found $ echo $? 127
Exit Code Number | Meaning | Example | Comments |
---|---|---|---|
1 | Catchall for general errors | let “var1 = 1/0” | Miscellaneous errors, such as “divide by zero” and other impermissible operations |
2 | Misuse of shell builtins (according to Bash documentation) | empty_function() {} | Missing keyword or command, or permission problem (and diff return code on a failed binary file comparison). |
126 | Command invoked cannot execute | /dev/null | Permission problem or command is not an executable |
127 | “command not found” | illegal_command | Possible problem with $PATH or a typo |
128 | Invalid argument to exit | exit 3.14159 | exit takes only integer args in the range 0 – 255 (see first footnote) |
128+n | Fatal error signal “n” | kill -9 $PPID of script | $? returns 137 (128 + 9) |
130 | Script terminated by Control-C | Ctl-C | Control-C is fatal error signal 2, (130 = 128 + 2, see above) |
255* | Exit status out of range | exit -1 | exit takes only integer args in the range 0 – 255 |