### Bash loop constructs Source: https://mywiki.wooledge.org/BashProgramming Provides examples of standard for loops, while loops for file processing, and C-style for loops for numeric iteration. ```bash # Standard for loop for file in ./*; do diff "$file" "Old/$file"; done # While loop for file reading while IFS=: read -r user pwdhash uid gid _; do echo "user $user has UID $uid and primary GID $gid" done < /etc/passwd # Loop over find output find . -type f -name '*.wav' -exec \ sh -c 'for f; do lame "$f" "${f%.wav}.mp3"; done' x {} + # C-style for loop time for ((i=1; i<=10000; i++)); do something; done ``` -------------------------------- ### Bash Loop Structures Source: https://mywiki.wooledge.org/BashSheet Examples of various loop constructs including for, while, until, and select for iterative tasks and process monitoring. ```bash for file in *.mp3; do openssl md5 "$file"; done > mysongs.md5 for file; do cp "$file" /backup/; done for (( i = 0; i < 50; i++ )); do printf "%02d," "$i"; done while read _ line; do echo "$line"; done < file until myserver; do echo "My Server crashed with exit code: $?; restarting it in 2 seconds .."; sleep 2; done select fruit in Apple Pear Grape Banana Strawberry; do (( credit -= 2, health += 5 )); echo "You purchased some $fruit. Enjoy!"; done ``` -------------------------------- ### Launch Interactive Shell with Custom RC File (Bash) Source: https://mywiki.wooledge.org/BashFAQ Explains how to start an interactive Bash shell using a specific startup file other than the default ~/.bashrc. This is achieved using the --rcfile option. ```bash bash --rcfile /path/to/custom/.bashrc ``` -------------------------------- ### Run Command on Files with Specific Extension (Bash) Source: https://mywiki.wooledge.org/BashFAQ Provides an example of how to execute a command on all files that match a specific pattern, such as all files with the .gz extension. This utilizes shell globbing. ```bash command *.gz ``` -------------------------------- ### Demonstrating Arithmetic Code Injection Vulnerability Source: https://mywiki.wooledge.org/BashPitfalls An example showing how unvalidated input passed to an arithmetic expansion can lead to arbitrary command execution. ```bash echo 'a[$(echo injection >&2)]' | bash -c 'read num; echo $((num+1))' ``` -------------------------------- ### Bash Positional Parameters Example Source: https://mywiki.wooledge.org/BashSheet Demonstrates how positional parameters ($1, $2, etc.) are used to access arguments passed to a Bash script or function. It shows how arguments are assigned to these parameters and how quoting affects their expansion. ```bash echo "First argument: $1" echo "Second argument: $2" # Example with quoted argument # ./script "foo bar" hubble # $1 will be "foo bar", $2 will be "hubble" ``` -------------------------------- ### Bash: Trace Variable Origin with Set -x Source: https://mywiki.wooledge.org/BashFAQ Demonstrates how to use Bash's trace mode (`set -x`) to find out where a variable in an interactive shell was set. This is useful for debugging unexpected variable values. Note that this does not work for variables set by programs running before Bash starts. ```bash # Enable trace mode set -x # Your commands here, e.g.: my_variable="some_value" echo "The variable is: $my_variable" # Disable trace mode if needed set +x ``` -------------------------------- ### Bash Handling Octal Input Source: https://mywiki.wooledge.org/BashProgramming Demonstrates how to correctly handle potential octal inputs (leading zeros) in arithmetic contexts by forcing base-10 evaluation. This prevents unexpected behavior due to Bash's default octal interpretation of numbers starting with '0'. ```bash day=$(date +%d) day=$((10#$day)) ``` -------------------------------- ### Defining Bash Variable Types Source: https://mywiki.wooledge.org/BashProgramming Examples showing how to define standard string variables, indexed arrays, and associative arrays in Bash. Associative arrays require explicit declaration using the -A flag. ```bash # String variable a=b # Indexed array my_array=(element1 element2 element3) my_array[42]=foo # Associative array declare -A hash hash[key]=value ``` -------------------------------- ### Correct Usage of tr for Case Conversion Source: https://mywiki.wooledge.org/BashPitfalls Provides correct examples for using the 'tr' command for case conversion, addressing issues with shell globbing and locale-dependent character sets. It offers solutions for converting ASCII letters and locale-aware conversions. ```bash # Use this if you want to change the case of the 26 latin letters LC_COLLATE=C tr A-Z a-z ``` ```bash # Use this if you want the case conversion to depend upon the locale, which might be more like what a user is expecting tr '[:upper:]' '[:lower:]' ``` -------------------------------- ### Quote variables and handle leading dashes Source: https://mywiki.wooledge.org/BashPitfalls Explains the necessity of double-quoting variables to prevent word splitting and using '--' or path prefixes to prevent filenames starting with dashes from being interpreted as command options. ```bash cp -- "$file" "$target" for i in ./*.mp3; do cp "$i" /target done for i in *.mp3; do cp "./$i" /target done ``` -------------------------------- ### Creating and Appending to Indexed Arrays Source: https://mywiki.wooledge.org/BashSheet Demonstrates basic array initialization, handling elements with spaces, using glob patterns for file lists, and appending elements to existing arrays. ```bash myarray=( foo bar quux ) myarray=( "foo bar" quux ) myfiles=( *.txt ) myfiles+=( *.html ) ``` -------------------------------- ### Tilde Expansion with Export Command Source: https://mywiki.wooledge.org/BashPitfalls Illustrates correct methods for using tilde expansion with the 'export' command to set environment variables. It shows problematic and correct syntaxes, including using parameter expansion for robustness. ```bash foo=~/bar; export foo ``` ```bash export foo="$HOME/bar" ``` ```bash export foo="${HOME%/}/bar" ``` -------------------------------- ### Bash: Create a Simple Menu with Select Source: https://mywiki.wooledge.org/BashFAQ Demonstrates how to create a simple menu using the `select` command in Bash. This method is suitable for very basic interactive menus where custom styling is not required. It relies on built-in Bash features. ```bash select item in "Option 1" "Option 2" "Exit"; do case $item in "Option 1") echo "You chose Option 1" ;; "Option 2") echo "You chose Option 2" ;; "Exit") echo "Exiting." break ;; *) echo "Invalid option. Please try again." ;; esac done ``` -------------------------------- ### Bash: Expand Array Elements Source: https://mywiki.wooledge.org/BashSheet Expands elements from an array starting at a specified index and for a given count. If the parameter is '@', it expands positional parameters. ```bash "${@:5}" "${@:2:4}" "${array:start:count}" ``` -------------------------------- ### Bash: Expand Substring Source: https://mywiki.wooledge.org/BashSheet Expands a portion (substring) of a parameter's value, starting at a specified offset and optionally for a given length. The offset is 0-based. ```bash "${line:5}" "${line:5:10}" "${line:offset:length}" ``` -------------------------------- ### Working with Associative Arrays Source: https://mywiki.wooledge.org/BashSheet Demonstrates the creation of associative arrays using 'declare -A' and how to map string keys to values. ```bash declare -A homedirs=( ["Peter"]=~pete ["Johan"]=~jo ["Robert"]=~rob ) homedirs["John"]=~john ``` -------------------------------- ### Bash Special Parameter: $# for Argument Count Source: https://mywiki.wooledge.org/BashSheet Shows how the $# special parameter is used to get the number of positional parameters passed to a script or function. This is often used to check if any arguments were provided. ```bash if (( ! $# )); then echo "No arguments were passed." >&2 exit 1 fi echo "Number of arguments: $#" ``` -------------------------------- ### Bash Input and Output Handling Source: https://mywiki.wooledge.org/BashSheet Demonstrates reading input into variables and formatted output using echo and printf. ```bash read firstName lastName phoneNumber address echo "I really don't like $nick. He can be such a prick." printf "I really don't like %s. He can be such a prick." "$nick" ``` -------------------------------- ### Find Process IDs Safely (Shell) Source: https://mywiki.wooledge.org/BashPitfalls Demonstrates various methods to find process IDs (PIDs) reliably, avoiding common issues like including the grep process itself. It covers `ps`, `pgrep`, `pidof`, and `pkill`, highlighting their usage and potential caveats. ```shell $ ps ax | grep gedit 10530 ? S 6:23 gedit 32118 pts/0 R+ 0:00 grep gedit ``` ```shell ps ax | grep -v grep | grep gedit # will work, but ugly ``` ```shell ps ax | grep '[g]edit' # quote to avoid shell GLOB ``` ```shell $ ps -C gedit PID TTY TIME CMD 10530 ? 00:06:23 gedit ``` ```shell $ pgrep gedit 10530 ``` ```shell $ ps -C gedit | awk '{print $1}' | tail -n1 ``` ```shell $ ps -C gedit -opid= ``` ```shell $ pidof gedit 10530 ``` ```shell $ pgrep -fl firefox 3128 /usr/lib/firefox/firefox 7120 /usr/lib/firefox/plugin-container /usr/lib/flashplugin-installer/libflashplayer.so -greomni /usr/lib/firefox/omni.ja 3128 true plugin ``` -------------------------------- ### Bash: Uppercase Characters in String Source: https://mywiki.wooledge.org/BashSheet Expands a parameter's value after converting specified characters to uppercase. The first example uppercases the first character, the second all characters, and the third specific characters. ```bash "${var^}" "${var^^}" "${var^^[ac]}" ``` -------------------------------- ### Bash Quoting Example Source: https://mywiki.wooledge.org/BashSheet Highlights the importance of quoting variables in Bash to prevent word splitting and globbing. Unquoted expansions can lead to unexpected behavior when parameters contain whitespace or special characters. ```bash filePath="/path/to/my file with spaces" # Without quotes, word splitting occurs echo The file is in: $filePath # With quotes, the entire value is treated as a single string echo "The file is in: $filePath" ``` -------------------------------- ### Navigate to User Home Directory Source: https://mywiki.wooledge.org/BashSheet Changes the current working directory to the specified user's home directory using the tilde expansion. ```bash cd ~lhunath ``` -------------------------------- ### Forced Base 10 Arithmetic in Bash Source: https://mywiki.wooledge.org/BashPitfalls Ensures arithmetic interpretation is strictly base 10, even for potentially negative numbers. Handles cases where the input string might start with a sign. ```bash i=$(( 10#$i )) ``` ```bash i=$(( ${i%%[!+-]*}10#${i#[-+]} )) ``` -------------------------------- ### Bash Arithmetic and Pattern Matching Source: https://mywiki.wooledge.org/BashSheet Shows how to perform arithmetic operations and use the [[ command for advanced pattern matching and tests. ```bash ((completion = current * 100 / total)) [[ $foo = /* ]] && echo "foo contains an absolute pathname." ``` -------------------------------- ### Bash Special Parameter: $$ for Process ID Source: https://mywiki.wooledge.org/BashSheet Explains and shows how to use the $$ special parameter to get the Process ID (PID) of the current Bash script. This is commonly used for creating PID files. ```bash echo "Current script PID: $$" # Example: Creating a PID file # echo "$$" > /var/run/my_script.pid ``` -------------------------------- ### Accessing and Expanding Array Elements Source: https://mywiki.wooledge.org/BashSheet Demonstrates how to retrieve specific elements by index, expand all elements as separate arguments, and merge elements into a single string. ```bash echo "${names[5]}" echo "${names[@]}" (IFS=,; echo "${names[*]}") ``` -------------------------------- ### Idiomatic Exit Status Handling Source: https://mywiki.wooledge.org/BashPitfalls Shows how to test command success directly instead of checking $? and how to use case statements for handling specific exit codes. ```bash if cmd; then # Success logic fi cmd status=$? case $status in 0) echo success >&2 ;; 1) echo 'Must supply a parameter, exiting.' >&2 exit 1 ;; *) echo "Unknown error $status, exiting." >&2 exit "$status" esac ``` -------------------------------- ### Split File into Line Ranges (Bash) Source: https://mywiki.wooledge.org/BashFAQ Shows how to split a file into segments based on line ranges, like lines 1-10, 11-20, etc. It utilizes the POSIX 'split' utility for this purpose. ```bash # Example: Split file into chunks of 100 lines each split -l 100 input_file.txt output_prefix_ ``` -------------------------------- ### Input Redirection with Here-Strings and Here-Documents Source: https://mywiki.wooledge.org/BashSheet Provides methods to feed multi-line or single-line strings directly into a command's standard input without needing external files. ```bash cat <<< "Single line input" cat <&1 | tee log ``` -------------------------------- ### Bash: Handle 'value too great for base' Octal Error Source: https://mywiki.wooledge.org/BashFAQ Illustrates the 'value too great for base' error that occurs in Bash arithmetic when a number with a leading zero (intended as octal) is not a valid octal number (e.g., 09). The example shows the error and implies the need for careful number formatting. ```bash $ echo $((09)) bash: 09: value too great for base (error token is "09") ``` -------------------------------- ### Bash Function: Exporting with find Source: https://mywiki.wooledge.org/BashProgramming Demonstrates exporting a Bash function (`foo`) and then using it with the `find` command to execute it on each file found. This pattern is useful for processing files. ```bash foo() { ...; } export -f foo find . -type f -exec bash -c 'for f; do foo "$f"; done' x {} + ``` -------------------------------- ### Verifying Bash Arithmetic with 'let' Source: https://mywiki.wooledge.org/BashPitfalls A diagnostic snippet demonstrating that the 'let' command remains a reliable way to perform arithmetic on associative arrays even when 'assoc_expand_once' is toggled. ```bash ( shopt -u assoc_expand_once; key=\'\]; typeset -A arr; arr[$key]=0; let 'arr[$key]++'; typeset -p arr BASH_VERSINFO ) ``` -------------------------------- ### Implementing Shell Conditional Statements Source: https://mywiki.wooledge.org/BashPitfalls Demonstrates the correct syntax for 'if' statements, including the use of 'grep' as a condition and the requirement for whitespace when using the '[' command. ```bash if COMMANDS then elif then else fi # Using grep in a condition if grep -q fooregex myfile; then # logic here fi # Correct usage of '[' with whitespace if [ bar = "$foo" ]; then # logic here fi ``` -------------------------------- ### Source External Configuration Files Source: https://mywiki.wooledge.org/BashSheet Executes commands from specified files into the current shell environment. It searches for bashlib in the system PATH and sources .foorc from the local directory. ```bash source bashlib; source ./.foorc ``` -------------------------------- ### Redirect Output of Multiple Commands (Bash) Source: https://mywiki.wooledge.org/BashFAQ Shows how to redirect the standard output of a single command. For multiple commands, techniques like grouping commands within braces or using process substitution might be necessary, depending on the specific redirection needs. ```bash # Redirecting output of a single command command > output.txt # Redirecting output of multiple commands (example using grouping) { command1; command2; } > output.txt ``` -------------------------------- ### Bash Builtin Commands and Declarations Source: https://mywiki.wooledge.org/BashSheet Covers common builtins for variable declaration, environment management, and system checks. ```bash while ! ssh lhunath@lyndir.com; do :; done alias l='ls -al' declare -i myNumber=5 export AUTOSSH_PORT=0 foo() { local bar=fooBar; echo "Inside foo(), bar is $bar"; }; bar=normalBar; foo if ! type -P ssh >/dev/null; then echo "Please install OpenSSH." >&2; exit 1; fi ``` -------------------------------- ### Bash Command Grouping and Subshells Source: https://mywiki.wooledge.org/BashSheet Demonstrates using command groups for logical operators and subshells to isolate environment changes like IFS or directory navigation. ```bash [[ $1 ]] || { echo "You need to specify an argument!" >&2; exit 1; } (IFS=','; echo "The array contains these elements: ${array[*]}") (cd "$1" && tar -cvjpf archive.tbz2 .) ``` -------------------------------- ### POSIX Arithmetic Evaluation with [ ] Source: https://mywiki.wooledge.org/BashPitfalls Shows the POSIX-compliant way to perform arithmetic comparisons using the [ ] (test) command with the -gt operator. Ensures portability when (( )) is not available. ```bash # POSIX [ "$foo" -gt 7 ] # Also right! [ "$((foo > 7))" -ne 0 ] # POSIX-compatible equivalent to ((, for more general math operations. ``` -------------------------------- ### Standard Output and Error Redirection Source: https://mywiki.wooledge.org/BashSheet Demonstrates how to redirect standard output and error streams to files using various operators. These commands allow for overwriting, appending, and merging streams. ```bash echo "Hello" > file.txt echo "Append" >> file.txt command 2> error.log command &> all_output.log command &>> all_output.log ``` -------------------------------- ### Redirecting Command Output to While Loops Source: https://mywiki.wooledge.org/BashPitfalls Compares different methods for feeding command output into a while loop. It recommends using process substitution or pipes to avoid the data modification issues associated with command substitution. ```bash # Recommended: Process Substitution while … done < <(foo) # Alternative: Pipeline foo | while … done # Discouraged: Command Substitution while … done <<< "$(foo)" ``` -------------------------------- ### Iterate over files safely in Bash Source: https://mywiki.wooledge.org/BashPitfalls Demonstrates methods to process files recursively while handling spaces and special characters. Includes using find with exec, process substitution, and bash globstar. ```bash find . -type f -name '*.mp3' -exec some command {} + while IFS= LC_ALL=C read -r -d '' file; do some command "$file" done < <(find . -type f -name '*.mp3' -print0) shopt -u failglob; shopt -s globstar nullglob for file in ./**/*.mp3; do some command "$file" done ``` -------------------------------- ### Bash Command Separators and Execution Source: https://mywiki.wooledge.org/BashSheet Illustrates different ways to separate and execute commands in Bash. This includes sequential execution using semicolons or newlines, asynchronous execution with ampersands, and conditional execution using AND (&&) and OR (||) operators. ```bash # Sequential execution with semicolon command1; command2 ``` ```bash # Sequential execution with newline command1 command2 ``` ```bash # Asynchronous execution (background) command1 & command2 ``` ```bash # AND conditional execution command1 && command2 ``` ```bash # OR conditional execution command1 || command2 ``` -------------------------------- ### Understanding `if` statement with `[` command Source: https://mywiki.wooledge.org/BashPitfalls Explains that the `[` (or `test`) command is a regular command, not a syntax marker, when used with `if` statements. The `if` keyword expects a command to execute. ```bash # Bash if [ "foo" = "bar" ]; then echo "foo equals bar" fi ``` -------------------------------- ### Bash: Comparing Variables with '[[' Source: https://mywiki.wooledge.org/BashPitfalls Explains the use of the Bash/Ksh '[[ ... ]]' construct for variable comparisons, highlighting its advantages over '[' such as automatic handling of word splitting and globbing, and the ability to use '==' for pattern matching. ```bash [[ $foo == bar ]] # Right! ``` ```bash match=b*r [[ $foo == "$match" ]] # Good! Unquoted would also match against the pattern b*r. ``` -------------------------------- ### Safely Processing Files with Globs Source: https://mywiki.wooledge.org/BashPitfalls Demonstrates how to safely iterate over files and prevent them from being interpreted as command options by using relative paths or the -- separator. ```bash shopt -u failglob; shopt -s nullglob for path in ./*; do if [[ ${path##*/} != *.* ]]; then rm "$path" fi done for file in *; do if [[ $file != *.* ]]; then rm "./$file" fi done for file in *.*; do rm "./$file" done # Using -- to signal end of options for file in *; do if [[ $file != *.* ]]; then rm -- "$file" fi done ``` -------------------------------- ### Iterating Over Arrays Source: https://mywiki.wooledge.org/BashSheet Shows how to loop through array values or array keys using standard and associative array syntax. ```bash for file in "${myfiles[@]}"; do echo "$file"; done for index in "${!myfiles[@]}"; do echo "File number $index is ${myfiles[index]}"; done ``` -------------------------------- ### Bash: Quoting Command Substitution in 'cd' Source: https://mywiki.wooledge.org/BashPitfalls Illustrates the importance of quoting command substitutions, such as in the 'cd' command, to prevent word splitting and pathname expansion. Shows the correct nested quoting for safety. ```bash cd -P -- "$(dirname -- "$f")" ``` -------------------------------- ### Add Timestamps to Stream Output Source: https://mywiki.wooledge.org/BashFAQ Demonstrates how to prepend timestamps to each line of a stream using Bash 4.2+ built-in features to avoid the performance overhead of forking external processes. ```bash while IFS= read -r line; do printf '%(%Y-%m-%d %H:%M:%S)T %s\n' -1 "$line" done ``` -------------------------------- ### Correct Redirection of Standard Output and Error Source: https://mywiki.wooledge.org/BashPitfalls Explains the correct order for redirecting both stdout and stderr to a file, emphasizing that redirections are processed left-to-right. ```bash somecmd >>logfile 2>&1 ``` -------------------------------- ### Safe String Formatting with printf (Shell) Source: https://mywiki.wooledge.org/BashPitfalls Illustrates the danger of using variables directly as format strings with `printf`. It shows the correct way to print strings using `%s` to prevent format string exploits, especially when the variable content is not fully controlled. ```shell printf %s "$foo" printf '%s\n' "$foo" ``` -------------------------------- ### Test Bash read behavior with NUL-delimited streams Source: https://mywiki.wooledge.org/BashPitfalls A demonstration script to verify how different versions of Bash handle NUL-delimited input by reading a stream and printing the results in a quoted format. ```bash names() { printf 'FOO\0\315\0\226\0'; } export -f names while IFS= read -rd '' f; do printf '<%q> ' "$f"; done < <(names); echo ``` -------------------------------- ### String Comparison vs. Pattern Matching (Shell) Source: https://mywiki.wooledge.org/BashPitfalls Details the difference between string equality and pattern matching within `[[ ... ]]`. It shows how unquoted variables on the right side of `=` trigger pattern matching, and how quoting ensures literal string comparison. ```shell if [[ $foo = "$bar" ]] ``` ```shell [[ $foo = "*.glob" ]] # Wrong! *.glob is treated as a literal string. [[ $foo = *.glob ]] # Correct. *.glob is treated as a glob-style pattern. ``` -------------------------------- ### Logical AND/OR/NOT in Shell Patterns (Bash) Source: https://mywiki.wooledge.org/BashFAQ Explains that standard shell globs are not very powerful for complex pattern matching. For more advanced logic, shell conditional expressions or external tools are typically used. ```bash # Example using extended globbing (Bash specific) shopt -s extglob ls !(file1.txt|file2.txt) shopt -u extglob ``` -------------------------------- ### Embedding Text Blocks in Shell Scripts Source: https://mywiki.wooledge.org/BashPitfalls Demonstrates efficient ways to embed multi-line text in scripts using cat, echo, and printf. These methods avoid common errors associated with using echo with here-documents. ```bash cat <