Chapter 10. Loops and Branches

Operations on code blocks are the key to structured and organized shell scripts. Looping and branching constructs provide the tools for accomplishing this.

10.1. Loops

A loop is a block of code that iterates (repeats) a list of commands as long as the loop control condition is true.

for loops

for arg in [list]

This is the basic looping construct. It differs significantly from its C counterpart.

for arg in [list]
do
 command(s)...
done

Note

During each pass through the loop, arg takes on the value of each successive variable in the list.

   1 for arg in "$var1" "$var2" "$var3" ... "$varN"  
   2 # In pass 1 of the loop, arg = $var1	    
   3 # In pass 2 of the loop, arg = $var2	    
   4 # In pass 3 of the loop, arg = $var3	    
   5 # ...
   6 # In pass N of the loop, arg = $varN
   7 
   8 # Arguments in [list] quoted to prevent possible word splitting.

The argument list may contain wild cards.

If do is on same line as for, there needs to be a semicolon after list.

for arg in [list] ; do


Example 10-1. Simple for loops

   1 #!/bin/bash
   2 # Listing the planets.
   3 
   4 for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto
   5 do
   6   echo $planet  # Each planet on a separate line.
   7 done
   8 
   9 echo
  10 
  11 for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto"
  12 # All planets on same line.
  13 # Entire 'list' enclosed in quotes creates a single variable.
  14 do
  15   echo $planet
  16 done
  17 
  18 exit 0

Note

Each [list] element may contain multiple parameters. This is useful when processing parameters in groups. In such cases, use the set command (see Example 14-15) to force parsing of each [list] element and assignment of each component to the positional parameters.


Example 10-2. for loop with two parameters in each [list] element

   1 #!/bin/bash
   2 # Planets revisited.
   3 
   4 # Associate the name of each planet with its distance from the sun.
   5 
   6 for planet in "Mercury 36" "Venus 67" "Earth 93"  "Mars 142" "Jupiter 483"
   7 do
   8   set -- $planet  # Parses variable "planet" and sets positional parameters.
   9   # the "--" prevents nasty surprises if $planet is null or begins with a dash.
  10 
  11   # May need to save original positional parameters, since they get overwritten.
  12   # One way of doing this is to use an array,
  13   #        original_params=("$@")
  14 
  15   echo "$1		$2,000,000 miles from the sun"
  16   #-------two  tabs---concatenate zeroes onto parameter $2
  17 done
  18 
  19 # (Thanks, S.C., for additional clarification.)
  20 
  21 exit 0

A variable may supply the [list] in a for loop.


Example 10-3. Fileinfo: operating on a file list contained in a variable

   1 #!/bin/bash
   2 # fileinfo.sh
   3 
   4 FILES="/usr/sbin/accept
   5 /usr/sbin/pwck
   6 /usr/sbin/chroot
   7 /usr/bin/fakefile
   8 /sbin/badblocks
   9 /sbin/ypbind"     # List of files you are curious about.
  10                   # Threw in a dummy file, /usr/bin/fakefile.
  11 
  12 echo
  13 
  14 for file in $FILES
  15 do
  16 
  17   if [ ! -e "$file" ]       # Check if file exists.
  18   then
  19     echo "$file does not exist."; echo
  20     continue                # On to next.
  21    fi
  22 
  23   ls -l $file | awk '{ print $9 "         file size: " $5 }'  # Print 2 fields.
  24   whatis `basename $file`   # File info.
  25   # Note that the whatis database needs to have been set up for this to work.
  26   # To do this, as root run /usr/bin/makewhatis.
  27   echo
  28 done  
  29 
  30 exit 0

If the [list] in a for loop contains wildcards (* and ?) used in filename expansion, then globbing takes place.


Example 10-4. Operating on files with a for loop

   1 #!/bin/bash
   2 # list-glob.sh: Generating [list] in a for-loop, using "globbing"
   3 
   4 echo
   5 
   6 for file in *
   7 #           ^  Bash performs filename expansion
   8 #+             on expressions that globbing recognizes.
   9 do
  10   ls -l "$file"  # Lists all files in $PWD (current directory).
  11   #  Recall that the wild card character "*" matches every filename,
  12   #+ however, in "globbing," it doesn't match dot-files.
  13 
  14   #  If the pattern matches no file, it is expanded to itself.
  15   #  To prevent this, set the nullglob option
  16   #+   (shopt -s nullglob).
  17   #  Thanks, S.C.
  18 done
  19 
  20 echo; echo
  21 
  22 for file in [jx]*
  23 do
  24   rm -f $file    # Removes only files beginning with "j" or "x" in $PWD.
  25   echo "Removed file \"$file\"".
  26 done
  27 
  28 echo
  29 
  30 exit 0

Omitting the in [list] part of a for loop causes the loop to operate on $@ -- the positional parameters. A particularly clever illustration of this is Example A-16. See also Example 14-16.


Example 10-5. Missing in [list] in a for loop

   1 #!/bin/bash
   2 
   3 #  Invoke this script both with and without arguments,
   4 #+ and see what happens.
   5 
   6 for a
   7 do
   8  echo -n "$a "
   9 done
  10 
  11 #  The 'in list' missing, therefore the loop operates on '$@'
  12 #+ (command-line argument list, including whitespace).
  13 
  14 echo
  15 
  16 exit 0

It is possible to use command substitution to generate the [list] in a for loop. See also Example 15-49, Example 10-10 and Example 15-43.


Example 10-6. Generating the [list] in a for loop with command substitution

   1 #!/bin/bash
   2 #  for-loopcmd.sh: for-loop with [list]
   3 #+ generated by command substitution.
   4 
   5 NUMBERS="9 7 3 8 37.53"
   6 
   7 for number in `echo $NUMBERS`  # for number in 9 7 3 8 37.53
   8 do
   9   echo -n "$number "
  10 done
  11 
  12 echo 
  13 exit 0

Here is a somewhat more complex example of using command substitution to create the [list].


Example 10-7. A grep replacement for binary files

   1 #!/bin/bash
   2 # bin-grep.sh: Locates matching strings in a binary file.
   3 
   4 # A "grep" replacement for binary files.
   5 # Similar effect to "grep -a"
   6 
   7 E_BADARGS=65
   8 E_NOFILE=66
   9 
  10 if [ $# -ne 2 ]
  11 then
  12   echo "Usage: `basename $0` search_string filename"
  13   exit $E_BADARGS
  14 fi
  15 
  16 if [ ! -f "$2" ]
  17 then
  18   echo "File \"$2\" does not exist."
  19   exit $E_NOFILE
  20 fi  
  21 
  22 
  23 IFS=$'\012'       # Per suggestion of Anton Filippov.
  24                   # was:  IFS="\n"
  25 for word in $( strings "$2" | grep "$1" )
  26 # The "strings" command lists strings in binary files.
  27 # Output then piped to "grep", which tests for desired string.
  28 do
  29   echo $word
  30 done
  31 
  32 # As S.C. points out, lines 23 - 30 could be replaced with the simpler
  33 #    strings "$2" | grep "$1" | tr -s "$IFS" '[\n*]'
  34 
  35 
  36 # Try something like  "./bin-grep.sh mem /bin/ls"  to exercise this script.
  37 
  38 exit 0

More of the same.


Example 10-8. Listing all users on the system

   1 #!/bin/bash
   2 # userlist.sh
   3 
   4 PASSWORD_FILE=/etc/passwd
   5 n=1           # User number
   6 
   7 for name in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE" )
   8 # Field separator = :    ^^^^^^
   9 # Print first field              ^^^^^^^^
  10 # Get input from password file               ^^^^^^^^^^^^^^^^^
  11 do
  12   echo "USER #$n = $name"
  13   let "n += 1"
  14 done  
  15 
  16 
  17 # USER #1 = root
  18 # USER #2 = bin
  19 # USER #3 = daemon
  20 # ...
  21 # USER #30 = bozo
  22 
  23 exit 0
  24 
  25 #  Exercise:
  26 #  --------
  27 #  How is it that an ordinary user (or a script run by same)
  28 #+ can read /etc/passwd?
  29 #  Isn't this a security hole? Why or why not?

A final example of the [list] resulting from command substitution.


Example 10-9. Checking all the binaries in a directory for authorship

   1 #!/bin/bash
   2 # findstring.sh:
   3 # Find a particular string in binaries in a specified directory.
   4 
   5 directory=/usr/bin/
   6 fstring="Free Software Foundation"  # See which files come from the FSF.
   7 
   8 for file in $( find $directory -type f -name '*' | sort )
   9 do
  10   strings -f $file | grep "$fstring" | sed -e "s%$directory%%"
  11   #  In the "sed" expression,
  12   #+ it is necessary to substitute for the normal "/" delimiter
  13   #+ because "/" happens to be one of the characters filtered out.
  14   #  Failure to do so gives an error message (try it).
  15 done  
  16 
  17 exit 0
  18 
  19 #  Exercise (easy):
  20 #  ---------------
  21 #  Convert this script to take command-line parameters
  22 #+ for $directory and $fstring.

The output of a for loop may be piped to a command or commands.


Example 10-10. Listing the symbolic links in a directory

   1 #!/bin/bash
   2 # symlinks.sh: Lists symbolic links in a directory.
   3 
   4 
   5 directory=${1-`pwd`}
   6 #  Defaults to current working directory,
   7 #+ if not otherwise specified.
   8 #  Equivalent to code block below.
   9 # ----------------------------------------------------------
  10 # ARGS=1                 # Expect one command-line argument.
  11 #
  12 # if [ $# -ne "$ARGS" ]  # If not 1 arg...
  13 # then
  14 #   directory=`pwd`      # current working directory
  15 # else
  16 #   directory=$1
  17 # fi
  18 # ----------------------------------------------------------
  19 
  20 echo "symbolic links in directory \"$directory\""
  21 
  22 for file in "$( find $directory -type l )"   # -type l = symbolic links
  23 do
  24   echo "$file"
  25 done | sort                                  # Otherwise file list is unsorted.
  26 #  Strictly speaking, a loop isn't really necessary here,
  27 #+ since the output of the "find" command is expanded into a single word.
  28 #  However, it's easy to understand and illustrative this way.
  29 
  30 #  As Dominik 'Aeneas' Schnitzer points out,
  31 #+ failing to quote  $( find $directory -type l )
  32 #+ will choke on filenames with embedded whitespace.
  33 #  Even this will only pick up the first field of each argument.
  34 
  35 exit 0
  36 
  37 
  38 # Jean Helou proposes the following alternative:
  39 
  40 echo "symbolic links in directory \"$directory\""
  41 # Backup of the current IFS. One can never be too cautious.
  42 OLDIFS=$IFS
  43 IFS=:
  44 
  45 for file in $(find $directory -type l -printf "%p$IFS")
  46 do     #                              ^^^^^^^^^^^^^^^^
  47        echo "$file"
  48        done|sort

The stdout of a loop may be redirected to a file, as this slight modification to the previous example shows.


Example 10-11. Symbolic links in a directory, saved to a file

   1 #!/bin/bash
   2 # symlinks.sh: Lists symbolic links in a directory.
   3 
   4 OUTFILE=symlinks.list                         # save file
   5 
   6 directory=${1-`pwd`}
   7 #  Defaults to current working directory,
   8 #+ if not otherwise specified.
   9 
  10 
  11 echo "symbolic links in directory \"$directory\"" > "$OUTFILE"
  12 echo "---------------------------" >> "$OUTFILE"
  13 
  14 for file in "$( find $directory -type l )"    # -type l = symbolic links
  15 do
  16   echo "$file"
  17 done | sort >> "$OUTFILE"                     # stdout of loop
  18 #           ^^^^^^^^^^^^^                       redirected to save file.
  19 
  20 exit 0

There is an alternative syntax to a for loop that will look very familiar to C programmers. This requires double parentheses.


Example 10-12. A C-like for loop

   1 #!/bin/bash
   2 # Two ways to count up to 10.
   3 
   4 echo
   5 
   6 # Standard syntax.
   7 for a in 1 2 3 4 5 6 7 8 9 10
   8 do
   9   echo -n "$a "
  10 done  
  11 
  12 echo; echo
  13 
  14 # +==========================================+
  15 
  16 # Now, let's do the same, using C-like syntax.
  17 
  18 LIMIT=10
  19 
  20 for ((a=1; a <= LIMIT ; a++))  # Double parentheses, and "LIMIT" with no "$".
  21 do
  22   echo -n "$a "
  23 done                           # A construct borrowed from 'ksh93'.
  24 
  25 echo; echo
  26 
  27 # +=========================================================================+
  28 
  29 # Let's use the C "comma operator" to increment two variables simultaneously.
  30 
  31 for ((a=1, b=1; a <= LIMIT ; a++, b++))  # The comma chains together operations.
  32 do
  33   echo -n "$a-$b "
  34 done
  35 
  36 echo; echo
  37 
  38 exit 0

See also Example 26-15, Example 26-16, and Example A-6.

---

Now, a for loop used in a "real-life" context.


Example 10-13. Using efax in batch mode

   1 #!/bin/bash
   2 # Faxing (must have 'efax' package installed).
   3 
   4 EXPECTED_ARGS=2
   5 E_BADARGS=65
   6 
   7 if [ $# -ne $EXPECTED_ARGS ]
   8 # Check for proper no. of command line args.
   9 then
  10    echo "Usage: `basename $0` phone# text-file"
  11    exit $E_BADARGS
  12 fi
  13 
  14 
  15 if [ ! -f "$2" ]
  16 then
  17   echo "File $2 is not a text file."
  18   #     File is not a regular file, or does not exist.
  19   exit $E_BADARGS
  20 fi
  21   
  22 
  23 fax make $2              #  Create fax formatted files from text files.
  24 
  25 for file in $(ls $2.0*)  #  Concatenate the converted files.
  26                          #  Uses wild card (filename "globbing")
  27 			 #+ in variable list.
  28 do
  29   fil="$fil $file"
  30 done  
  31 
  32 efax -d /dev/ttyS3 -o1 -t "T$1" $fil   # Finally, do the work.
  33 
  34 
  35 #  As S.C. points out, the for-loop can be eliminated with
  36 #     efax -d /dev/ttyS3 -o1 -t "T$1" $2.0*
  37 #+ but it's not quite as instructive [grin].
  38 
  39 exit 0

while

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is true (returns a 0 exit status). In contrast to a for loop, a while loop finds use in situations where the number of loop repetitions is not known beforehand.

while [condition]
do
 command...
done

As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

while [condition] ; do

Note that certain specialized while loops, as, for example, a getopts construct, deviate somewhat from the standard template given here.


Example 10-14. Simple while loop

   1 #!/bin/bash
   2 
   3 var0=0
   4 LIMIT=10
   5 
   6 while [ "$var0" -lt "$LIMIT" ]
   7 do
   8   echo -n "$var0 "        # -n suppresses newline.
   9   #             ^           Space, to separate printed out numbers.
  10 
  11   var0=`expr $var0 + 1`   # var0=$(($var0+1))  also works.
  12                           # var0=$((var0 + 1)) also works.
  13                           # let "var0 += 1"    also works.
  14 done                      # Various other methods also work.
  15 
  16 echo
  17 
  18 exit 0


Example 10-15. Another while loop

   1 #!/bin/bash
   2 
   3 echo
   4                                # Equivalent to:
   5 while [ "$var1" != "end" ]     # while test "$var1" != "end"
   6 do
   7   echo "Input variable #1 (end to exit) "
   8   read var1                    # Not 'read $var1' (why?).
   9   echo "variable #1 = $var1"   # Need quotes because of "#" . . .
  10   # If input is 'end', echoes it here.
  11   # Does not test for termination condition until top of loop.
  12   echo
  13 done  
  14 
  15 exit 0

A while loop may have multiple conditions. Only the final condition determines when the loop terminates. This necessitates a slightly different loop syntax, however.


Example 10-16. while loop with multiple conditions

   1 #!/bin/bash
   2 
   3 var1=unset
   4 previous=$var1
   5 
   6 while echo "previous-variable = $previous"
   7       echo
   8       previous=$var1
   9       [ "$var1" != end ] # Keeps track of what $var1 was previously.
  10       # Four conditions on "while", but only last one controls loop.
  11       # The *last* exit status is the one that counts.
  12 do
  13 echo "Input variable #1 (end to exit) "
  14   read var1
  15   echo "variable #1 = $var1"
  16 done  
  17 
  18 # Try to figure out how this all works.
  19 # It's a wee bit tricky.
  20 
  21 exit 0

As with a for loop, a while loop may employ C-like syntax by using the double parentheses construct (see also Example 9-31).


Example 10-17. C-like syntax in a while loop

   1 #!/bin/bash
   2 # wh-loopc.sh: Count to 10 in a "while" loop.
   3 
   4 LIMIT=10
   5 a=1
   6 
   7 while [ "$a" -le $LIMIT ]
   8 do
   9   echo -n "$a "
  10   let "a+=1"
  11 done           # No surprises, so far.
  12 
  13 echo; echo
  14 
  15 # +=================================================================+
  16 
  17 # Now, repeat with C-like syntax.
  18 
  19 ((a = 1))      # a=1
  20 # Double parentheses permit space when setting a variable, as in C.
  21 
  22 while (( a <= LIMIT ))   # Double parentheses, and no "$" preceding variables.
  23 do
  24   echo -n "$a "
  25   ((a += 1))   # let "a+=1"
  26   # Yes, indeed.
  27   # Double parentheses permit incrementing a variable with C-like syntax.
  28 done
  29 
  30 echo
  31 
  32 # Now, C programmers can feel right at home in Bash.
  33 
  34 exit 0

Note

A while loop may have its stdin redirected to a file by a < at its end.

A while loop may have its stdin supplied by a pipe.

until

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is false (opposite of while loop).

until [condition-is-true]
do
 command...
done

Note that an until loop tests for the terminating condition at the top of the loop, differing from a similar construct in some programming languages.

As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

until [condition-is-true] ; do


Example 10-18. until loop

   1 #!/bin/bash
   2 
   3 END_CONDITION=end
   4 
   5 until [ "$var1" = "$END_CONDITION" ]
   6 # Tests condition here, at top of loop.
   7 do
   8   echo "Input variable #1 "
   9   echo "($END_CONDITION to exit)"
  10   read var1
  11   echo "variable #1 = $var1"
  12   echo
  13 done  
  14 
  15 exit 0