Bourne Shell Scripting/Appendix D: Cookbook

Post a new Cookbook entry

If you use the title box, then you do not need to put a title in the body.

Branch on extensions

When writing a bash script which should do different things based on the extension of a file, the following pattern is helpful.

 #filepath should be set to the name(with optional path) of the file in question
 ext=${filepath##*.}
 if [[ "$ext" == txt ]] ; then 
   #do something with text files
 fi

(Source: slike.com Bash FAQ).

Rename several files

This recipe shows how to rename several files following a pattern.

In this example, the user has huge collection of screenshots. This user wants to rename the files using a Bourne-compatible shell. Here is an "ls" at the shell prompt to show you the filenames. The goal is to rename images like "snapshot1.png" to "nethack-kernigh-22oct2005-01.png".

 $ ls
 snapshot1.png   snapshot25.png  snapshot40.png  snapshot56.png  snapshot71.png
 snapshot10.png  snapshot26.png  snapshot41.png  snapshot57.png  snapshot72.png 
 snapshot11.png  snapshot27.png  snapshot42.png  snapshot58.png  snapshot73.png
 snapshot12.png  snapshot28.png  snapshot43.png  snapshot59.png  snapshot74.png
 snapshot13.png  snapshot29.png  snapshot44.png  snapshot6.png   snapshot75.png
 snapshot14.png  snapshot3.png   snapshot45.png  snapshot60.png  snapshot76.png
 snapshot15.png  snapshot30.png  snapshot46.png  snapshot61.png  snapshot77.png
 snapshot16.png  snapshot31.png  snapshot47.png  snapshot62.png  snapshot78.png
 snapshot17.png  snapshot32.png  snapshot48.png  snapshot63.png  snapshot79.png
 snapshot18.png  snapshot33.png  snapshot49.png  snapshot64.png  snapshot8.png
 snapshot19.png  snapshot34.png  snapshot5.png   snapshot65.png  snapshot80.png
 snapshot2.png   snapshot35.png  snapshot50.png  snapshot66.png  snapshot81.png
 snapshot20.png  snapshot36.png  snapshot51.png  snapshot67.png  snapshot82.png
 snapshot21.png  snapshot37.png  snapshot52.png  snapshot68.png  snapshot83.png
 snapshot22.png  snapshot38.png  snapshot53.png  snapshot69.png  snapshot9.png
 snapshot23.png  snapshot39.png  snapshot54.png  snapshot7.png
 snapshot24.png  snapshot4.png   snapshot55.png  snapshot70.png

First, to add a "0" (zero) before snapshots 1 through 9, write a for loop (in effect, a short shell script).

• Use ? which is a filename pattern for a single character. Using it, I can match snapshots 1 through 9 but miss 10 through 83 by saying snapshot?.png.
• Use ${parameter#pattern} to substitute the value of parameter with the pattern removed from the beginning. This is to get rid of "snapshot" so I can put in "snapshot0".
• Before actually running the loop, insert an "echo" to test that the commands will be correct.

 $ for i in snapshot?.png; do echo mv "$i" "snapshot0${i#snapshot}"; done
 mv snapshot1.png snapshot01.png
 mv snapshot2.png snapshot02.png
 mv snapshot3.png snapshot03.png
 mv snapshot4.png snapshot04.png
 mv snapshot5.png snapshot05.png
 mv snapshot6.png snapshot06.png
 mv snapshot7.png snapshot07.png
 mv snapshot8.png snapshot08.png
 mv snapshot9.png snapshot09.png

That seems good, so run it by removing the "echo".

 $ for i in snapshot?.png; do mv "$i" "snapshot0${i#snapshot}"; done

An ls confirms that this was effective.

Now change prefix "snapshot" to "nethack-kernigh-22oct2005-". Run a loop similar to the previous one:

 $ for i in snapshot*.png; do \
 > mv "$i" "nethack-kernigh-22oct2005-${i#snapshot}" \
 > done

This saves the user from typing 83 "mv" commands.

Long command line options

The builtin getopts does not support long options so the external getopt is required. (On some systems, getopt also does not support long options, so the next example will not work.)

eval set -- $(getopt -l install-opts: "" "$@")
while true; do
    case "$1" in
        --install-opts)
            INSTALL_OPTS=$2
            shift 2
            ;;
        --)
            shift
            break
            ;;
    esac
done

echo $INSTALL_OPTS

The call to getopt quotes and reorders the command line arguments found in $@. set then makes replaces $@ with the output from getopt

Another example of getopt use can also be found in the Advanced Bash Script Guide

Process certain files through xargs

In this recipe, we want to process a large list of files, but we must run one command for each file. In this example, we want to convert the sampling rates of some sound files to 44100 hertz. The command is sox file.ogg -r 44100 conv/file.ogg, which converts file.ogg to a new file conv/file.ogg. We also want to skip files that are already 44100 hertz.

First, we need the sampling rates of our files. One way is to use the file command:

 $ file *.ogg
 audio_on.ogg:            Ogg data, Vorbis audio, mono, 44100 Hz, ~80000 bps
 beep_1.ogg:              Ogg data, Vorbis audio, stereo, 44100 Hz, ~193603 bps
 cannon_1.ogg:            Ogg data, Vorbis audio, mono, 48000 Hz, ~96000 bps
 ...

(The files in this example are from Secret Maryo Chronicles.) We can use grep -v to filter out all lines that contain '44100 Hz':

 $ file *.ogg | grep -v '44100 Hz'
 cannon_1.ogg:            Ogg data, Vorbis audio, mono, 48000 Hz, ~96000 bps
 ...
 jump_small.ogg:          Ogg data, Vorbis audio, mono, 8000 Hz, ~22400 bps
 live_up.ogg:             Ogg data, Vorbis audio, mono, 22050 Hz, ~40222 bps
 ...

We finished with "grep" and "file", so now we want to remove the other info and leave only the filenames to pass to "sox". We use the text utility cut. The option -d: divides each line into fields at the colon; -f1 selects the first field.

 $ file *.ogg | grep -v '44100 Hz' | cut -d: -f1
 cannon_1.ogg
 ...
 jump_small.ogg
 live_up.ogg
 ...

We can use another pipe to supply the filenames on the standard input, but "sox" expects them as arguments. We use xargs, which will run a command repeatedly using arguments from the standard input. The -n1 option specifies one argument per command. For example, we can run echo sox repeatedly:

 $ file *.ogg | grep -v '44100 Hz' | cut -d: -f1 | xargs -n1 echo sox
 sox cannon_1.ogg
 ...
 sox itembox_set.ogg
 sox jump_small.ogg
 ...

However, these commands are wrong. The full command for cannon_1.ogg, for example, is sox cannon_1.ogg -r 44100 conv/cannon_1.ogg. "xargs" will insert incoming data into placeholders indicated by "{}". We use this strategy in our pipeline. If we have doubt, then first we can build a test pipeline with "echo":

 $ file *.ogg | grep -v '44100 Hz' | cut -d: -f1 | \
 > xargs -i 'echo sox {} -r 44100 conv/{}'
 sox cannon_1.ogg -r 44100 conv/cannon_1.ogg
 ...
 sox itembox_set.ogg -r 44100 conv/itembox_set.ogg
 sox jump_small.ogg -r 44100 conv/jump_small.ogg
 ...

It worked, so let us remove the "echo" and run the "sox" commands:

 $ mkdir conv
 $ file *.ogg | grep -v '44100 Hz' | cut -d: -f1 | \
 > xargs -i 'sox {} -r 44100 conv/{}'

After a wait, the converted files appear in the conv subdirectory. The above three lines alone did the entire conversion.

Simple playlist frontend for GStreamer

If you have GStreamer, the command gst-launch filesrc location=filename ! decodebin ! audioconvert ! esdsink will play a sound or music file of any format for which you have a GStreamer plugin. This script will play through a list of files, optionally looping through them. (Replace "esdsink" with your favorite sink.)

#!/bin/sh
loop=false
if test x"$1" == x-l; then
  loop=true
  shift
fi

while true; do
  for i in "$@"; do
    if test -f "$i"; then
      echo "${0##*/}: playing $i" > /dev/stderr
      gst-launch filesrc location="$i" ! decodebin ! audioconvert ! esdsink
    else
      echo "${0##*/}: not a file: $i" > /dev/stderr
    fi
  done
  if $loop; then true; else break; fi
done

This script demonstrates some common Bourne shell tactics:

• "loop" is a boolean variable. It works because its values "true" and "false" are both Unix commands (and sometimes shell builtins), thus you can use them as conditions in if and while statements.
• The shell builtin "shift" removes $1 from the argument list, thus shifting $2 to $1, $3 to $2, and so forward. This script uses it to process an "-l" option.
• The substitution ${0##*/} gives everything in $0 after the last slash, thus "playlist", not "/home/musicfan/bin/playlist".