Bash Shell Scripting/Simple Commands

A simple command consists of a sequence of words separated by spaces or tabs. The first word is taken to be the name of a command, and the remaining words are passed as arguments to the command. We have already seen a number of examples of simple commands; here are some more:

• cd ..
  • This command uses cd ("change directory"; a built-in command for navigating through the filesystem) to navigate "up" one directory.
  • The notation .. means "parent directory". For example, /foo/bar/../baz.txt is equivalent to /foo/baz.txt.
• rm foo.txt bar.txt baz.txt
  • Assuming the program rm ("remove") is installed, this command deletes the files foo.txt, bar.txt, and baz.txt in the current directory.
  • Bash finds the program rm by searching through a configurable list of directories for a file named rm that is executable (as determined by its file-permissions).
• /foo/bar/baz bip.txt
  • This command runs the program located at /foo/bar/baz, passing bip.txt as the sole argument.
  • /foo/bar/baz must be executable (as determined by its file-permissions).
  • Warning: Please ensure that there is NO SPACE between the forward slash and any files following it. e.g. assuming the "foo" folder exists in the "root" directory, then executing the following command: "rm -r / foo" will destroy your computer if performed with "sudo" access. You have been warned. If you don't understand the preceding, don't worry about it for the moment.
  • If /foo/bar/baz is a text-file rather than a binary program, and its first line begins with #!, then the remainder of that line determines the interpreter to use to run the file. For example, if the first line of /foo/bar/baz is #!/bin/bash, then the above command is equivalent to /bin/bash /foo/bar/baz bip.txt.

That example with /foo/bar/baz bears special note, since it illustrates how you can create a Bash-script that can be run like an ordinary program: just include #!/bin/bash as the first line of the script (assuming that that is where Bash is located on your system; if not, adjust as needed), and make sure the script has the right file-permissions to be readable and executable. For the remainder of this book, all examples of complete shell scripts will begin with the line #!/bin/bash.

Navigation

As we saw in the first example, the utiliy command cd serves to navigate with given arguments. In Unix and Unix-like systems like GNU/Linux there are 2 types of paths that are named as relative and absolute file path. Relative paths are relative to your contemporary position whereas absolute paths are specific. Here is an example of navigating to an absolute path:

cd 'path/to/directory'

There are specific arguments for relative path navigation that make different kinds of shortcuts possible. Here are some of those arguments:

To navigate up 2 directories you need to type:

cd ../../

To navigate to the path you were before navigating to your current location you need to type:

cd -

	Tip: Use cd with no argument for navigating directly to the default home directory on your system.

To print the path that you are currently in, its possible to use another built-in Unix utility named pwd.

Quoting

We saw above that the command rm foo.txt bar.txt baz.txt removes three separate files: foo.txt, bar.txt, and baz.txt. This happens because Bash splits the command into four separate words based on whitespace, and three of those words become arguments to the program rm. But what if we need to delete a file whose name contains a space?

	Caution: In Unix, GNU/Linux distributions and other Unix-like systems, file names can contain spaces, tabs, newlines, and even control characters.

Bash offers several quoting mechanisms that are useful for this case; the most commonly used are single-quotes ' and double-quotes ". Either of these commands will delete a file named this file.txt:

rm 'this file.txt'

rm "this file.txt"

Within the quotation marks, the space character loses its special meaning as a word-separator. Normally we wrap an entire word in quotation marks, as above, but in fact, only the space itself actually needs to be enclosed; this' 'file.txt or this" "file.txt is equivalent to 'this file.txt'.

Another commonly-used quoting mechanism is the backslash \, but it works slightly differently; it quotes (or "escapes") just one character. This command, therefore, is equivalent to the above:

rm this\ file.txt

In all of these cases, the quoting characters themselves are not passed in to the program. (This is called quote removal.) As a result, rm has no way of knowing whether it was invoked — for example — as rm foo.txt or as rm 'foo.txt'.

Filename expansion and tilde expansion

Bash supports a number of special notations, known as expansions, for passing commonly-used types of arguments to programs.

One of these is filename expansion, where a pattern such as *.txt is replaced with the names of all files matching that pattern. For example, if the current directory contains the files foo.txt, bar.txt, this file.txt, and something.else, then this command:

echo *.txt

is equivalent to this command:

echo 'bar.txt' 'foo.txt' 'this file.txt'

Here the asterisk * means "zero or more characters"; there are a few other special pattern characters (such as the question mark ?, which means "exactly one character"), and some other pattern-matching rules, but this use of * is by far the most common use of patterns.

Filename expansion is not necessarily limited to files in the current directory. For example, if we want to list all files matching t*.sh inside the directory /usr/bin, we can write this:

echo /usr/bin/t*.sh

which may expand to something like this:

echo /usr/bin/test.sh /usr/bin/time.sh

If no files match a specified pattern, then no substitution happens; for example, this command:

echo asfasefasef*avzxv

will most likely just print asfasefasef*avzxv.

Caution: If any filenames begin with a hyphen -, then filename-expansion can sometimes have surprising consequences. For example, if a directory contains two files, named -n and tmp.txt, then cat * expands to cat -n tmp.txt, and cat will interpret -n as an option rather than a filename; instead, it is better to write cat ./* or cat -- *, which expands to cat ./-n ./tmp.txt or cat -- -n tmp.txt, removing this problem.

What happens if we have an actual file named *.txt that we want to refer to? (Yes, filenames can contain asterisks!) Then we can use either of the quoting styles we saw above, or use backslash to remove special meaning from asterisk. Either of these:

cat '*.txt'

cat "*.txt"

cat \*.txt

will print the actual file *.txt, rather than printing all files whose names end in .txt.

Another, similar expansion is tilde expansion. Tilde expansion has a lot of features, but the main one is this: in a word that consists solely of a tilde ~, or in a word that begins with ~/ (tilde-slash), the tilde is replaced with the full path to the current user's home directory. For example, this command:

echo ~/*.txt

will print out the names of all files named *.txt in the current user's home directory.

Brace expansion

Similar to filename expansion is brace expansion, which is a compact way of representing multiple similar arguments. The following four commands are equivalent:

ls file1.txt file2.txt file3.txt file4.txt file5.txt

ls file{1,2,3,4,5}.txt

ls file{1..5..1}.txt

ls file{1..5}.txt

The first command lists each argument explicitly. The other three commands all use brace expansion to express the arguments more tersely: in the second command, all the possibilities 1 through 5 are given, separated by commas; in the third command, a numeric sequence is given ("from 1 to 5, incrementing by 1"); and the fourth command is the same as the third, but leaves the ..1 implicit.

We can also list the files in the opposite order:

ls file5.txt file4.txt file3.txt file2.txt file1.txt

ls file{5,4,3,2,1}.txt

ls file{5..1..-1}.txt

ls file{5..1}.txt

with the default increment size being -1 when the endpoint of the sequence is less than the starting-point.

Since in Bash, the first word of a command is the program that is run, we could also write the command this way:

{ls,file{1..5}.txt}

but obviously that is not conducive to readability. (The same sort of thing, incidentally, can be done with filename expansion.)

Brace expansion, like filename expansion, can be disabled by any of the quoting mechanisms; '{', "{", or \{ produces an actual literal curly-brace.

Redirecting output

Bash allows a command's standard output (file-descriptor 1) to be sent to a file, rather than to the console. For example, the common utility program cat writes out a file to standard output; if we redirect its standard output to a file, then we have the effect of copying the contents of one file into another file.

If we want to overwrite the destination file with the command's output, we use this notation:

cat input.txt > output.txt

If we want to keep the existing contents of the destination file as they are, and merely append the command's output to the end, we use this notation:

cat input.txt >> output.txt

However, not everything that a program writes to the console goes through standard output. Many programs use standard error (file-descriptor 2) for error-messages and some types of "logging" or "side-channel" messages. If we wish for standard error to be combined with standard output, we can use either of these notations:

cat input.txt &>> output.txt

cat input.txt >> output.txt 2>&1

If we wish for standard error to be appended to a different file from standard output, we use this notation:

cat input.txt >> output.txt 2>> error.txt

In fact, we can redirect only standard error, while leaving standard output alone:

cat input.txt 2>> error.txt

In all of the above examples, we can replace >> with > if we want to overwrite the redirect-target rather than append to it.

Later we will see some more advanced things that we can do with output redirection.

Redirecting input

Just as Bash allows a program's output to be sent into a file, it also allows a program's input to be taken from a file. For example, the common Unix utility cat copies its input to its output, such that this command:

cat < input.txt

will write out the contents of input.txt to the console.

As we have already seen, this trick is not needed in this case, since cat can instead be instructed to copy a specified file to its output; the above command is simply equivalent to this one:

cat input.txt

This is the rule rather than the exception; most common Unix utilities that can take input from the console also have the built-in functionality to take their input from a file instead. In fact, many, including cat, can take input from multiple files, making them even more flexible than the above. The following command prints out input1.txt followed by input2.txt:

cat input1.txt input2.txt

Nonetheless, input redirection has its uses, some of which we will see later on.

A preview of pipelines

Although they are outside the scope of this chapter, we now have the background needed for a first look at pipelines. A pipeline is a series of commands separated by the pipe character |. Each command is run at the same time, and the output of each command is used as the input to the next command.

For example, consider this pipeline:

cat input.txt | grep foo | grep -v bar

We have already seen the cat utility; cat input.txt simply writes the file input.txt to its standard output. The program grep is a common Unix utility that filters ("greps", in Unix parlance) based on a pattern; for example, the command grep foo will print to its standard output any lines of input that contain the string foo. The command grep -v bar uses the -v option to invert the pattern; the command prints any lines of input that don't contain the string bar. Since the input to each command is the output of the previous command, the net result is that the pipeline prints any lines of input.txt that do contain foo and do not contain bar.