Last modified on 24 January 2011, at 21:20

LPI Linux Certification/Use Streams, Pipes And Redirects

Detailed ObjectiveEdit

Weight: 4

Description:
Candidates should be able to redirect streams and connect them in order to efficiently process textual data. Tasks include redirecting standard input, standard output and standard error, piping the output of one command to the input of another command, using the output of one command as an argument for another command and sending output to both standard output and a file.

  • Key knowledge area(s):
    • Redirect standard input, standard output and standard error.
    • Pipe the output of one command to the input of another command.
    • Use the output of one command as an argument to another command.
    • Send output to both standard output and a file.
  • The following is a partial list of the used files, terms and utilities:
    • tee
    • xargs

Standard input and standard outputEdit

For each command executed in a terminal, there is: a standard input value 0 (default keyboard), a standard output value 1 (default terminal), and a standard output for errors value 2 (default terminal).

Each channel can also be identified by an address: &0 for input, &1 for output, And &2 for errors.

Each channel [n] can be redirected. [n]< file: Default value of n is 0 and it reads standard input from file. [n]> file: Default value is 1 and it sends standard output to file, overwriting the file if it exists. (Thus clobbering the file.) [n]>>file: Default value is 1 and it appends standard output to file. <<word: Read standard input until word is reached. `command`: Substitute the command name with the result.

Examples:

$ pwd > file  # out=file, in=none, error=terminal
cat chap* >book # out=book, in=none, error=terminal
mv /etc/* . 2>error # out=terminal, in=none, error=error
echo end of file >> book # out=book, in=none, error=terminal
set -o noclobber # Shell does not clobber existing files.
ls > list 2>&1 # ls and errors are redirected to list.
ls 2>&1 > list # Errors are redirected to standard output and ls output is redirected to list.
cat `ls /etc/*.conf` > conffile 2>>/tmp/errors

Concatenate all the configuration files from /etc dir in conffile and append errors in file /tmp/errors.

Redirecting with pipes

Pipes are an efficient way to apply multiple commands concurrently.

command1 | command2

The standard output of command1 will be piped to the standard input of command2. The standard error is not piped.

Examples:

ls -l /dev | more
ls -l /etc/*.conf | grep user | grep 500
ls -l /bin | mail `users`

To redirect the standard output to a file and to the terminal at the same time, use tee.

ls -l /dev | tee file
ls -l /etc | tee -a file # Append to the file

Building arguments

The xargs utility constructs an argument list for a command using standard input.

xargs [options] [command]

The xargs command creates an argument list for command from standard input. It is typically used with a pipe.

Common options: -p: prompt the user before executing each command.

Examples:

ls f* | xargs cat # Print to standard output the content of all files starting with f.
find ~ -name 'proj1*' print | xargs cat

Search in the home directory for files starting with proj1 and send it to the standard input of cat.

Use the /dev/null device file to discard output or error messages.

Try the following:

grep try /etc/*
grep try /etc/* 2> /dev/null
grep try /etc/* > /dev/null 2> /dev/null

ExercisesEdit

1. Create a file list.bin that will contain all the filenames from the /bin directory.
2. Write a command that will append the list of files from /usr/local/bin to the file named list.bin and discard any error output.
3. Split your list.bin file into files that are 50 lines long and remove list.bin.
4. From the split files recreate list.bin (but with inversed order).
5. Simplify the following commands:

ls *.c | xargs rm
ls [aA]* | xargs cat
cat `ls *.v` 2>/dev/null

6. Use find to do the following command:

more `ls *.c`

7. Write a command that will create a file list.sbin with the contents of /sbin and at the same time display it to standard output.
8. Create a file that within the filename you include the creation time.
9. Create a file that will contain all the filenames in reverse order with extension .conf from the /etc directory.