Ict-innovation/LPI/103.4

103.4 Streams, Pipes and Re-directs

Candidates should be able to redirect streams and connect them in order to efficiently process textual data. Tasks include redirecting standard input, standard output and standard error, piping the output of one command to the input of another command, using the output of one command as an argument for another command and sending output to both standard output and a file.

Key Knowledge Areas

• Redirecting standard input, standard output and standard error.
• Pipe the output of one command to the input of another command.
• Use the output of one command as arguments to another command.
• Send output to both stdout and a file.

Input, Output, Redirection

UNIX processes use streams to get input, (standard input stream) send output to, (standard output stream) and a stream to send error messages to (the standard error stream). These streams can be redirected for any process. In most cases standard input (stdin) is the keyboard, and the two output descriptors, standard out (stdout) and standard error (stderr),go to the screen. Sometimes it is convenient to redirect these standard streams so that the process received input from a file and/or sends output to file.

Numerical values for stdin, stderr and stdout
stdin	0
stdout	1
stderr	2

When redirecting or interacting with these streams we refer to them by their numerical values.

Standard Out Redirection

To redirect standard output from the screen to a file for example you use the “>” symbol.

For example:

$ find / -iname *.txt > textfiles.txt

This will run the find utility and output the result to the textfiles.txt file. No output is visible on the screen.. The textfiles.txt file will be created first if it doesn’t exist and overwritten if not. To append to a file rather than create a new one the ‘>>’ operator can be used.

Standard Error Redirect

Standard error redirect uses the same format as for standard input redirect but you need to specify that the stderr stream and not stdout is to be redirected. This is done via placing the stderr stream id before the redirect symbol.

$ myapp 2> error.txt

As per the stdout example above this will create a new file. To append to an existing file you would use “2>>” to redirect standard error.

Redirect stdout and stderr

To redirect stdout and stderr at the same time you would use the “&>” or “&>>” operator. This will direct both standard out and standard error to the same file.

Standard In Redirection

To have the process read input from a file rather than get its input from the keyboard you would use the “<' symbol as in the example below:

$ mysql -u root -p < createtable.sql

Here the mysql command line interface is told to take its standard input from a file called createtable.sql rather than read input from the keyboard. This file would contain SQL statements necessary to create a table for example.

Piped Commands

The pipe command is used to redirect the standard output from one process to the standard input of another.

program1 | program2

Pipes are represented by the “|” symbol. The data stream goes from the left to the right. The next figure illustrates how the stdout for one process is redirected to the stdin for another process.

# ls-l | less

The tee Command

command | tee FILENAME

This command is used after a pipe and takes a filename as an argument. The standard output from the previous command is then sent to the file given as an argument but tee also lets the stream through to stdout. The stdout has been duplicated in this way.

xargs

This tool is often thought of as a companion tool to find. In fact xargs will process each line of standard output as an argument for another tool. We could use xargs to delete all files belonging to a user with:

$ find / -type f -user 502 | xargs rm –f

If the list of filenames is very long, xargs will split it into pieces and invoke the rm command several times, one for each piece. This is sometimes useful if the argument list would otherwise be too long to handle.

$ ls |xargs rm -f

---

Used files, terms and utilities:

• tee
• xargs

Previous Chapter | Next Chapter