Cloud DevOps Admin Guide

Unix provides the capability to change where standard input comes from, or where output goes using a concept called Input/Output (I/O) redirection. I/O redirection is accomplished using a redirection operator which allows the user to specify the input or output data be redirected to (or from) a file. Note that redirection always results in the data stream going to or coming from a file.

The simplest case to demonstrate this is basic output redirection. The output redirection operator is the > (greater than) symbol, and the general syntax looks as follows:

command > output_file_spec

Spaces around the redirection operator are not mandatory, but do add readability to the command. Thus in our ls example from above, we can observe the following use of output redirection:

$ ls > my_files [Enter]
$

Notice there is no output appearing after the command, only the return of the prompt. Why is this, you ask? This is because all output from this command was redirected to the file my_files. Observe in the following diagram, no data goes to the terminal screen, but to the file instead.



Examining the file as follows results in the contents of the my_files being displayed:


$ cat my_files [Enter]
foo
bar
fred
barney
dino $


In this example, if the file my_files does not exist, the redirection operator causes its creation, and if it does exist, the contents are overwritten. Consider the example below:

$ echo "Hello World!" > my_files [Enter]
$ cat my_files [Enter]
Hello World!

Notice here that the previous contents of the my_files file are gone, and replaced with the string "Hello World!" This might not be the most desirable behavior, so the shell provides us with the capability to append output to files. The append operator is the >>. Thus we can do the following:

$ ls > my_files [Enter]
$ echo "Hello World!" >> my_files [Enter]
$ cat my_files [Enter]
foo
bar
fred
barney
dino
Hello World!

The first output redirection creates the file if it does not exist, or overwrites its contents if it does, and the second redirection appends the string "Hello World!" to the end of the file. When using the append redirection operator, if the file does not exist, >> will cause its creation and append the output (to the empty file). The ability also exists to redirect the standard input using the input redirection operator, the < (less than) symbol. Note the point of the operator implies the direction. The general syntax of input redirection looks as follows:

command < input_file_spec

Looking in more detail at this, we will use the wc (word count) command. The wc command counts the number of bytes, word and lines in a file. Thus if we do the following using the file created above, we see:

$ wc my_files [Enter]
6 7 39 my_files

where the output indicates 6 lines, 7 words and 39 bytes, followed by the name of the file wc opened.

We can also use wc in conjunction with input redirection as follows:

$ wc < my_files [Enter]
6 7 39

Note here that the numeric values are as in the example above, but with input redirection, the file name is not listed. This is because the wc command does not know the name of the file, only that it received a stream of bytes to count.

Someone will certainly ask if input redirection and output redirection can be combined, and the answer is most definitely yes. They can be combined as follows:

$ wc < my_files > wc_output [Enter]
$
There is no output sent to the terminal screen since all output was sent to the file wc_output. If we then looked at the contents of wc_output, it would contain the same data as above.

To this point, we have discussed the standard input stream (descriptor 0) and the standard output stream (descriptor 1). There is another output stream called standard error (stderr) which has file descriptor 2. Typically when programs return errors, they return these using the standard error channel. Both stdout and stderr direct output to the terminal by default, so distinguishing between the two may be difficult. However each of these output channels can be redirected independently. Refer to the diagram below:





The standard error redirection operator is similar to the stdout redirection operator and is the 2> (two followed by the greater than, with no spaces) symbol, and the general syntax looks as follows:

 command 2> output_file_spec

Thus to show an example, we observe the following:

$ ls foo bar 2> error_file [Enter]
foo
$ cat error_file [Enter] 
 ls: bar: No such file or directory
Note here that only the standard output appears once the standard error stream is redirected into the file named error_file, and when we display the contents of error_file, it contains what was previously displayed on the termimal. To show another example:

$ ls foo bar > foo_file 2> error_file [Enter]
$
$ cat foo_file [Enter]
foo
$ cat error_file [Enter] 
ls: bar: No such file or directory

In this case both stdout and stderr were redirected to file, thus no output was sent to the terminal. The contents of each output file was what was previously displayed on the screen.

Note there are numerous ways to combine input, output and error redirection. Another relevant topic that merits discussion here is the special file named /dev/null (sometimes referred to as the "bit bucket"). This virtual device discards all data written to it, and returns an End of File (EOF) to any process that reads from it. I informally describe this file as a "garbage can/recycle bin" like thing, except there's no bottom to it. This implies that it can never fill up, and nothing sent to it can ever be retrieved. This file is used in place of an output redirection file specification, when the redirected stream is not desired. For example, if you never care about viewing the standard output, only the standard error channel, you can do the following:

$ ls foo bar > /dev/null [Enter] 
ls: bar: No such file or directory
In this case, successful command output will be discarded. The /dev/null file is typically used as an empty destination in such cases where there is a large volume of extraneous output, or cases where errors are handled internally so error messages are not warranted. One final miscellaneous item is the technique of combining the two output streams into a single file. This is typically done with the 2>&1 command, as follows:

$ command > /dev/null 2>&1 [Enter]
$
Here the leftmost redirection operator (>) sends stdout to /dev/null and the 2>&1 indicates that channel 2 should be redirected to the same location as channel 1, thus no output is returned to the terminal.

Redirection Summary

Redirection Operator	Resulting Operation
command > file	stdout written to file, overwriting if file exists
command >> file	stdout written to file, appending if file exists
command < file	input read from file
command 2> file	stderr written to file, overwriting if file exsits
command 2>> file	stderr written to file, appending if file exists
command > file 2>&1	stdout written to file, stderr written to same file descriptor