How to process command output through multiple command-line programs with pipes in Linux
A pipeline is a sequence of one or more commands separated by the pipe character (|). A pipe connects the standard output of the ﬁrst command to the standard input of the next command.
Pipelines allow the output of a process to be manipulated and formatted by other processes before it is output to the terminal. One useful mental image is to imagine that data is “ﬂowing” through the pipeline from one process to another, being altered slightly by each command in the pipeline through which it ﬂows.
This example takes the output of the ls command and uses less to display it on the terminal one screen at a time.
[[email protected] ~]$ ls -l /usr/bin | less
The output of the ls command is piped to wc -l, which counts the number of lines received from ls and prints that to the terminal.
[[email protected] ~]$ ls | wc -l
In this pipeline, head will output the ﬁrst 10 lines of output from ls -t, with the ﬁnal result redirected to a ﬁle.
[[email protected] ~]$ ls -t | head -n 10 > /tmp/ten-last-changed-files
Pipelines, Redirection, and the tee Command
When redirection is combined with a pipeline, the shell sets up the entire pipeline ﬁrst, then it redirects input/output. If output redirection is used in the middle of a pipeline, the output will go to the ﬁle and not to the next command in the pipeline.
In this example, the output of the ls command goes to the ﬁle, and less display nothing on the terminal.
[[email protected] ~]$ ls > /tmp/saved-output | less
The tee command overcomes this limitation. In a pipeline, tee copies its standard input to its standard output and also redirects its standard output to the ﬁles named as arguments to the command. If you imagine data as water ﬂowing through a pipeline, tee can be visualized as a “T” joint in the pipe which directs output in two directions.
Pipeline Examples Using the tee Command
This example redirects the output of the ls command to the ﬁle and passes it to less to be displayed on the terminal one screen at a time.
[[email protected] ~]$ ls -l | tee /tmp/saved-output | less
If tee is used at the end of a pipeline, then the ﬁnal output of a command can be saved and output to the terminal at the same time.
[[email protected] ~]$ ls -t | head -n 10 | tee /tmp/ten-last-changed-files
[[email protected] ~]$ find -name / passwd 2>&1 | less