How to run commands affecting many files by using pattern matching features of the Bash shell

Command-Line Expansions

The Bash shell has multiple ways of expanding a command line including pattern matching, home directory expansion, string expansion, and variable substitution. Perhaps the most powerful of these is the path name-matching capability, historically called globbing. The Bash globbing feature, sometimes called “wildcards”, makes managing large numbers of ﬁles easier. Using metacharacters that “expand” to match ﬁle and path names being sought, commands perform on a focused set of ﬁles at once.

Pattern Matching

Globbing is a shell command-parsing operation that expands a wildcard pattern into a list of matching path names. Command-line metacharacters are replaced by the match list prior to command execution. Patterns that do not return matches display the original pattern request as literal text. The following are common metacharacters and pattern classes.

PATTERN	MATCHES
*	Any string of zero or more characters.
?	Any single character.
[abc…]	Any one character in the enclosed class (between the square brackets).
[!abc…]	Any one character not in the enclosed class.
[^abc…]	Any one character not in the enclosed class.
[[:alpha:]]	Any alphabetic character.
[[:lower:]]	Any lowercase character.
[[:upper:]]	Any uppercase character.
[[:alnum:]]	Any alphabetic character or digit.
[[:punct:]]	Any printable character not a space or alphanumeric.
[[:digit:]]	Any single digit from 0 to 9.
[[:space:]]	Any single white space character. This may include tabs, newlines, carriage returns, form feeds, or spaces.

For the next few examples, pretend that you have run the following commands to create some sample ﬁles.

[[email protected] ~]$ mkdir glob; cd glob
[[email protected] glob]$ touch alfa bravo charlie delta echo able baker cast dog easy
[[email protected] glob]$ ls
able  alfa  baker  bravo  cast  charlie  delta  dog  easy  echo
[[email protected] glob]$

The ﬁrst example will use simple pattern matches with the asterisk (*) and question mark (?) characters, and a class of characters, to match some of those ﬁle names.

[[email protected] glob]$ ls a*
able  alfa
[[email protected] glob]$ ls *a*
able  alfa  baker  bravo  cast  charlie  delta  easy
[[email protected] glob]$ ls [ac]*
able  alfa  cast  charlie
[[email protected] glob]$ ls ????
able  alfa  cast  easy  echo
[[email protected] glob]$ ls ?????
baker  bravo  delta
[[email protected] glob]$

Tilde Expansion

The tilde character (~), matches the current user’s home directory. If it starts a string of characters other than a slash (/), the shell will interpret the string up to that slash as a user name, if one matches, and replace the string with the absolute path to that user’s home directory. If no user name matches, then an actual tilde followed by the string of characters will be used instead.

In the following example the echo command is used to display the value of the tilde character. The echo command can also be used to display the values of brace and variable expansion characters, and others.

[[email protected] glob]$ ls ~root
/root
[[email protected] glob]$ ls ~user
/home/user
[[email protected] glob]$ ls ~/glob
able  alfa  baker  bravo  cast  charlie  delta  dog  easy  echo
[[email protected] glob]$ echo ~/glob
/home/user/glob
[[email protected] glob]$

Brace Expansion

Brace expansion is used to generate discretionary strings of characters. Braces contain a commaseparated list of strings, or a sequence expression. The result includes the text preceding or following the brace deﬁnition. Brace expansions may be nested, one inside another. Also doubledot syntax (..) expands to a sequence such that {m..p} will expand to m n o p.

[[email protected] glob]$ echo {Sunday,Monday,Tuesday,Wednesday}.log
Sunday.log Monday.log Tuesday.log Wednesday.log
[[email protected] glob]$ echo file{1..3}.txt
file1.txt file2.txt file3.txt
[[email protected] glob]$ echo file{a..c}.txt
filea.txt fileb.txt filec.txt
[[email protected] glob]$ echo file{a,b}{1,2}.txt
filea1.txt filea2.txt fileb1.txt fileb2.txt
[[email protected] glob]$ echo file{a{1,2},b,c}.txt
filea1.txt filea2.txt fileb.txt filec.txt
[[email protected] glob]$

A practical use of brace expansion is to quickly create a number of ﬁles or directories.

[[email protected] glob]$ mkdir ../RHEL{6,7,8}
[[email protected] glob]$ ls ../RHEL*
RHEL6 RHEL7 RHEL8
[[email protected] glob]$

Variable Expansion

A variable acts like a named container that can store a value in memory. Variables make it easy to access and modify the stored data either from the command line or within a shell script. You can assign data as a value to a variable using the following syntax:

[[email protected] ~]$ VARIABLENAME=value

You can use variable expansion to convert the variable name to its value on the command line. If a string starts with a dollar sign ($), then the shell will try to use the rest of that string as a variable name and replace it with whatever value the variable has.

[[email protected] ~]$ USERNAME=operator
[[email protected] ~]$ echo $USERNAME
operator

To help avoid mistakes due to other shell expansions, you can put the name of the variable in curly braces, for example ${VARIABLENAME}.

[[email protected] ~]$ USERNAME=operator
[[email protected] ~]$ echo ${USERNAME}
operator

Shell variables and ways to use them will be covered in more depth later in this course.

Command Substitution

Command substitution allows the output of a command to replace the command itself on the command line. Command substitution occurs when a command is enclosed in parentheses and preceded by a dollar sign ($). The $(command) form can nest multiple command expansions inside each other.

[[email protected] glob]$ echo Today is $(date +%A).
Today is Wednesday.
[[email protected] glob]$ echo The time is $(date +%M) minutes past $(date +%l%p).
The time is 26 minutes past 11AM.
[[email protected] glob]$

NOTE: An older form of command substitution uses backticks: `command`. Disadvantages to the backticks form include: 1) it can be easy to visually confuse backticks with single quote marks, and 2) backticks cannot be nested.

Protecting Arguments from Expansion

Many characters have special meaning in the Bash shell. To keep the shell from performing shell expansions on parts of your command line, you can quote and escape characters and strings. The backslash (\) is an escape character in the Bash shell. It will protect the character immediately following it from expansion.

[[email protected] glob]$ echo The value of $HOME is your home directory.
The value of /home/user is your home directory.
[[email protected] glob]$ echo The value of \$HOME is your home directory.
The value of $HOME is your home directory.
[[email protected] glob]$

In the preceding example, protecting the dollar sign from expansion caused Bash to treat it as a regular character and it did not perform variable expansion on $HOME. To protect longer character strings, single quotes (’) or double quotes (") are used to enclose strings. They have slightly different effects. Single quotes stop all shell expansion. Double quotes stop most shell expansion. Use double quotation marks to suppress globbing and shell expansion, but still allow command and variable substitution.

[[email protected] glob]$ myhost=$(hostname -s); echo $myhost
host
[[email protected] glob]$ echo "***** hostname is ${myhost} *****"
***** hostname is host *****
[[email protected] glob]$

Use single quotation marks to interpret all text literally.

[[email protected] glob]$ echo "Will variable $myhost evaluate to $(hostname -s)?"
Will variable myhost evaluate to host?
[[email protected] glob]$ echo 'Will variable $myhost evaluate to $(hostname -s)?'
Will variable $myhost evaluate to $(hostname -s)?
[[email protected] glob]$

Note: The single quote (’) and the command substitution backtick (`) can be easy to confuse, both on the screen and on the keyboard. Using one when you mean to use the other will lead to unexpected shell behavior.