How to run commands affecting many files by using pattern matching features of the Bash shell

Command-Line Expansions

The Bash shell has multiple ways of expanding a command line including pattern matching, home directory expansion, string expansion, and variable substitution. Perhaps the most powerful of these is the path name-matching capability, historically called globbing. The Bash globbing feature, sometimes called “wildcards”, makes managing large numbers of ﬁles easier. Using metacharacters that “expand” to match ﬁle and path names being sought, commands perform on a focused set of ﬁles at once.

Pattern Matching

Globbing is a shell command-parsing operation that expands a wildcard pattern into a list of matching path names. Command-line metacharacters are replaced by the match list prior to command execution. Patterns that do not return matches display the original pattern request as literal text. The following are common metacharacters and pattern classes.

PATTERN	MATCHES
*	Any string of zero or more characters.
?	Any single character.
[abc…]	Any one character in the enclosed class (between the square brackets).
[!abc…]	Any one character not in the enclosed class.
[^abc…]	Any one character not in the enclosed class.
[[:alpha:]]	Any alphabetic character.
[[:lower:]]	Any lowercase character.
[[:upper:]]	Any uppercase character.
[[:alnum:]]	Any alphabetic character or digit.
[[:punct:]]	Any printable character not a space or alphanumeric.
[[:digit:]]	Any single digit from 0 to 9.
[[:space:]]	Any single white space character. This may include tabs, newlines, carriage returns, form feeds, or spaces.

For the next few examples, pretend that you have run the following commands to create some sample ﬁles.

[user@host ~]$ mkdir glob; cd glob
[user@host glob]$ touch alfa bravo charlie delta echo able baker cast dog easy
[user@host glob]$ ls
able  alfa  baker  bravo  cast  charlie  delta  dog  easy  echo
[user@host glob]$

The ﬁrst example will use simple pattern matches with the asterisk (*) and question mark (?) characters, and a class of characters, to match some of those ﬁle names.

[user@host glob]$ ls a*
able  alfa
[user@host glob]$ ls *a*
able  alfa  baker  bravo  cast  charlie  delta  easy
[user@host glob]$ ls [ac]*
able  alfa  cast  charlie
[user@host glob]$ ls ????
able  alfa  cast  easy  echo
[user@host glob]$ ls ?????
baker  bravo  delta
[user@host glob]$

Tilde Expansion

The tilde character (~), matches the current user’s home directory. If it starts a string of characters other than a slash (/), the shell will interpret the string up to that slash as a user name, if one matches, and replace the string with the absolute path to that user’s home directory. If no user name matches, then an actual tilde followed by the string of characters will be used instead.

In the following example the echo command is used to display the value of the tilde character. The echo command can also be used to display the values of brace and variable expansion characters, and others.

[user@host glob]$ ls ~root
/root
[user@host glob]$ ls ~user
/home/user
[user@host glob]$ ls ~/glob
able  alfa  baker  bravo  cast  charlie  delta  dog  easy  echo
[user@host glob]$ echo ~/glob
/home/user/glob
[user@host glob]$

Brace Expansion

Brace expansion is used to generate discretionary strings of characters. Braces contain a commaseparated list of strings, or a sequence expression. The result includes the text preceding or following the brace deﬁnition. Brace expansions may be nested, one inside another. Also doubledot syntax (..) expands to a sequence such that {m..p} will expand to m n o p.

[user@host glob]$ echo {Sunday,Monday,Tuesday,Wednesday}.log
Sunday.log Monday.log Tuesday.log Wednesday.log
[user@host glob]$ echo file{1..3}.txt
file1.txt file2.txt file3.txt
[user@host glob]$ echo file{a..c}.txt
filea.txt fileb.txt filec.txt
[user@host glob]$ echo file{a,b}{1,2}.txt
filea1.txt filea2.txt fileb1.txt fileb2.txt
[user@host glob]$ echo file{a{1,2},b,c}.txt
filea1.txt filea2.txt fileb.txt filec.txt
[user@host glob]$

A practical use of brace expansion is to quickly create a number of ﬁles or directories.

[user@host glob]$ mkdir ../RHEL{6,7,8}
[user@host glob]$ ls ../RHEL*
RHEL6 RHEL7 RHEL8
[user@host glob]$

Variable Expansion

A variable acts like a named container that can store a value in memory. Variables make it easy to access and modify the stored data either from the command line or within a shell script. You can assign data as a value to a variable using the following syntax:

[user@host ~]$ VARIABLENAME=value

You can use variable expansion to convert the variable name to its value on the command line. If a string starts with a dollar sign ($), then the shell will try to use the rest of that string as a variable name and replace it with whatever value the variable has.

[user@host ~]$ USERNAME=operator
[user@host ~]$ echo $USERNAME
operator

To help avoid mistakes due to other shell expansions, you can put the name of the variable in curly braces, for example ${VARIABLENAME}.

[user@host ~]$ USERNAME=operator
[user@host ~]$ echo ${USERNAME}
operator

Shell variables and ways to use them will be covered in more depth later in this course.

Command Substitution

Command substitution allows the output of a command to replace the command itself on the command line. Command substitution occurs when a command is enclosed in parentheses and preceded by a dollar sign ($). The $(command) form can nest multiple command expansions inside each other.

[user@host glob]$ echo Today is $(date +%A).
Today is Wednesday.
[user@host glob]$ echo The time is $(date +%M) minutes past $(date +%l%p).
The time is 26 minutes past 11AM.
[user@host glob]$

NOTE: An older form of command substitution uses backticks: `command`. Disadvantages to the backticks form include: 1) it can be easy to visually confuse backticks with single quote marks, and 2) backticks cannot be nested.

Protecting Arguments from Expansion

Many characters have special meaning in the Bash shell. To keep the shell from performing shell expansions on parts of your command line, you can quote and escape characters and strings. The backslash (\) is an escape character in the Bash shell. It will protect the character immediately following it from expansion.

[user@host glob]$ echo The value of $HOME is your home directory.
The value of /home/user is your home directory.
[user@host glob]$ echo The value of \$HOME is your home directory.
The value of $HOME is your home directory.
[user@host glob]$

In the preceding example, protecting the dollar sign from expansion caused Bash to treat it as a regular character and it did not perform variable expansion on $HOME. To protect longer character strings, single quotes (’) or double quotes (") are used to enclose strings. They have slightly different effects. Single quotes stop all shell expansion. Double quotes stop most shell expansion. Use double quotation marks to suppress globbing and shell expansion, but still allow command and variable substitution.

[user@host glob]$ myhost=$(hostname -s); echo $myhost
host
[user@host glob]$ echo "***** hostname is ${myhost} *****"
***** hostname is host *****
[user@host glob]$

Use single quotation marks to interpret all text literally.

[user@host glob]$ echo "Will variable $myhost evaluate to $(hostname -s)?"
Will variable myhost evaluate to host?
[user@host glob]$ echo 'Will variable $myhost evaluate to $(hostname -s)?'
Will variable $myhost evaluate to $(hostname -s)?
[user@host glob]$

Note: The single quote (’) and the command substitution backtick (`) can be easy to confuse, both on the screen and on the keyboard. Using one when you mean to use the other will lead to unexpected shell behavior.