How bash does expansions and substitutions?

    Before executing your commands,bash checks whether there are any syntax elements in the command line that should be interpreted rather than taken literally.After splitting the command line into tokens(words),bash scans for these special elements and interprets them,resulting in a changed command line:the elements are said to be expanded or subistituted to new text and maybe new tokens(words).

    Bash performs expansions and substitutions in a defined order.This explains why globbing(pathname expansion),for example,is safe to use on filenames with spaces(becasue it happens after the final word splitting).

    The order is (from first to last):

1. brace expansion
2. tlide expansion
3. parameter expansion    arithemetic expansion    command expansion
4. word spliting
5. pathname expansion

    Process substitution is performed simultaneously with parameter expansion,command substitution and arithmetic expansion.It is only performed when the underlying operating system supports it.

    The 3 steps parameter expansion, arimethic expansion and command expansion happen at the same time in a left-to-right fashion on the commandline.This means:

i=1

echo $i $((i++)) $i

expansions and substitutions in details

What is brace expansion?

{string1,string2,......,stringN}

{<START>..<END>}

{<START>..<END>..<INCR>}(bash 4)

<PREAMBLE>{........}

{........}<POSTSCRIPT>

<PREAMBLE>{........}<POSTSCRIPT>

    Brace expansion is used to generate arbitrary strings.The specified strings are used to generate all possible combinations with the optional surrounding preambles and postscripts.

    Usually it's used to generate mass-arguments for a command,that follow a specific naming-scheme.

    It is the very first step in expansion-handling,it's important to understand that.When you use

echo {a,b}$PATH

    then the brace expansion does not expand the variable -- this is in a later step.Brace expansion just makes it being:

echo a$PATH b$PATH

    Another common pitfall is to assume that a range like {1..200} can be expressed with variables using a=1;b=2;{$a..$b}.Due to what I described above,it simply is not possible,because it's the very first step in doing expansions.A possible way to achieve this,if you really can't handle this in another way,is using the eval command,which basically evaluates a commandline twice:

what is eval?

eval [arg ...]

    the eval method requires that the entire command be properly escaped to avoid unexpected expansions.If the sequence expansion is to be assigned to an array,another method is using declaration commands:

declare -a 'pics=  (img{'"$a..$b"'}.png)'; mv "${
    pics[@]}" ../imgs

What is declare?

declare: declare [-aAfFgilrtux] [-p] [name[=value] ...]

Both the exact quoting,and explicitly including "-a" are important.

The brace expansion is present in two basic forms,strings list and ranges

Strings list

Without the optional preamble and postscript strings,the result is just a space-separated list of the given strings:

$ echo {I,want,my,money,back}I want my money back

With preamble or postscript strings, the result is a space-separated list of all possible combinations of preamble, specified strings and postscript:

$ echo _{I,want,my,money,back}_I _want _my _money _back$ echo {I,want,my,money,back}_I_ want_ my_ money_ back_$ echo _{I,want,my,money,back}-_I- _want- _my- _money- _back-

The brace expansion is only performed,if the given string list is really a list of strings.i.e.if there's minimum one "," (comma)! Something like {money} doesn't expand to something special,it's really only the text "{money}".

Ranges

{
    
     ..
     
      }
     
    

Brace expansion using ranges is written giving the startpoint and the endpoint of the range.This is a "sequence expression".The sequence can be of two types:

• integers (optionally zero padded, optionally with a given increment)
• characters

$ echo {5..12}5 6 7 8 9 10 11 12$ echo {c..k}c d e f g h i j k

$ echo {5..k}{5..k}

When you zeropad one of the numbers(or both) in a range,then the generated range is zeropadded,too:

$ echo {01..10}01 02 03 04 05 06 07 08 09 10

Similar to the expansion using stringlists,you can add preamble and postscript strings:

$ echo 1.{0..9}1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9$ echo ---{A..E}------A--- ---B--- ---C--- ---D--- ---E---

Combining and nesting

When you combine more brace expansions,you effectively use a brace expansion as preamble or postscript for another one.Let's generate all possible combinations of uppercase letters and digits:

$ echo {A..Z}{0..9}A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 C0 C1 C2 C3 C4 C5 C6C7 C8 C9 D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 F0 F1 F2 F3F4 F5 F6 F7 F8 F9 G0 G1 G2 G3 G4 G5 G6 G7 G8 G9 H0 H1 H2 H3 H4 H5 H6 H7 H8 H9 I0I1 I2 I3 I4 I5 I6 I7 I8 I9 J0 J1 J2 J3 J4 J5 J6 J7 J8 J9 K0 K1 K2 K3 K4 K5 K6 K7K8 K9 L0 L1 L2 L3 L4 L5 L6 L7 L8 L9 M0 M1 M2 M3 M4 M5 M6 M7 M8 M9 N0 N1 N2 N3 N4N5 N6 N7 N8 N9 O0 O1 O2 O3 O4 O5 O6 O7 O8 O9 P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 Q0 Q1Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 S0 S1 S2 S3 S4 S5 S6 S7 S8S9 T0 T1 T2 T3 T4 T5 T6 T7 T8 T9 U0 U1 U2 U3 U4 U5 U6 U7 U8 U9 V0 V1 V2 V3 V4 V5V6 V7 V8 V9 W0 W1 W2 W3 W4 W5 W6 W7 W8 W9 X0 X1 X2 X3 X4 X5 X6 X7 X8 X9 Y0 Y1 Y2Y3 Y4 Y5 Y6 Y7 Y8 Y9 Z0 Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9

Brace expansions can be nested,but too much of it usually makes you losing overview a bit.

Here's a sample to generate the alphabet,first the uppercase letters,then the lowercase ones:

$ echo {
   {A..Z},{a..z}}A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z

Common use and examples

wget http://docs.example.com/documentation/slides_part{1,2,3,4,5,6}.htmlwget http://docs.example.com/documentation/slides_part{1..6}.htmlmkdir /home/bash/test/{foo,bar,baz,cat,dog}/home/bash/test/{foo,bar,baz,cat,dog}somecommand -v{,,,,}

/home/bash/test/{foo,bar,baz,cat,dog} generates /home/bash/test/foo /home/bash/test/bar /home/bash/test/baz /home/bash/test/cat /home/bash/test/dog

somecommand -v{,,,,} generates somecommand -v -v -v -v -v.

New in Bash 4.0

Zero padded number expansion

Prefix either of the numbers in a numeric range with 0 to pad the expanded numbers with the correct amount of zeros:

$ echo {0001..5}0001 0002 0003 0004 0005

Increment

{
    
     ..
     
      ..
      
       }
      
     
    

$ echo {1..10..2}1 3 5 7 9$ echo {10..1..2}10 8 6 4 2

Interesting feature:The increment specification also works for letter-ranges:

$ echo {a..z..3}a d g j m p s v y

What is Tilde expansion?

~~/...~NAME~NAME/...~+~+/...~-~-/...

The Tilde expansion is used to expand to several specific pathnames:

• home directories
• current working directory
• previous working directory

Tilde expansion is only performed,when the tlide-construct is at the beginning of a word,or a separte word.

If there's nothing to expand,i.e. in case of a wrong username or any other error condition,the tlide construct is not replaced,it says what it is.

Tlide expansion is also performed everytime a variable is assigned:

• after the

  first =:TARGET=~moonman/share
• after

  every :(colon) in the assigned value:TARGET=file:~moonman/share

This way you can correctly use the the tlide expansino in your PATH:

PATH=~/mybins:~peter/mybins:$PATH

~/"my directory"

Home directory

~~
    

This form expands to the home-directory of the current user(~) or the home directory of the given user(<NAME>).

If the given user doesn't exist(or if his home directory isn't determinable,for some reason),it doesn't expand to something else,it stays what it is.The requested home directory is found by asking the operating system for the associated home for <NAME>.

To find the home directory of the current user(~),Bash has a precedence:

• expand to the value of HOME if it's defined
• expand to the home directory of the user executing the shell(operating system)

That means,the variable HOME can override the "real" home directory,at least regarding tilde expansion.

Current working directory

~+

echo "CWD is $PWD"

echo "CWD is" ~+

Previous working directory

~-

$ pwd/home/bash$ cd /etc$ echo ~-/home/bash

What is parameter expansion?

one core functionality of Bash is to manage parameters.A parameter is an entity that stores values and is referenced  by a name,a number or a special symbol.

• Parameters referenced by a name are called variables.(this also applies to arrays)
• Parameters referenced by a number are called positional parameters and reflect the arguments given to a shell.
• Parameters referenced by a special symbol are auto-set parameters that have different special meanings and uses.

Parameter expansion is the procedure to get the value from the referenced entity,like expanding a variable to print its value.On expansion time you can do very nasty things with the parameter or its value.These things are described here.

If you saw some parameter expansion syntax somewhere,and you need to check what it can be,try the overview section below!

Arrays can be special cases for parameter expansion,every applicable description mentions arrays below.

Simple usage

• $PARAMETER
• ${PARAMETER}

The easiest form is to just use a parameter's name within braces.This is identical to using $Foo like you see it everywhere,but has the advantage that it can be immediately followed by characters that would be interpreted as part of the parameter name otherwise.Compare these two expressions,where we want to print a word with a trailing 's'.

echo "The plural of $WORD is most likely $WORDs"echo "The plural of $WORD is most likely ${WORD}s"

Why does the first one fail?It prints nothing,because a parameter(variable) named "WORDs" is undefined and thus printed as ""(nothing).Without using braces for parameter expansion,Bash will interpret the sequence of all valid characters from the introducing "$" up to the last valid character as name of the parameter.When using braces you just force bash to only interpret the name inside your braces.

Also,please remember,that parameter names are (like nearly everything in UNIX@) case sentitive!

The second form with the curly braces is also needed to access positional parameters(arguments to script) beyond $9:

echo "Argument  1 is: $1"echo "Argument 10 is: ${10}"

Arrays:

For arrays you always need the braces.The arrays are expanded by individual indexes or mass arguments.An individual index behaves like a normal parameter.

Purpose

An array is a parameter that holds mappings from keys to values.Arrays are used to store a collection of parameters into a parameter.Arrays(in any programming language) are a useful and common composite data structure,and one of the most important scripting features in Bash and other shells.

Here is an abstract representation of an array named NAMES.The indexes go from 0 to 3.

NAMES 0: Peter 1: Anna 2: Greg 3: Jan