Shell Script Programming

Calling commands as many as you want

Simply speaking, shell programming is a bunch of shell commands with a control a little. You already have used a various command, like cd, grep and so on.

If you want to print the string, "This is shell programming", in your shell, you could enter the command like below.

$ echo "This is shell programming"

Okay, that's very simple. Then now, let's repeat the string 100 times. How do you do that?

Here's other example. Now, you have to do more complex job. You have a linux service application and you should install and configure it to the server. There's a lot of works to do. For instance, copying a execution file and resource files from a local computer to a runnig server, setting or getting some environment variables, print error messages if there is an error... What's your plan to do those jobs?

Create a script file

The shell script file is runable by shell itself. Normally, a file extension of shell script is '.sh'. However, it is not mandantory. Let's begin with very simple shell script.

$ touch run.sh
$ sh run.sh     # or "bash run.sh"

Great. You have created the first shell script successfully! In order to run a script file, there are two ways. The first one is to use bash program itself with a script file as an argument, shown in before and the second one is to run the script file alone. Let me show you the second method.

$ chmod +x run.sh
$ ./run.sh

Note that the script file must have the permission to be run.

Until now, our script file do nothing. right? Eventhough we run the script, there is no output. Let's put echo command into the run.sh file.

$ echo "echo hello" > run.sh
$ ./run.sh
hello

I have used the redirection, '>', to insert the command, "echo hello" into the run.sh file.

Variables

Let's talk about variables. What is variables? In programming, the variables is some kinds of a storage that is able to save a data. It might be allocated in computer memory. It means that you are able to keep results of any commands or any values for later use.

#!/bin/sh
my_var=123

That's the first part of using variables. You should have a variable name with an initialization value, '123' in the above example. It means that

+----my_var----+
|     123      |
+--------------+

At now, how do we know the variable, my_var, has the value, 123, exactly? The best simplest way to prove that is to print out the variable. Let me show you this in the command prompt. We can also use a variable in shell.

$ my_var=123
$ echo my_var
my_var

Above commands are not working as our intution. Why? The command, 'echo', shows it's arguments by itself as a string literal.

$ my_var=123
$ echo $my_var

Here is the rule to use a shell variable.

$variable_name     # get a value stored in variable_name

Shadowing variables

shadow_var.sh

#!/bin/sh

var=23
echo "Before shadowing..."
echo $var

var=42
echo "After shadowing..."
echo $var

Before shadowing...
23
After shadowing...
42

Combine commands with variables

In the previous example, we used a command, echo. We also call any other commands through a script file! Now, let's save the result of the command into a variable.

commands.sh

#!/bin/sh
list_file=`ls`
echo $list_file

The other way to save the result of a command,

#!/bin/sh
list_file=$(ls)
echo $list_files

(Advanced) echo, quotation marks, and variables

(Advanced) Variable Scope

Let's see what happens when you're running a infinit loop shell script. The point of this example is that a new shell is created from a current shell.

$ ./infinite.sh &
$ ps -auf | grep bash
sjc      204765  0.0  0.0   9392  5548 pts/4    Ss   14:34   0:00 -bash
sjc      219497  0.0  0.0   7896  3728 pts/4    S    15:53   0:00  \_ /bin/bash ./infinite.sh
sjc      219630  0.0  0.0   7008  2144 pts/4    S+   15:54   0:00  \_ grep --color=auto bash
$ sudo kill -9 219497    # Terminate the infinite loop

Environment Variables

Let me show you already defined environment variables in your shell session.

$ printenv
SHELL=/bin/bash
NAME=AAAA-AAA
LOGNAME=sjc
...

Each ouput format is NAME=VALUE per line. NAME is an environment variable name as you expected. Environment variables are a kind of special variables widely used in your system.

(Optional) Operator

As I know, operators in Shell programming are not used commonly.

arithmetic operators

+, -, *, /, %

(Advanced) Standard Input, Standard Output, Standard Error

Control (Important!)

Shell scripting supports control statements also like other programming languages. Control statement means that you could control which commands should be executed upon your condition, true or false. The important thing is to use control statements

if-then statements format

if [ condition ]               if [ condition ]; then
then                   or          {body}
    {body}                     fi
fi

Three things you have to keep in mind.

an white space between each of '[' and ']' characters is needed to be interpreted by shell correctly.
If you want to use then in the same line with if statements, semicolon(;) must be added.
fi must be added at the end of the condition block.

if [condition]    # illegal
if[ condition ]   # illegal
if[condition]     # illegal
if [ condition ]  # legal

What is the condition? How do we check it? The condition is actually an expression which is able to evaluated by Shell. Normally, condition contains the expression to compare something. For instance, the expression could compare two numbers, the expression could check the existence of a file or a directory.

You can find a various expresison operators by using below command.

$ man test

It might be hard to remember all operators.

Let's do with a little more complex scenarios.

if-then-else statements format

if [ condition ]; then
    {if-body}
else
    {else-body}
fi

if-then-elif-else statements format

if [ conditon1 ]; then
    {if-body}
elif [ condition2 ]; then
    {elif-body}
else
    {else-body}
fi

(Optional) test vs []

condition/test.sh and condition/if.sh

Use logical operators

How to use logical operators, like AND or OR in Shell programming? Firstly, let's combine commands with a logical operator.

If the $$ operator is used between two commands, the second command would be executed only if the first command is done successfully, which means the return, exit value should be zero.

$ ehco "print file list" && ls
print file list
# ls

The other operator, || provides the opposite way of && operator. The condition to execute the second command is that the first command should be failed. It means that the most common case of using || operator is the error check.

$ ee "print file list" || ls
# show ls

You can also use logical operators in the test command with if statement.

AND	OR
-a	-o

That's a very intuitive character. Isn't it? Let me show you a simple example to present how to use.

$ num1=10
$ num2=9
$ test $num1 -gt $num2 -a $num1 -eq 10
$ echo $?
0
$ test $num1 -gt $num2 -a $num1 -eq 9
$ echo $?
1

Other ways to compare numbers and strings

sample files: comp-str.sh, comp-num.sh

Various type of expressions

Taking an argument as an variable

We have seen a few examples about output.

By now, we assign and use a variable in the shell script file. The problem is,

How do we change a compareble value at running a shell script?

To do that, Shell provides the method to take an user input by the command prompt.

io/argv.sh

echo "The command name: $0"
echo "The first argument: $1"
echo "The second argument: $2"

$ sh argv.sh
The command name: argv.sh
The first argument:
The second argument:

$ sh argv.sh 123
The command name: argv.sh
The first argument: 123
The second argument:

$ sh argv.sh 123 "zoo"
The command name: argv.sh
The first argument: 123
The second argument: zoo

(Advanced) Special argument variables

io/argv2.sh

#!/bin/sh

echo -n "The number of arguments: "
echo $#

echo -n "All of arguments: "
echo $*

echo -n "Run 'whoamyou': "
whoamyou
echo $?

echo -n "Run 'whoami': "
whoami
echo $?

Run this script with two arguments.

$ ./argv2.sh 123 56
The number of arguments: 2
All of arguments: 123 56
Run 'whoamyou': ./argv2.sh: 10: whoamyou: not found
127
Run 'whoami': yourname
0

case statements format

case {variable} in
    {pattern-1})
        {statements-1}
        ;;
    {pattern-2})
        {statements-2}
        ;;
    *)
        {statements-3}
        ;;
esac

First, try to find {pattern-1} in {variable}. If there is a matched pattern in the variable,

GLOB Pattern

pattern	description	example
*
?
[]
{}

Loop

Using for loop

for-basic.sh

#!/bin/sh
for i in 1 2 3; do
  echo $i
done

for variable in value1 value2 ... valueN; do
: variable will be assigned by from value1 to valueN sequentially. For instance, in the first loop, variable has value1, then in the second loop, variable has value2, and so on. Note that the defualt variable scope is global so variable assigned in the for loop could be used out of the loop block.

done
: Indicate the end of the for loop

Uset the for loop with the shell command, ls to list files in the current directory.

#!/bin/sh
for file in $(ls); do
  echo "--- $file"
done

The result of $(ls) is all of the name of files. For instance, file1.txt, file2.txt. so this sentence, for file in $(ls) is same with for file in file1.txt file2.txt. As the result, the variable, file, has file1.txt then file2.txt.

Using while loop

#!/bin/sh
num=1
while [ $num -le 10 ]; do
  echo $num
  num=$(($num + 1))
done

while condition; do : The condition is same expression used in if-then statements. done

Tip

In order to indicate the start and the end of the loop, do-done is used. Don't be confused with if-then-fi.

break

#!/bin/sh
num=1
while [ $num -le 10 ]; do
  echo $num
  num=$(($num + 1))

  [ $num -eq 5 ] && break
done

continue

#!/bin/sh
num=1                           # assign 1 to the variable, num.
while [ $num -le 10 ]; do       # Check the variable, num, is less than or equal to 10.
  num=$(($num + 1))             # Increate the num by 1.
  if [ $num -le 5 ]; then       # If the variable, num is less than or equal to 5, 
    continue                    # back to the condition statements of while loop.
  fi
  echo $num                     # Print the variable, num.
done                            # End of the loop block

loop count	num before while	while condition	num before if	if condition	echo?
1	1	true	2	true	no
2	2	true	3	true	no
3	3	true	4	true	no
4	4	true	5	true	no
5	5	true	6	false	yes
6	6	true	7	false	yes
7	7	true	8	false	yes
9	9	true	10	false	yes
10	10	true	11	false	yes
11	11	false	-	-	-

Infinite Loop

#!/bin/sh
while :; do
  echo "Press [Ctrl-C] to exit"
done

#!/bin/bash
for (( ; ; )); do
  echo "Press [Ctrl-C] to exit"
done

Note that the infinite loop using for can be run only by bash shell. So I recommend using the while-style infinite loop for compatibility.

Function

Definition and usage

funcion_name()
{
    {statements}
}

Exit

Exit command is to terminate the running script itself. You can write the exit command as zero at the end of the script if that script was doing all well or else you should use the exit command with non-zero value.

#!/bin/bash
check_home_dir() {
  # If there is no HOME directory, which means 'test' command has returned false(non-zero value), exit the script with 1.
  test -d $HOME || exit 1
}
my_func $1
exit 0

(Advanced) Source (.)

Dot(.) command is for loading a script into the current script.

#!/bin/bash
cd ~
cd ..
pwd

Let's compare betwen two execution ways.

# Run the script only
$ ./dot-test.sh
# Use dot command
$ . dot-test.sh

The terminal output will be same with two cases, your home directory path. However, your current directory wouldn't be same. Because if you run a script, a subshell is created. Which means that if you run a script as an ordinary way like the first case, the current directory is moved to '/home' and also print '/home' in the subshell. So that after the termination of the subshell, the original shell that runs dot-test.sh is remaining on the current directory where the script was run.

The command case of usage for this sourcing mechanism is to use variables and functions in other scripts, like a library.

(Advanced) Commands within two blocks, () and {}

(Appendix) Test Command Operators Summary

string compare

format	description
-n STRING	length is non-zero
-z STRING	length is zero
STRING1 = STRING2	same string
STRING1 != STRING2	not same string

number(integer) compare

format	symbolic	note
NUMBER1 -eq NUMBER2	NUMBER1 == NUMBER2	equal
NUMBER1 -ne NUMBER2	NUMBER1 != NUMBER2	not equal
NUMBER1 -gt NUMBER2	NUMBER1 > NUMBER2	greater than
NUMBER1 -ge NUMBER2	NUMBER1 >= NUMBER2	greater than & equal to
NUMBER1 -lt NUMBER2	NUMBER1 < NUMBER2	less than
NUMBER1 -le NUMBER2	NUMBER1 <= NUMBER2	less than & equal to

file or directory existence

format	description	note
-e FILE	FILE exists?	exists
-d FILE	FILE exists? and directory?	directory exists
-r FILE	FILE exists? and regular file?	regular file
-s FILE	FILE exists? and size > 0?	has size

file or directory permission

format	description
-r FILE	FILE exists? and readable?
-w FILE	FILE exists? and writeable?
-x FILE	FILE exists? and executable?

file or directory compare

format	description
FILE1 -ef FILE2	FILE1 is equal to FILE2?
FILE1 -nt FILE2	FILE1 is newer than FILE2?
FILE1 -ot FILE2	FILE1 is older than FILE2?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shell Script Programming

Calling commands as many as you want

Create a script file

Variables

Shadowing variables

Combine commands with variables

(Advanced) echo, quotation marks, and variables

(Advanced) Variable Scope

Environment Variables

(Optional) Operator

(Advanced) Standard Input, Standard Output, Standard Error

Control (Important!)

(Optional) test vs []

Use logical operators

Other ways to compare numbers and strings

Various type of expressions

Taking an argument as an variable

(Advanced) Special argument variables

Loop

Using for loop

Using while loop

break

continue

Infinite Loop

Function

Exit

(Advanced) Source (.)

(Advanced) Commands within two blocks, () and {}

(Appendix) Test Command Operators Summary

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Shell Script Programming

Calling commands as many as you want

Create a script file

Variables

Shadowing variables

Combine commands with variables

(Advanced) echo, quotation marks, and variables

(Advanced) Variable Scope

Environment Variables

(Optional) Operator

(Advanced) Standard Input, Standard Output, Standard Error

Control (Important!)

(Optional) test vs []

Use logical operators

Other ways to compare numbers and strings

Various type of expressions

Taking an argument as an variable

(Advanced) Special argument variables

Loop

Using for loop

Using while loop

break

continue

Infinite Loop

Function

Exit

(Advanced) Source (.)

(Advanced) Commands within two blocks, () and {}

(Appendix) Test Command Operators Summary