EEEN11202 course notes
2.2.1. Shell scripting¶
Danger
Lab A is different to all of the other labs in the course. You can run Lab A on your physical computer, OR in our devcontainer/codespace. The notes support both.
Remember that our devcontainer/codespace is a Linux environment. If using the devcontainer/codespace, follow the instructions for macOS/Linux in the notes.
This may be a little confusing at first - you need to keep track of which computer (your actual computer or our devcontainer) you’re actually working on. It is probably simpler if everyone uses the devcontainer/codespace and follows the macOS/Linux instructions, but we also wanted to give Windows users the option to make some PowerShell scripts if they would like to. The assignment associated with this lab will accept both .sh and .ps1 files.
2.2.1.1. Setup¶
Follow the instructions for getting access to a suitable programming environment on your computer. If you have any difficulties, ask a demonstrator in a lab session for help.
You don’t need to actually start VSCode yet. We’re going to start using only the command line interface (also known as the terminal, shell, or console) that’s built into your computer.
2.2.1.2. Launching the terminal¶
On a desktop/laptop type device you will already have a command line interface installed. Start this, following the instructions given below for your operating system.
The terminal is called PowerShell. You have a choice for how to start this.
You can start it directly by typing in
powershellto the search box in the start menu.
Screenshot of the Windows start menu and VSCode, software from Microsoft. See course copyright statement.¶
Or, you can first install Windows terminal. Launch the Microsoft Store and search for
terminal. Then install Windows Terminal.Then launch it by typing in
terminalto the search box in the start menu.
Screenshot of the Windows start menu and VSCode, software from Microsoft. See course copyright statement.¶
We think that Windows Terminal provides a slightly nicer interface (to exactly the same thing) and so our screenshots will make us of it.
Click on the launchpad / start icon for your operating system and then search for terminal.
You should be presented with a terminal command line interface that looks like the below. Note that we’ve combined the instructions for macOS and Linux in the below, as these terminals will take the same commands and it reduces the number of pictures needed. The only meaningful difference at this point is that macOS uses % to indicate where to enter commands, while Linux uses $ by default. Widows uses > to show where to enter commands.
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
2.2.1.3. Simple terminal commands¶
There are a large number of different commands that you can enter at the prompt to interact with your computer. We’ll go through some common examples below.
After a while you’ll probably memorize some common ones, but in general you can look up or ask AI to help with any that you don’t know. Important for this course isn’t to memorize all of the different commands, but to have an understanding of how the command line works so that you can use it to interact with the computer.
2.2.1.3.1. Displaying the current folder¶
To display the address of the current folder type in
Get-Location
and press the Enter key on the keyboard.
Remember, you don’t type in the >, that just to show you where the command prompt is.
This will display something like the below, where we’ve also shown how this appears in the Windows graphical interface.
Screenshot of the Windows terminal and File Explorer, software from Microsoft. See course copyright statement.¶
pwd
and press the Enter key on the keyboard.
Remember, you don’t type in the $, that just to show you where the command prompt is.
pwd stands for print working directory. This will display something like the below.
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
The address shown will be different depending on your computer settings and user name. Here the user name is alex. You should be able to open File Explorer/Finder/similar and find the same location on your computer. It’s not that you can’t use the graphical interface to view this information, it’s that entering written commands gives us precise control and a log of what we’ve done. If you press the up arrow ↑ on the keyboard you can see previous commands that you’ve entered. Press it multiple times to see earlier commands. Or, enter
Get-History
history
to see a log of all of the commands that you’ve run. This makes it easy to re-run them.
2.2.1.3.2. Listing the files in the current folder¶
To get a list of the files in the current folder type in
Get-ChildItem
and press the Enter key on the keyboard.
This will display something like
Screenshot of the Windows terminal and File Explorer, software from Microsoft. See course copyright statement.¶
ls
and press the Enter key on the keyboard.
This will display something like
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
You may have different files, or none at all, displayed depending on what’s on your computer in your current location.
Again, you should be able to open File Explorer/Finder/similar and find the same location and files on your computer.
2.2.1.3.3. Folders¶
Make a folder called test by entering the command
New-Item -ItemType Directory -Path test
mkdir test
Then move into the folder test by entering the command
Set-Location -Path test
cd test
You can enter
Get-ChildItem
ls
again to check the location. It should display something similar to
C:\Users\alex\test
/Users/alex/test
or
/home/alex/test
depending on the location you started from. You can go up a level, that is to exit the test folder, by entering
cd ..
cd ..
2.2.1.3.4. Command switches¶
Lots of commands accept options that change their behavior. These are known as switches or as arguments. Generally these are entered with a hyphen - or two hyphens -- after the main command. Try entering
Get-ChildItem -Attributes Directory
This will display only folders, rather than everything that’s in the current location.
ls -la
This lists the contents of the folder, together with additional information such as when the items were last changed. An example is below. What’s shown will depend on what files and folders you have on your computer.
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
2.2.1.3.5. Combining commands¶
There are a number of ways in which you can combine togethers, for example to send the output of one command as the input to another command. Enter
Get-ChildItem | Out-File -FilePath list.txt
This will take the output from Get-ChildItem and write it to a text file called list.txt rather than displaying it to the screen. You can read the contents of this file using
Get-Content -Path list.txt
ls -la | tee list.txt
This will take the output from ls -la and write it to a text file called list.txt rather than displaying it to the screen. You can read the contents of this file using
cat list.txt
Aside
You can also use the redirection operator >> for a similar effect, but we won’t cover that here.
2.2.1.3.6. Further commands¶
There are many further command line commands that come by default with your computer. Most apps you install can also be controlled from the command line, even if you usually use the graphical interface. We won’t aim to cover these here, but you’ll encounter many such commands as you move through the course.
For now, spend a bit of time moving between folders using the commands given above to get a feeling for how it works.
2.2.1.4. Scripts¶
The above commands work very well when working interactively with the computer. That is, entering one command, observing the output, then entering the next command, and so on.
Often we need to carry out more complex processes though, which might need lots of commands. We might want to run these multiple times, once a day for example to run a series of steps repeatedly. We can collect together shell commands into a shell script to help us with this. A shell script is a file containing a list of commands to be run one-after-another.
Aside
You can probably accomplish the same automation, and more, using Python, which we’ll meet later. However, for simple automation tasks that can be overkill. The shell commands are built into the operating system and so are always available. For simple automation tasks shell scripts are widely used.
2.2.1.4.1. Getting started¶
This will display the welcome page, and pre-downloaded files for Lab A, similar to the below. Remember that VSCode is very configurable. Don’t worry if your screen isn’t exactly the same, as long as it’s broadly similar it will be fine.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
You may (but may not) be asked whether you trust the authors. This is a security setting. Letting code run on your computer can change settings, delete files, or do other nefarious things. It’s our own code we’re writing though, so select Yes, I trust the authors.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
You need to make a file to store your shell script in. Click New File.... This will display a box to enter the file name.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
Enter the filename as
hello_world.ps1
hello_world.sh
You’ll need to press Enter on the keyboard. This will display another box confirming where you want the file stored. We want it in the Lab A folder, which should come up as the default location. Press OK.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
Due to our VSCode setup, where each Lab is stored as a separate workspace, VSCode will ask you which lab you want the script to be associated with. Select Lab A.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
Once successful, you’ll see the hello_world file in VSCode. You’ll have an area to start adding your shell commands to, as shown below. (This figure assumes you’re using Windows, so the script has a .ps1 extension).
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
2.2.1.4.2. A simple script¶
In your file, enter the code below. (It’s fine to just copy and paste this.)
# Display a welcome message
$name = "Alex"
Write-Host "Hello $name!"
# Display a different message depending on the value of $hour
$hour = 13
if ($hour -ge 12) {
Write-Host "It is the afternoon."
} else {
Write-Host "It is the morning."
}
This is a very simple script. It:
Contains some comments, lines starting with
#to explain what’s going on.Variables to store data in. In Powershell, variables start with a
$.Write-Hostis used to display output to the screen.There is then an if statement to change what’s displayed depending on the value of
$hour. Here-gemeans greater than or equal to.
#!/usr/bin/env sh
# Display a welcome message
name="Alex"
echo "Hello $name!"
# Display a different message depending on the value of $hour
hour=13
if [ "$hour" -ge 12 ]; then
echo "It is the afternoon."
else
echo "It is the morning."
fi
This is a very simple script.
The line starting
#!is known as a shebang line. This tells the computer which language to use to interpret the script. You may well have more than one installed. Theshtells the computer to use whatever the system default is.There are then some comments, lines starting with
#to explain what’s going on.Variables store data. Here variables start with a
$when being used, but the$isn’t needed when putting a value in the console.echois used to display output to the screen.There is then an if statement to change what’s displayed depending on the value of
$hour. Here-gemeans greater than or equal to.
Save the file by selecting File / Save.
2.2.1.4.3. Running the script¶
To run the script, you have two options. Both options do exactly the same thing, just one method is built into VSCode and one method is built into the computer’s command line.
Option 1. Press the run button that appears in the VSCode GUI. This will run the script, and you’ll see appropriate text in the terminal, as shown below.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
Important
There appears to be a bug with the current VSCode Powershell extension which means nothing happens when you press this button. If you find this, click on the Extensions tab on the left of the VSCode window and scroll down to see the Powershell extension. Uninstall the extension, and then install it again.
Option 2. In a Powershell terminal on the computer enter the command
powershell.exe -noprofile -executionpolicy bypass -file .\hello_world.ps1
.\hello_world.ps1 is a relative address. It assumes your terminal is in the same folder as the script.
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
By default, Windows blocks users from running arbitrary scripts for security reasons. The extra commands above tell Windows to ignore the security settings for this run of the script. This is fine for this lab, but of course you should take care when running scripts from others. You can change the Windows security settings so you don’t need to give an override each time, but we won’t cover that here.
Option 1. In the VSCode terminal enter the commands
chmod u+x ./hello_world.sh
./hello_world.sh
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
Option 2. In a computer terminal enter the commands
chmod u+x ./hello_world.sh
./hello_world.sh
Screenshot of the Windows terminal, software from Microsoft. See course copyright statement.¶
In both cases, chmod u+x changes the security settings, allowing the script to be run (executed x) by the user (u). You only need to set this once for each file, then the setting will be remembered. ./hello_world.sh actually runs the code.
./hello_world.sh is a relative address. It assumes your terminal is in the same folder as the script.
Try changing the values of $name and $hour and re-run the script to check it displays what you would expect.
As a challenge, get your script to automatically read the system time to determine which message to display. That is, rather than typing in a value for $hour, use a command to set it for you. On Windows, read about the Get-Date command. On macOS/Linux, read about the date command. These give full dates by default, you’ll need to add some switches for them to return only the current hour. Search the Internet or ask a chat bot for the options to provide.
Solution
# Display a welcome message
$name = "Alex"
Write-Host "Hello $name!"
# Display a different message depending on the value of $hour
$hour = Get-Date -Format HH
if ($hour -ge 12) {
Write-Host "It is the afternoon."
} else {
Write-Host "It is the morning."
}
#!/usr/bin/env sh
# Display a welcome message
name="Alex"
echo "Hello $name!"
# Display a different message depending on the value of $hour
hour="$(date +"%H")"
if [ "$hour" -ge 12 ]; then
echo "It is the afternoon."
else
echo "It is the morning."
fi
2.2.1.4.4. More scripts¶
Many of the programming concepts introduced in Part 1, such as loops and different data types are possible in shell scripts if you need them. (Although if you’re doing much more than simple computer automation you’re likely better to use Python, as we’ll learn about later in the course.)
At the end of your script file from above add the code
# Display colours of the rainbow
$colors = @("Red","Orange","Yellow","Green","Blue","Indigo","Violet")
For ($i=0; $i -lt $colors.Length; $i++) {
Write-Host $colors[$i]
}
# Display the contents of /
Get-ChildItem -Path / | ForEach-Object {
Write-Host "$_"
}
Save the file and press the Run button again. You should have an output like the below.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
This script now contains two different types of for loop.
The first has a list of colours. Each of these colours is accessed in turn, and
Write-Hostis used to display them to the screen.The second uses
ForEach-Object, and takes its input fromGet-ChildItem. That is, it looks up what what is in the folder asked for. The results are passed to the for loop, so that each item can be used in turn, with the name of each item automatically stored in$_. In the for loop, here we just display the name of each time. You could imagine carrying out more tasks, such as changing the name of files, or checking whether a file contains particular text, and so on. You would just add more lines of code between the curly brackers{}of theForEach-Objectcommand.
Aside
You’ll notice the code above displays the contents of /, not C:\. This is because for the screenshots its being run in the devcontainer, which is a Linux environment. If you run it on your own Windows computer you’ll want to display the contents of C:\ instead.
# Display colours of the rainbow
set -- "Red" "Orange" "Yellow" "Green" "Blue" "Indigo" "Violet"
for a in "$@"; do
echo $a
done
# Display the contents of /
for f in /*; do
echo "/ contains $f"
done
Save the file and run the script again. You should have an output like the below.
Screenshot of VSCode, software from Microsoft. See course copyright statement.¶
This script now contains two different types of for loop.
The first has a list of colours, stored using
setinto a variable called$@. Each of these colours is accessed in turn and put into a variable called$a.echois used to display$ato the screen.The second uses
/*to find all of the files that are stored in/.*indicates find everything. These are put into a variable called$f, andechois used to display$fto the screen.
Aside
The above syntax, in particular the set command is slightly awkward. Recall that our shebang line is #!/usr/bin/env sh, to use the system’s default terminal. As we don’t necessarily know what the default will be, we’re restricted to a small set of commands that we know will work in every shell. There is a standard known as POSIX which defines this. For most Linux systems the default shell is bash, and so we could use #!/usr/bin/env bash as the shebang line. The macOS the default shell is zsh, and so #!/usr/bin/env zsh. Doing this allows many more commands and a much nicer array syntax. However, it would mean our code was no longer portable, able to run on any system. As we don’t know what computer you’re using, we went for the fully portable option.
There are of course many more possible commands and things that you can investigate. That’s not our aim here. Our aim is to give you a brief introduction and some insights into what’s possible, so you can then build on this if you need to, or if you encounter shell scripts in your future work.
If you are having any issues writing or running the code, we’ve pre-made scripts called my_script.ps1 and my_script.sh which contain the final code for this part of the lab. These were automatically downloaded to your Lab A folder as part of the VSCode setup. Have a look at these and use them to help debug or fix your code.