Add permission and pipelines

Author: shf

.gitignore

@@ -0,0 +1,3 @@
+Gemfile.lock
+.jekyll-cache/
+_site/

Feedback.md

@@ -0,0 +1,12 @@
+---
+nav_exclude: true
+---
+
+### What to keep:
+
+- Hands-on experience
+- Reference to the website
+
+### What to improve:
+
+- Drop loops and introduce permissions, ownership, and process management

Gemfile

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+git_source(:github) {|repo_name| "https://github.com/ubc-library-rc/intro-shell/" }
+
+# gem "rails"
+
+gem "jekyll"
+gem 'github-pages', group: :jekyll_plugins
+

Participating Onliine.md renamed to Participating Online.md

@@ -10,14 +10,18 @@
 
 The shell (sometimes referred to as the “Unix shell”, for the operating system where it was first developed) is a program that allows you to interact with your computer using typed text commands. It is the primary interface used on Linux and Unix-based systems, such as macOS, and can be optionally installed on other operating systems such as older versions of Windows. As of Windows 10 the Unix shell comes pre-installed but must be turned on.
 
+The most popular Unix shell is Bash (the Bourne Again SHell). Using the shell has a steep learning curve compared with the GUI. While a GUI presents you with choices to select, the command-line interface (CLI) does not present the choices to you. The syntax of the shell allows you to make powerful pipelines by coupling existing commands and handle repetitive tasks. Many shells support scripting as well.
+
+In addition, the command line is often the best option to interact with remote machines and servers. As clusters and cloud computing systems become more common, researchers need to learn the necessary skills to tackle file handling and job submission tasks.
+
 ## Why should I use it?
 
 Most of the time when we work with files on a computer the graphical user interface (GUI) that we are familiar with does the job. This is how we visually navigate, scroll, and click through information on our computers.
 
-It's when we are dealing with a large number of files or files that are scattered across many different locations that the shell is very useful. Any time that you need to do something very repetitive to a large number of files, for example renaming everything in a folder or separating out all of a certain type of file, there is likely a way to use the shell to speed up your work.
+It's when we are dealing with a large number of files or folders that are scattered across many different locations that the shell is very useful. Any time that you need to do something very repetitive to a large number of files, for example renaming everything in a folder or separating out all of a certain type of file, there is likely a way to use the shell to speed up your work.
 
 ## How do I get to it?
 
 For Mac and Linux, which are Unix based systems, you can access the Unix shell through an application called "Terminal". It is findable like any other piece of software installed on your computer.
 
-For Windows 10 users the Unix shell needs to be turned on but is available as a part of the operating system. Previous versions of Windows required you to install it separately.
+For Windows 10 users, the Unix shell needs to be turned on but is available as a part of the operating system. You can find more information in per [Simplified Installation](https://docs.microsoft.com/en-us/windows/wsl/install-win10#simplified-installation-for-windows-insiders).Previous versions of Windows required you to install it separately.

content/01-what-is-the-shell.md

@@ -6,7 +6,7 @@
 ---
 # Navigating the file system
 
-Let's start by opening the unix shell. There are a few things you are likely to see right away. A cursor that is flashing next to a $ indicates where the command line begins. If you see the dollar sign it means that the command line is ready to take commands.
+Let's start by opening the unix shell. There are a few things you are likely to see right away. A cursor that is flashing next to a $, a **prompt**, indicates where the command line begins. If you see the dollar sign it means that the command line is ready to take commands.
 
 ~~~
 $
@@ -15,7 +15,8 @@ $
 You are communicating with your computer by giving it commands and having it react to those commands.
 
 ## Orienting yourself
-Let's enter our first command "pwd" which stands for "print working directory" and will tell us where we are in the system. The term "directory" refers to what you might visually think of as a "folder" when browsing your file system.
+
+Let's enter our first command `pwd` which stands for **print working directory** and will tell us where we are in the system. The term "directory" refers to what you might visually think of as a "folder" when browsing your file system.
 
 Input
 {: .label .label-green}
@@ -28,6 +29,8 @@ Output
 /Users/egrguric
 ~~~
 
+We can use commands to open GUI for applications. For example, `open` opens finders on Mac. A dot, `.`, points to the current working directory. 
+
 Input
 {: .label .label-green}
 ~~~
@@ -36,6 +39,10 @@ $ open .
 
 ![Finder window](finder.png)
 
+In a Unix filesystem, the **root directory** is referred by a single slash character, `/`. Inside this directory, there are several important directories: `bin` contains built-in programs, `data` contains miscellaneous data, `Users` contains user data, and `tmp` contains temporary files that do not need to stored long-term. 
+
+To see the content of the filesystem in the terminal, `ls` is used.
+
 Input
 {: .label .label-green}
 ~~~
@@ -50,6 +57,8 @@ Desktop			Movies			Vagrants
 Documents		Music
 ~~~
 
+We can tailor the output of commands by adding options and arguments. For example, `-l` option makes `ls` use a long listing format showing additional information such as the file size and the time of its last modification. 
+
 Input
 {: .label .label-green}
 ~~~
@@ -70,6 +79,8 @@ drwx------+   4 egrguric  staff    128 25 Jul 13:56 Pictures
 drwxr-xr-x+   4 egrguric  staff    128 18 Jul  2019 Public
 ~~~
 
+By adding `-h` option, the file size becomes human readable.
+
 Input
 {: .label .label-green}
 ~~~
@@ -91,8 +102,32 @@ drwxr-xr-x+   4 egrguric  staff   128B 18 Jul  2019 Public
 
 ~~~
 
+## Permissions and ownership
+
+The first field of information in the list above is the file type. A file is represented by a hyphen, `-`, and a directory is represented by the letter `d`. The rest of it represent the permission groups: owner, group, and other.
+
+In Unix, there are three permission groups. The first three letters shows the permissions used by the assigned owner of the file or directory. Each permission group has three permissions, called a permission set. A permission set consists of read, write, and execute permissions, represented by `r`, `w`, and `x`, respectively. A dash symbol in place of a character in a permission set indicated that a particular permission is denied. Linux assigns initial permissions automatically when a new file or directory is created. 
+
+Every file is owned by a specific user (or UID) and a specific group (or GID). To change the user or group of a file, we can use `chown` command.
+
+Input
+{: .label .label-green}
+~~~
+$ chown user filename
+$ chown user:group filename
+~~~
+
+The `chmod` command is used to alter the permission of a file. For example, to add execute permission for the owner of a file:
+
+Input
+{: .label .label-green}
+~~~
+$ chmod u+x filename
+~~~
+
 ## Getting help
-Commands come with instructions that you can access from the command line by invoking the "manual". You can also access this manual documentation online.
+
+Every shell command, including `ls`, has lots of other options. There are two common ways to find out how to use a command and the options available for it. We can read the **manual** of a command with `man`
 
 Input
 {: .label .label-green}
@@ -129,11 +164,19 @@ DESCRIPTION
 ~~~
 Notice the colon : which indicates that there is more text and you can keep paging through it by hitting "enter". To quit the manual hit the "q" key which stands for "quit".
 
-You can only go to places that exist.
+We can also pass `--help` optino to the command:
+
+Input
+{: .label .label-green}
+~~~
+$ ls --help
+~~~
+
+You can also find the manual pages on the web. 
 
 ## Moving around
 
-To move between directories we can use the "cd" command which stands for "change directory". This will move you one step forward in the system.
+To move between directories we can use the `cd` command which stands for **change directory**. It should be followed by a directory name to change the working directory. To move to the `Desktop` directory:
 
 Input
 {: .label .label-green}
@@ -149,7 +192,7 @@ Output
 
 When typing a directory name, the case of the directory doesn't matter on Mac or PC but does on Linux.
 
-To move backwards in a directory use "cd .." which takes you back one step or "cd -" which jumps you back to the beginning and prints the directory (in the same way pwd would). 
+To move backwards in a directory use "cd .." which takes you back one step or "cd -" which moves you back to the previous directory and prints the directory (in the same way pwd would). 
 
 Input
 {: .label .label-green}

content/02-navigating-the-file-system.md

@@ -23,10 +23,11 @@ Output
 /Users/egrguric/desktop
 ~~~
 
-If a directory of filename is too long to type you can use "tab" to autocomplete directory and file names quickly.
+If a directory of filename is too long to type, you can type the first few letters to differentiate the directory and use "tab" to autocomplete directory and file names quickly.
 
 ## Creating a directory
-Let's try creating a directory and moving in and out of it.
+
+Let's try creating a directory and moving into it.
 
 Input
 {: .label .label-green}
@@ -47,7 +48,8 @@ Output
 
 ![Finder window with empty directory](myfirstdir.png)
 
-# Creating a file
+## Creating a file
+
 Now let's create a file. There are built-in text editors accessible through the shell. These editors are handy if you need to make a quick change from the shell or want to avoid having to open another program while you are working in it. Nano is a commonly used text editor.
 
 Input
@@ -64,6 +66,12 @@ myfile
 myfirstdirectory
 ~~~
 
+Note: The command line interpreter might confuse the spaces used in the names of files and directories with spaces used to separate arguments. Thus, it is better to avoid them in names of files and directories and use `.`, `-`, or `_` instead. If you need to refer to names of files or directories that have spaces or other special characters, you should surround the name in quotes (`""`).
+
+Note: `nano` is one of the least complex text editors in the Terminal. However, it is not popular among professional developer because it is not powerful and flexible enough. Since `nano` is installed by default on Unix shells, it is also a good choice for editing text files on remote machines. If you would like to know more about text editors, check out the [Introduction to Development Environment](https://ubc-library-rc.github.io/intro-development-environment/content/02.Editors.html) workshop.
+
+The convention for naming files is `filename.extension`, where `extension` identifies the type of the file. For example, `.txt` signals a plain text file while `.png` signals a PNG image, and so on. Each filetype has its own encoding and should be openned by an application that decodes the format. Using the correct extension might cause the operating system to try and open it with the appropriate application.
+
 ## Moving, copying, and deleting files and directories
 
 Now let's try moving our new file into our new directory to organize things a bit.
@@ -81,7 +89,7 @@ Output
 myfile.txt
 ~~~
 
-The mv function also renames files.
+The first argument contains the file we are moving and the second one is where it is to go. To rename a file, you can use the new name as the destination in a `mv` command. 
 
 Input
 {: .label .label-green}
@@ -95,14 +103,18 @@ Output
 mynewfile.txt
 ~~~
 
-The "cp" command lets you copy files, for example if you want to create a backup.
+Note: Be careful when specifying the file name. `mv` silently overwrite any existing files with the same name and could cause data loss. With the option `-i`, you can set `mv` to ask for confirmation before writing. 
+
+The "cp" command lets you copy files, for example if you want to create a backup, and it works very much like `mv`. 
 
 Input
 {: .label .label-green}
 ~~~
 $ cp mynewfile.txt backup_mynewfile.txt
 ~~~
 
+You can use the `-r` option to copy a directory and all its contents. `-r` specifies the recursive copying.
+
 And you can delete files through the "rm" command which stands for remove.
 
 Input
@@ -117,7 +129,9 @@ Output
 mynewfile.txt
 ~~~
 
-You can also delete directories but that requires an additional (and dangerous) flag. -r stands for "recursive" which essentially tells the command "rm" to keep going through all of the files that are inside the directory as well until there's nothing left.
+You can also delete directories but that requires an additional (and dangerous) flag. `-r` stands for "recursive" which essentially tells the command "rm" to keep going through all of the files that are inside the directory as well until there's nothing left. Be careful when using this option, as the Unix shell does not have a trash bin and deleted files are unlinked from the file system. Tools for finding and recovering deleted files exists, but there's no guarantee they will work in any particular situation. 
+
+Note: with `-i` option, the shell prompt before removal and we have a chance to checkl that we are deleting only the files we want to remove.
 
 Input
 {: .label .label-green}
@@ -133,6 +147,16 @@ Output
 
 ~~~
 
+### Using wildcards
+
+`*` is a wildcard, which matches zero or more characters. When the shell sees a wildcard, it expands the wildcard and creates a list of matching filesnames before running the command. For example, if you want to copy all the PDF files in a directory, you can run the following command:
+
+Input
+{: .label .label-green}
+~~~
+$ cp firstdirectory/*.pdf seconddirectory/
+~~~ 
+
 ## Reading files
 
 Let's make a few files to work with for the next activity.
@@ -161,7 +185,7 @@ Input
 $ ls -lh
 ~~~
 
-But you can also quickly open the entirety of the file from the command line.
+But you can also quickly open the entirety of the file from the command line. The `cat` command gets its name from 'concatenate' and prints the contents of files on the screen.
 
 Input
 {: .label .label-green}
@@ -197,6 +221,8 @@ Part 4
 Part 5
 ~~~
 
+If the file is large, you might want to get only the first few lines. WE can run another command called `head` for that. Using `-n 5` option with `head` tells it to print only the first 5 lines of the file. The command `head` defaults to showing the first ten lines in a file.
+
 Input
 {: .label .label-green}
 ~~~
@@ -218,9 +244,23 @@ Step 4
 
 ~~~
 
-You can ask head to show more or less as needed.
+You can use this method to quickly view all of a type of file in a directory.
+
+Input
+{: .label .label-green}
+~~~
+$ head *.txt
+~~~
 
-"Tail" works similarly
+Or only the first line of all of the files which might be useful if there are a lot of them. 
+
+Input
+{: .label .label-green}
+~~~
+$ head -n1 *.txt
+~~~
+
+`tail` works similarly and prints the end of your files.
 
 Input
 {: .label .label-green}
@@ -243,6 +283,8 @@ Step 6
 World domination!
 ~~~
 
+`less` can be used to read the content of a file **one screen** at a time. 
+
 Input
 {: .label .label-green}
 ~~~
@@ -266,22 +308,39 @@ Step 5
 :
 ~~~
 
-Use the enter key to move through the file and "q" to "quit" or exit the file.
+Use the enter key to move through the file and `q` to "quit" or exit the file.
 
-The command "head" defaults to showing the first ten lines in a file.
+## Finding things
 
-You can use this method to quickly view all of a type of file in a directory.
+`grep` is the name of a very useful command-line program that searches within files for a particular pattern. For example, to find lines that contain the number `3` in our file, we run:
 
 Input
 {: .label .label-green}
 ~~~
-$ head *.txt
+$ grep 3 file1.txt
+~~~
+Output
+{: .label .label-yellow}
+~~~
+Step 3
 ~~~
 
-Or only the first line of all of the files which might be useful if there are a lot of them. 
+To search for a phrase, we should put the phrase in quotes. 
+
+While `grep` finds lines in files, the `find` command find files within directories. 
 
 Input
 {: .label .label-green}
 ~~~
-$ head -n1 *.txt
+$ find .
+~~~
+
+As `.` means the current working directory, `find`'s output is the names of all the files and directories under the current working directory. With `-name` option, we can specify conditions on the names of the files or directories:
+
+Input
+{: .label .label-green}
 ~~~
+$ find . -name *.txt
+~~~
+
+