Skip to content

Commit b247cb3

Browse files
committed
Merge pull request #2 from dmlond/master
Here are the Answers to 5-11-15
2 parents 6d611c4 + 2e2f490 commit b247cb3

File tree

9 files changed

+220
-109
lines changed

9 files changed

+220
-109
lines changed

.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
1+
*bwa*
2+
.Rhistory
13
.vagrant
24
packer_cache/
5+
*sai*
6+
*fasta*
7+
*fastq*
8+
*sam*

conf/vagrant/Vagrantfile

+3-90
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,11 @@
1-
# -*- mode: ruby -*-
2-
# vi: set ft=ruby :
3-
41
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
52
VAGRANTFILE_API_VERSION = "2"
63

74
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
8-
# All Vagrant configuration is done here. The most common configuration
9-
# options are documented and commented below. For a complete reference,
10-
# please see the online documentation at vagrantup.com.
11-
12-
# Every Vagrant virtual environment requires a box to build off of.
5+
6+
# We are using Ubuntu 14.04 "Trusty Tahr"
137
config.vm.box = "ubuntu/trusty64"
148

15-
# Disable automatic box update checking. If you disable this, then
16-
# boxes will only be checked for updates when the user runs
17-
# `vagrant box outdated`. This is not recommended.
18-
# config.vm.box_check_update = false
19-
20-
# Create a forwarded port mapping which allows access to a specific port
21-
# within the machine from a port on the host machine. In the example below,
22-
# accessing "localhost:8080" will access port 80 on the guest machine.
23-
# config.vm.network "forwarded_port", guest: 80, host: 8080
24-
25-
# Create a private network, which allows host-only access to the machine
26-
# using a specific IP.
27-
# config.vm.network "private_network", ip: "192.168.33.10"
28-
29-
# Create a public network, which generally matched to bridged network.
30-
# Bridged networks make the machine appear as another physical device on
31-
# your network.
32-
# config.vm.network "public_network"
33-
34-
# If true, then any SSH connections made will enable agent forwarding.
35-
# Default value: false
36-
# config.ssh.forward_agent = true
37-
389
# Share an additional folder to the guest VM. The first argument is
3910
# the path on the host to the actual folder. The second argument is
4011
# the path on the guest to mount the folder. And the optional third
@@ -44,79 +15,21 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
4415
# Provider-specific configuration so you can fine-tune various
4516
# backing providers for Vagrant. These expose provider-specific options.
4617
# Example for VirtualBox:
47-
#
4818
config.vm.provider "virtualbox" do |vb|
19+
4920
# (Don't) boot with headless mode
5021
vb.gui = true
5122

5223
# Use VBoxManage to customize the VM. For example to change memory:
5324
vb.customize ["modifyvm", :id, "--memory", "2048"]
5425
end
55-
#
56-
# View the documentation for the provider you're using for more
57-
# information on available options.
58-
59-
# Enable provisioning with CFEngine. CFEngine Community packages are
60-
# automatically installed. For example, configure the host as a
61-
# policy server and optionally a policy file to run:
62-
#
63-
# config.vm.provision "cfengine" do |cf|
64-
# cf.am_policy_hub = true
65-
# # cf.run_file = "motd.cf"
66-
# end
67-
#
68-
# You can also configure and bootstrap a client to an existing
69-
# policy server:
70-
#
71-
# config.vm.provision "cfengine" do |cf|
72-
# cf.policy_server_address = "10.0.2.15"
73-
# end
7426

7527
# Enable provisioning with Puppet stand alone. Puppet manifests
7628
# are contained in a directory path relative to this Vagrantfile.
7729
# You will need to create the manifests directory and a manifest in
7830
# the file default.pp in the manifests_path directory.
79-
#
8031
config.vm.provision "puppet" do |puppet|
8132
puppet.manifests_path = "manifests"
8233
puppet.manifest_file = "default.pp"
8334
end
84-
85-
# Enable provisioning with chef solo, specifying a cookbooks path, roles
86-
# path, and data_bags path (all relative to this Vagrantfile), and adding
87-
# some recipes and/or roles.
88-
#
89-
# config.vm.provision "chef_solo" do |chef|
90-
# chef.cookbooks_path = "../my-recipes/cookbooks"
91-
# chef.roles_path = "../my-recipes/roles"
92-
# chef.data_bags_path = "../my-recipes/data_bags"
93-
# chef.add_recipe "mysql"
94-
# chef.add_role "web"
95-
#
96-
# # You may also specify custom JSON attributes:
97-
# chef.json = { mysql_password: "foo" }
98-
# end
99-
100-
# Enable provisioning with chef server, specifying the chef server URL,
101-
# and the path to the validation key (relative to this Vagrantfile).
102-
#
103-
# The Opscode Platform uses HTTPS. Substitute your organization for
104-
# ORGNAME in the URL and validation key.
105-
#
106-
# If you have your own Chef Server, use the appropriate URL, which may be
107-
# HTTP instead of HTTPS depending on your configuration. Also change the
108-
# validation key to validation.pem.
109-
#
110-
# config.vm.provision "chef_client" do |chef|
111-
# chef.chef_server_url = "https://api.opscode.com/organizations/ORGNAME"
112-
# chef.validation_key_path = "ORGNAME-validator.pem"
113-
# end
114-
#
115-
# If you're using the Opscode platform, your validator client is
116-
# ORGNAME-validator, replacing ORGNAME with your organization name.
117-
#
118-
# If you have your own Chef Server, the default validation client name is
119-
# chef-validator, unless you changed the configuration.
120-
#
121-
# chef.validation_client_name = "ORGNAME-validator"
12235
end

conf/vagrant/manifests/default.pp

+20-3
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@
3838
creates => '/usr/local/src/bwa-0.7.12.tar.bz2',
3939
require => Package[ 'wget' ];
4040
'unzip_bwa':
41-
command => 'bunzip2 bwa-0.7.12.tar.bz2',
41+
command => 'bunzip2 --keep bwa-0.7.12.tar.bz2',
4242
cwd => '/usr/local/src',
4343
creates => '/usr/local/src/bwa-0.7.12.tar',
4444
require => [ Exec[ 'dl_bwa' ], Package[ 'bzip2' ] ];
@@ -64,7 +64,7 @@
6464
creates => '/usr/local/src/samtools-1.2.tar.bz2',
6565
require => Package[ 'wget' ];
6666
'unzip_samtools':
67-
command => 'bunzip2 samtools-1.2.tar.bz2',
67+
command => 'bunzip2 --keep samtools-1.2.tar.bz2',
6868
cwd => '/usr/local/src',
6969
creates => '/usr/local/src/samtools-1.2.tar',
7070
require => [ Exec[ 'dl_samtools' ], Package[ 'bzip2' ] ];
@@ -82,5 +82,22 @@
8282
command => 'ln -s /usr/local/src/samtools-1.2/samtools /usr/local/bin/samtools',
8383
creates => '/usr/local/bin/samtools',
8484
require => Exec[ 'make_samtools' ];
85+
86+
# clone the project repo
87+
'clone_repo':
88+
command => 'git clone https://github.com/dmlond/arangs2015.git',
89+
cwd => '/home/vagrant',
90+
creates => '/home/vagrant/arangs2015',
91+
require => Package[ 'git' ];
92+
'chown_repo':
93+
command => 'sudo chown -R vagrant /home/vagrant/arangs2015',
94+
require => Exec[ 'clone_repo' ];
95+
'rm_repo_data':
96+
command => 'rm -rf /home/vagrant/arangs2015/data',
97+
require => Exec[ 'clone_repo' ];
98+
'symlink_data':
99+
command => 'ln -s /vagrant_data /home/vagrant/arangs2015/data',
100+
require => Exec[ 'rm_repo_data' ];
101+
85102

86-
}
103+
}

docs/2015-05-11/git/Worksheet.md

+82
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,116 @@
1+
create a ~/.gitconfig file with our [example](https://github.com/dmlond/arangs2015/blob/master/docs/2015-05-11/git/example_gitconfig)
12
- Clone your personal fork of our repository onto your machine
3+
choose the SSH clone url
4+
```bash
5+
$ git clone your_ssh_clone_url
6+
```
27
- Use git to show information about the commit history for the project
8+
```bash
9+
$ git log
10+
```
311
- Show information about files changed during the commits
12+
```bash
13+
$ git log --name-only
14+
```
415
- Show information about files changed, with the status of each change for
516
commit 4c4ff5ece4c3132c6fa29f3e826e3272686c78e0 (try 4c4ff5ece4)
17+
```bash
18+
$ git log -1 -U --name-status -c 4c4ff5ece4c3132c6fa29f3e826e3272686c78e0
19+
$ git log -1 -U --name-status 4c4ff5ece4
20+
$ git show 4c4ff5ece4
21+
```
622
- add a file to the repository (use your favorite text editor)
723
- use git status to find out the state of the repository
824
- try git status --porcelain
925
- What if you didnt like this file, how could you get rid of it (dont do it)
26+
```bash
27+
$ git checkout -- filename
28+
```
1029
- stage this file change
30+
```bash
31+
$ git add filename
32+
```
1133
- what do git status, and git status --porcelain show
1234
- How could you get rid of this file now? Go ahead and fully delete the file.
35+
```bash
36+
$ git reset HEAD file
37+
$ rm file
38+
```
1339
- Create a new file, and make sure it gets committed to the repository
1440
- What remote git repositories does your git repository know about?
41+
```bash
42+
$ git remote -v
43+
```
1544
- push your new changes to the file up to your github repository
45+
```bash
46+
$ git push
47+
```
1648
- Find the newly committed file in your github repository. Edit it using
1749
Github and make a change.
1850
- Go back to the commandline and use git to pull these changes
51+
```bash
52+
$ git pull
53+
```
1954
- unix rm the file you created.
55+
```bash
56+
$ rm file
57+
```
2058
- What do git status and git status --porcelain show?
2159
- Can you get it back?
60+
```bash
61+
$ git checkout -- file
62+
```
2263
- rm it again, then stage the removal.
64+
```bash
65+
$ rm file
66+
$ git rm file
67+
```
2368
- get it back again
69+
```bash
70+
$ git reset HEAD file
71+
$ git checkout -- file
72+
```
2473
- git rm the file. How is this different than using unix rm?
74+
you can git rm a file, and it will physically remove it and git rm it at
75+
the same time
76+
2577
- Commit the removal of the file, and push the changes up to Github
78+
```bash
79+
$ git commit
80+
```
2681
- Use the Github Web interface to inspect changes for this file. Can you
2782
still find the contents of the file in Github? Do you think it is a
2883
good idea to store usernames and passwords in publicly available GitHub accounts?
84+
85+
NO, because GitHub allows you to review the history of your files, where they can
86+
see the passwords. There are ways to remove them, but it is not easy
87+
2988
- See how far you can go with this by changing and staging various file combinations, then changing already staged files, then reverse staged changes, untracked changes of files with staged changes, and fully revert some of the files.
3089
- Avoiding what git command will ensure that none of these changes ever
3190
make it into the git log for the repository, or get pushed to a remote
91+
git commit
3292
- What if you had committed (without pushes) a bunch of changes to your local fork of the repo, but decided that you didnt like any of it, and would like to get rid of all those commits and revert to the state of the repo as it exists in GitHub. What would be the easiest way to do this?
93+
```bash
94+
$ cd ..
95+
$ rm -rf arangs2015
96+
$ git clone SSH-CLONE-URL
97+
```
3398
- Edit one of the files, but do not add the changes. Create a branch called 'try_bowtie'. Use git status to find out the state of the repo. Add and commit
3499
your changes to the branch. Use git to find out all the branches you have made (you might see branches pulled in when you forked the repository from someone else). Change to the master branch. Use git status to find out the state of the repository. Can you find your changes? Switch back to the 'try_bowtie' branch. Can you find your changes now? Switch back to the master branch. Merge try_bowtie in to the master branch. Use git status to find the status of the master branch. Commit and push these changes to Github. Use the Github web interface to find out if the branch was pushed. Remove the 'try_bowtie' branch from your repository.
100+
```bash
101+
$ echo "CHANGED" >> changed_file
102+
$ git checkout -b try_bowtie
103+
$ git status
104+
$ git add changed_file
105+
$ git commit changed_file
106+
$ git branch
107+
$ git checkout master
108+
$ git status
109+
$ git checkout try_bowtie
110+
$ git status
111+
$ git checkout master
112+
$ git merge try_bowtie
113+
$ git commit
114+
$ git push
115+
$ git branch -D try_bowtie
116+
```

docs/2015-05-11/github/Worksheet.md

+17-10
Original file line numberDiff line numberDiff line change
@@ -3,24 +3,31 @@ GitHub Worksheet
33

44
* Create your personal Github Account
55
* Find the Arangs2015 repository
6-
- Does it have an Open Source License?
7-
- Who Owns the Repository?
8-
- How many people have forked the repository?
9-
- How many contributors are working on this repository?
10-
- Who are they?
11-
- How many commits have been made? Who made the latest commit?
6+
- Does it have an Open Source License? MIT
7+
- Who Owns the Repository? dmlond
8+
- How many people have forked the repository? 11 (and counting)
9+
- How many contributors are working on this repository? 2
10+
- Who are they? dmlond, rvosa
11+
- How many commits have been made? 73 Who made the latest commit? dmlond
1212
* Take a look at the data directory.
13-
- Where did we get the data that we analyzed for our pipeline?
14-
- Do we make it easy for you to download the same data that we used?
13+
- Where did we get the data that we analyzed for our pipeline? sra dnanexus
14+
- Do we make it easy for you to download the same data that we used? yes by providing links to the actual files
1515
* Go back to the bin directory
16-
- What is consistently found in each directory?
16+
- What is consistently found in each directory? Readme.md
1717
- View the pipeline.sh file
1818
- View it in raw mode
19-
- Who can you blame for this file?
19+
- Who can you blame for this file? dmlond and rvosa
2020
- What is its history?
2121
* Go back to the repository root
2222
- What is the git clone url for the repository?
23+
https https://github.com/dmlond/arangs2015.git
24+
ssh [email protected]:dmlond/arangs2015.git
25+
You should clone the ssh one to push stuff back
2326
* Fork our Repository
2427
* Check to make sure the `git` executable is available. If not, install it.
28+
sudo yum install git
2529
* [Create an SSH key](https://help.github.com/articles/generating-ssh-keys/) so that you can commit and `push` changes back to your repository.
30+
```bash
31+
$ ssh-keygen -t rsa
32+
```
2633
* [Merge the changes made upstream into your local fork](https://help.github.com/articles/merging-an-upstream-repository-into-your-fork/)
+15-6
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,24 @@
11
We are going to try to run [pipeline.sh](https://github.com/dmlond/arangs2015/blob/master/bin/pipeline.sh). Have a look at the script.
22

3-
- How many values does the `$FASTQS` variable hold?
4-
- How many values will the `$SAIS` variable hold when running the script? Why?
5-
- How many times will the `$SAM` variable be assigned a value? Is it the same value? Why?
3+
- How many values does the `$FASTQS` variable hold? 2
4+
- How many values will the `$SAIS` variable hold when running the script? 2 Why? because it loops over $FASTQS
5+
- How many times will the `$SAM` variable be assigned a value? 2 Is it the same value? No Why? Because it uses a different value of $OUTFILE each time
66
- What's the function of the if/else statements, why would they be there?
7+
to check for the existence of files and not run parts of the pipeline if they already exist
78
- Which programs need to be installed to run pipeline.sh?
9+
bwa, samtools
810
- Which versions of these programs do we need (consult the download* scripts)?
9-
- Download and (try to) compile these programs.
11+
bwa 0.7.12
12+
samtools 1.2
13+
- Download and (try to) compile these programs.
1014
- Are there any libraries missing?
1115
- If there are missing libraries, try to install these with the package manager (`apt-get`)
1216
- Make sure the compiled programs are on the `$PATH`.
13-
- Download the data. If you use the download* scripts for this, where will the data end up?
14-
- Will the pipeline.sh be able to find the data there?
17+
```bash
18+
export PATH="/dir/of/program:${PATH}"
19+
```
20+
- Download the data. If you use the download* scripts for this, where will the data end up? ../data
21+
- Will the pipeline.sh be able to find the data there?
22+
yes, because it hardcodes it
1523
- Does it matter where you run the pipeline?
24+
yes, because it hardcodes data relative to the bin directory

docs/2015-05-11/mindmap_day1.xmind

72.2 KB
Binary file not shown.

docs/2015-05-11/shell/README.md

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
On Linux, every command (by first approximation) is an executable that resides somewhere on the file system.
2+
When you type a command, the operating system scans a select number of folders that might contain the
3+
executable that corresponds with the command you typed. This list of folders is defined by the environment
4+
variable `$PATH`. So, if you type a command that corresponds with an executable that is not in any of the
5+
folders that are in the `$PATH`, the command will not be found. It does not matter that you are INSIDE the
6+
folder with the executable RIGHT THERE: it is still not going to be found. How to change that? By updating
7+
the value of the `$PATH` environment variable. Like so:
8+
9+
`export PATH=/folder/of/executable:$PATH`
10+
11+
A couple of important points:
12+
- the folders in the list are separated by `:`
13+
- using the bash shell (!) the list is read from left to right. In CShell (`csh`) it is read right to left.
14+
- in general, shells split "words" on spaces, unless the whole sentence is inside quotes. So, if any of the
15+
folders has spaces in it (`/Documents and Settings/`) you have to quote the list, otherwise it will stop
16+
at `/Documents`
17+
- If you want to put a word right after `$PATH`, you can delimit the variable name with curly braces:
18+
`${PATH}foo`

0 commit comments

Comments
 (0)