You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: presentation/linux_bash_metacentrum_course.tex
+31-30Lines changed: 31 additions & 30 deletions
Original file line number
Diff line number
Diff line change
@@ -1170,7 +1170,7 @@ \section{Command line}
1170
1170
\end{itemize}
1171
1171
\end{frame}
1172
1172
1173
-
\begin{frame}{The command line can has various look and feel\ldots}{Change colors, font size, etc. for your terminal to like it more and work comfortably}
1173
+
\begin{frame}{The command line can have various look and feel\ldots}{Change colors, font size, etc. for your terminal to like it more and work comfortably}
1174
1174
\begin{center}
1175
1175
\includegraphics[height=6cm]{terminals.png}
1176
1176
\end{center}
@@ -1184,7 +1184,7 @@ \subsection{Screen}
1184
1184
\item Sometimes number of connections to the server is limited
1185
1185
\item\texttt{screen} is solution --- virtual terminals
1186
1186
\item Launch \texttt{screen} to start new screen terminal, read some info, confirm by \textbf{Space key} or \textbf{Enter}
1187
-
\item To detach from the screen press \texttt{Ctrl+A, D} --- screen is still running in background --- you can even log off
1187
+
\item To detach from the screen press \texttt{Ctrl+A, D} (quickly press \texttt{Ctrl+A}, release, press \texttt{D}) --- screen is still running in background --- you can even log off
1188
1188
\item To return back to running screen use \texttt{screen -r} --- if only one screen is running, you get back to it
1189
1189
\item If more screens are running, use \texttt{screen -r 1234} (the number is seen from \texttt{screen -r})
1190
1190
\item To cancel running screen press \texttt{Ctrl+D} (or type \texttt{exit} or \texttt{logout})
@@ -1484,8 +1484,7 @@ \subsection{Archives}
1484
1484
*.bz2 & bzip2 file & bunzip2 archive.bz2\\
1485
1485
*.xz & xz -zv file & xz -d archive.xz\\
1486
1486
*.lzma & lzma file & unlzma archive.lzma\\
1487
-
*.zip & zip -r archive.zip file1 file2 & unzip archive.zip\\
1488
-
*.rar & rar a archive.rar file1 file2 & unrar x archive.rar
1487
+
*.zip & zip -r archive.zip file1 file2 & unzip archive.zip
1489
1488
\end{tabular}
1490
1489
\end{center}
1491
1490
\end{frame}
@@ -1509,14 +1508,14 @@ \subsection{Archives}
1509
1508
1510
1509
\begin{frame}{Tasks with archives}
1511
1510
\begin{enumerate}
1512
-
\item Compress (and decompress) text file \texttt{Oxalis\_HybSeq\_nrDNA\_selection\_} \texttt{alignment.fasta} (target enrichment sequencing data of South African \textit{Oxalis}) from \texttt{scripts\_data} with various compressing tools.
1511
+
\item Compress (and decompress) text file \texttt{Oxalis\_HybSeq\_nrDNA\_selection\_} \texttt{alignment.fasta} from \texttt{scripts\_data} with various compressing tools.
1513
1512
\item Compare sizes of original file and compressed outputs.
1514
1513
\item Compress (and decompress) all \texttt{foto\_oxalis\_*.jpg} (\textit{Oxalis} photos) together from \texttt{scripts\_data} with various compressing tools.
1515
1514
\item Compare sizes of original files and compressed outputs.
1516
1515
\item Which compression tool seems to be the best? In terms of compressing ratio and time needed for compression.
1517
1516
\item Is more effective compression of text files or images? Why?
1518
1517
\item Why is even plain \texttt{tar} (without compression, it requires \texttt{gzip}, \texttt{bzip2}, \texttt{xz} or \texttt{lzma} to add compression) useful with FAT disks?
1519
-
\item Search the Internet to find out how to unpack \texttt{arj} archives from command line.
1518
+
\item Search the Internet to find out how to unpack \texttt{arj} and \texttt{rar} archives from command line.
1520
1519
\end{enumerate}
1521
1520
\end{frame}
1522
1521
@@ -1599,7 +1598,8 @@ \subsection{Searching}
1599
1598
man find
1600
1599
\end{bashcode}
1601
1600
\begin{itemize}
1602
-
\item\texttt{find} is extremely versatile and useful tool
1601
+
\item\texttt{find} is extremely versatile and useful tool --- master it
1602
+
\item\texttt{-print} is default action --- if it is missing and there is no other actions, results are printed to the screen
1603
1603
\end{itemize}
1604
1604
\end{frame}
1605
1605
@@ -1832,7 +1832,7 @@ \subsection{Variables}
1832
1832
\end{itemize}
1833
1833
\item\alert{\texttt{||}} --- second command is launched when first command fails (has non zero exit status):\\\texttt{cd newdir || \textbraceleft~mkdir newdir \&\& cd newdir; \textbraceright}
1834
1834
\begin{itemize}
1835
-
\item Easy way how to do something when previous command fails, either exit script (\texttt{\ldots{ }|| exit 1}) or somehow fix it (see above), report problem (\texttt{\ldots{ }|| \textbraceleft echo \enquote{It failed!}\&\& exit 1; \textbraceright}) or so
1835
+
\item Easy way how to do something when previous command fails, either exit script (\texttt{\ldots{ }|| exit 1}) or somehow fix it (see above), report problem (\texttt{\ldots{ }|| \textbraceleft echo "It failed!"\&\& exit 1; \textbraceright}) or so
1836
1836
\end{itemize}
1837
1837
\item\alert{\texttt{|}} --- pipe --- redirects standard output of one command into standard input of second command: compare \texttt{mount} and \texttt{mount | column -t}
1838
1838
\begin{itemize}
@@ -1960,7 +1960,7 @@ \subsection{Variables}
1960
1960
# sequenced) in genomic VCF with multiple individuals
1961
1961
zcat arabidopsis.vcf.gz | grep -o "DP=[0-9]\+" | sort | less
1962
1962
# Convert DNA sequence from FASTQ to FASTA (two of many options)
1963
-
# (discard FASTQ quality scores and keep only the sequence and its name)
1963
+
# Discard FASTQ quality scores and keep only the sequence and its name
1964
1964
bzcat Oxalis_hirta_R1.fastq.bz2 | sed -n '1~4s/^@/>/p;2~4p' > \
\begin{frame}[allowframebreaks]{CESNET and MetaCentrum}
4298
4299
\label{CESNET}
4299
4300
\begin{itemize}
4300
-
\item\href{https://www.cesnet.cz/?lang=en}{CESNET} is organization of Czech universities, Academy of Science and other organizations taking care about Czech backbone Internet, one of world leading institutions of this type
4301
-
\item CESNET provides various \href{https://www.cesnet.cz/services/?lang=en}{services}
4301
+
\item\href{https://www.cesnet.cz/?lang=en}{CESNET} (\href{https://www.cesnet.cz/}{česky}) is organization of Czech universities, Academy of Science and other organizations taking care about Czech backbone Internet, one of world leading institutions of this type
4302
+
\item CESNET provides various \href{https://www.cesnet.cz/services/?lang=en}{services} (\href{https://www.cesnet.cz/sluzby/}{česky})
\item Large \href{https://www.cesnet.cz/services/data-storage/?lang=en}{data storage}
4305
-
\item\href{https://www.cesnet.cz/services/filesender/?lang=en}{FileSender} to be able to send up to 1.9~TB file
4306
-
\item\href{https://www.metacentrum.cz/en/Sluzby/Cloud/}{Cloud} --- computing (HPC) cloud similar to e.g. Amazon Elastic Compute Cloud (EC2), Google Compute Engine or Microsoft Azure
4307
-
\item\href{https://www.cesnet.cz/services/owncloud/?lang=en}{ownCloud} to backup and/or sync data across devices (default capacity is 100~GB, user may ask for more) --- similar to e.g. Dropbox, Google Drive or Microsoft OneDrive
\item Large \href{https://www.cesnet.cz/services/data-storage/?lang=en}{data storage} (\href{https://www.cesnet.cz/sluzby/datova-uloziste/}{česky})
4306
+
\item\href{https://www.cesnet.cz/services/filesender/?lang=en}{FileSender} (\href{https://www.cesnet.cz/sluzby/filesender/}{česky}) to be able to send up to 1.9~TB file
4307
+
\item\href{https://www.metacentrum.cz/en/Sluzby/Cloud/}{Cloud} (\href{https://www.metacentrum.cz/cs/Sluzby/Cloud/}{česky}) --- computing (HPC) cloud similar to e.g. Amazon Elastic Compute Cloud (EC2), Google Compute Engine or Microsoft Azure
4308
+
\item\href{https://www.cesnet.cz/services/owncloud/?lang=en}{ownCloud} (\href{https://www.cesnet.cz/sluzby/owncloud/}{česky}) to backup and/or sync data across devices (default capacity is 100~GB, user may ask for more) --- similar to e.g. Dropbox, Google Drive or Microsoft OneDrive
4308
4309
\begin{itemize}
4309
4310
\item It is possible to connect by webDAV to ownCloud (slide \ref{transfers}) --- many applications support it
4310
4311
\item It is possible to share calendars and/or address books via calDav and cardDav among devices and/or people
\item To use MetaCentrum fill registration form \url{https://metavo.metacentrum.cz/en/application/form}
4322
+
\item To use MetaCentrum fill registration form \url{https://metavo.metacentrum.cz/en/application/form} (\href{https://metavo.metacentrum.cz/cs/application/}{česky})
4322
4323
\item To use data storage fill registration form \url{https://einfra.cesnet.cz/perun-registrar-fed/?vo=storage}
4323
4324
\item After registration for MetaCentrum, user can join MetaCloud via \url{https://perun.metacentrum.cz/fed/registrar/?vo=meta&group=metacloud}
4324
-
\item Users not having access to \href{https://www.eduid.cz/en/index}{EduID} have to register first at \href{https://www.eduid.cz/cs/hostel}{HostelID} (available only in Czech)
4325
+
\item Users not having access to \href{https://www.eduid.cz/en/index}{EduID} (\href{https://www.eduid.cz/cs/index}{česky}) have to register first at \href{https://www.eduid.cz/cs/hostel}{HostelID} (only in Czech)
4325
4326
\end{itemize}
4326
-
\item Information about data storage \url{https://du.cesnet.cz/en/start} contains detailed usage instructions
4327
-
\item Information about MetaCentrum \url{https://www.metacentrum.cz/en/} and wiki \url{https://wiki.metacentrum.cz/wiki/Main_Page} (main information for users containing all needed documentation)
4327
+
\item Information about data storage \url{https://du.cesnet.cz/en/start} (\href{https://du.cesnet.cz/cs/start}{česky}) contains detailed usage instructions
4328
+
\item Information about MetaCentrum \url{https://www.metacentrum.cz/en/} (\href{https://www.metacentrum.cz/cs/}{česky}) and wiki \url{https://wiki.metacentrum.cz/wiki/Main_Page} (\href{https://wiki.metacentrum.cz/wiki/Hlavn\%C3\%AD_strana}{česky}) (main information for users containing all needed documentation)
4328
4329
\item Information about MetaCloud \url{https://wiki.metacentrum.cz/wiki/Kategorie:Clouds}
4329
4330
\item Also available is Galaxy \url{https://galaxy.metacentrum.cz/galaxy/} (same login as to MetaCentrum) --- web based bioinformatics framework (more information at \href{https://wiki.metacentrum.cz/wiki/Galaxy}{wiki})
4330
-
\item Current state and usage of resources is available at \url{https://metavo.metacentrum.cz/en/}
4331
-
\item Manage your user account at \url{http://metavo.metacentrum.cz/en/myaccount/}
4331
+
\item Current state and usage of resources is available at \url{https://metavo.metacentrum.cz/en/} (\href{https://metavo.metacentrum.cz/cs/}{česky})
4332
+
\item Manage your user account at \url{http://metavo.metacentrum.cz/en/myaccount/} (\href{https://metavo.metacentrum.cz/cs/myaccount/}{česky})
4332
4333
\item Personal view on actual resources and running tasks is at \url{https://metavo.metacentrum.cz/pbsmon2/person}
4333
4334
\item List of available applications \url{https://wiki.metacentrum.cz/wiki/Kategorie:Applications}
4334
4335
\item MetaCentrum has 11~\href{https://wiki.metacentrum.cz/wiki/Frontend}{frontends} where users log and thousands of computers doing the calculations --- they are not accessed directly to run task
@@ -4344,7 +4345,7 @@ \subsection{Usage}
4344
4345
4345
4346
\begin{frame}[fragile]{MetaCentrum usage}
4346
4347
\begin{itemize}
4347
-
\item User can transfer data on one of \href{https://wiki.metacentrum.cz/wiki/Frontend}{frontends} (next slide; or data storage) by e.g. \texttt{scp} or \href{https://winscp.net/}{WinSCP} from Windows or \href{https://filezilla-project.org/}{FileZilla} from anywhere
4348
+
\item User can transfer data on one of \href{https://wiki.metacentrum.cz/wiki/Frontend}{frontends} (\href{https://wiki.metacentrum.cz/wiki/Celni_uzel}{česky}) (next slide; or data storage) by e.g. \texttt{scp} or \href{https://winscp.net/}{WinSCP} from Windows or \href{https://filezilla-project.org/}{FileZilla} from anywhere
4348
4349
\item Same credentials are used for all frontends, for SSH login as well as file transmissions
4349
4350
\end{itemize}
4350
4351
\vfill
@@ -4364,7 +4365,7 @@ \subsection{Usage}
4364
4365
\begin{frame}{File transfers to MetaCentrum}
4365
4366
\begin{itemize}
4366
4367
\item Graphical applications: e.g. \href{https://filezilla-project.org/}{FileZilla} or from most of Linux file managers
4367
-
\item Protocol is SSH/SSH2/SFTP/SCP, port 22, server is selected \href{https://wiki.metacentrum.cz/wiki/Frontend}{frontend's} address (e.g. \texttt{tarkil.metacentrum.cz}) --- select any and keep using it
4368
+
\item Protocol is SSH/SSH2/SFTP/SCP, port 22, server is selected \href{https://wiki.metacentrum.cz/wiki/Frontend}{frontend's} (\href{https://wiki.metacentrum.cz/wiki/Celni_uzel}{česky}) address (e.g. \texttt{tarkil.metacentrum.cz}) --- select any and keep using it
4368
4369
\item All servers are accessible under domain \texttt{*.metacentrum.cz}: \texttt{skirit}, \texttt{perian}, \texttt{onyx}, \texttt{zuphux} (located in Brno), \texttt{alfrid}, \texttt{nympha}, \texttt{minos} (in Pilsen), \texttt{tarkil} (in Prague), and \texttt{tilia} (in Průhonice) --- so that e.g. \texttt{tarkil.grid.cesnet.cz} is synonymous to \texttt{tarkil.metacentrum.cz}
4369
4370
\item See slide \ref{transfers} and following to command-line transfers of files
\itemSee \href{https://wiki.metacentrum.cz/wiki/Beginners_guide}{beginners guide} (\href{https://wiki.metacentrum.cz/wiki/Pruvodce_pro_zacatecniky}{česky}) and \href{https://wiki.metacentrum.cz/wiki/Categorized_list_of_topics}{list of topics} (\href{https://wiki.metacentrum.cz/wiki/Rozcestnik}{česky})
4427
4428
\item Personal view \url{https://metavo.metacentrum.cz/pbsmon2/person} has nice overview of available resources and tasks and allows comfortable construction of submission command
4428
4429
\end{itemize}
4429
4430
\vfill
@@ -4631,7 +4632,7 @@ \subsection{Data storage}
4631
4632
4632
4633
\begin{frame}{CESNET data storage}
4633
4634
\begin{itemize}
4634
-
\item Read documentation \url{https://du.cesnet.cz/} and connection instructions \url{https://du.cesnet.cz/en/navody/sluzby/start}
4635
+
\item Read documentation \url{https://du.cesnet.cz/} and connection instructions \url{https://du.cesnet.cz/en/navody/sluzby/start} (\href{https://du.cesnet.cz/cs/navody/sluzby/start}{česky})
4635
4636
\item Generally, it is possible to connect via FTPS, NFS, SAMBA (Windows network drive), SCP/SFTP or SSH --- more options how to get to same resource
4636
4637
\item See slide~\ref{CESNET} for general information and \ref{transfers} for connecting information
4637
4638
\item After logging via SSH, it is possible to work as on any other server
@@ -4645,7 +4646,7 @@ \subsection{Data storage}
4645
4646
4646
4647
\begin{frame}[fragile]{Shared space on CESNET data storage}
4647
4648
\begin{itemize}
4648
-
\item Users \href{https://du.cesnet.cz/en/uzivatelska_podpora/start}{can ask} for creation of shared space
4649
+
\item Users \href{https://du.cesnet.cz/en/uzivatelska_podpora/start}{can ask} (\href{https://du.cesnet.cz/cs/uzivatelska_podpora/start}{česky}) for creation of shared space
4649
4650
\begin{itemize}
4650
4651
\item Normally, the space is private only for particular user
4651
4652
\item Groups allow more users to share data
@@ -4687,7 +4688,7 @@ \subsection{Git principle}
4687
4688
4688
4689
\begin{frame}[allowframebreaks]{Git and version control}
4689
4690
\begin{itemize}
4690
-
\item\href{https://git-scm.com/}{Git} is \href{https://en.wikipedia.org/wiki/Version_control}{version controlling} system (nowadays the most common) --- it traces changes among all versions --- absolutely crucial for any software development
4691
+
\item\href{https://git-scm.com/}{Git} is \href{https://en.wikipedia.org/wiki/Version_control}{version controlling} (\href{https://cs.wikipedia.org/wiki/Verzov\%C3\%A1n\%C3\%AD}{česky}) system (nowadays the most common) --- it traces changes among all versions --- absolutely crucial for any software development
4691
4692
\item Older (nowadays not so common) version controlling systems is Subversion (SVN), there are many more (Bazaar, Mercurial,~\ldots)
4692
4693
\item Probably the best textbook for Git is \href{https://git-scm.com/book/en/v2}{Chacon's Pro Git}
4693
4694
\begin{itemize}
@@ -4722,7 +4723,7 @@ \subsection{Git principle}
4722
4723
\item\href{https://github.com/}{GitHub} is currently probably the most popular platform to host development of open-source projects, see \href{https://help.github.com/}{documentation}
4723
4724
\item Probably second most common on-line service providing Git repository is \href{https://about.gitlab.com/}{GitLab}
4724
4725
\begin{itemize}
4725
-
\item User can create an account on GitLab (as on GitHub), or download and install GitLab system on own server (like \href{https://sorbus.ibot.cas.cz/en/gitlab}{we did})
4726
+
\item User can create an account on GitLab (as on GitHub), or download and install GitLab system on own server (like \href{https://sorbus.ibot.cas.cz/en/gitlab}{we did}, \href{https://sorbus.ibot.cas.cz/cs/gitlab}{česky})
4726
4727
\end{itemize}
4727
4728
\item\href{https://sourceforge.net/}{SourceForge} used to be more popular in the past, still harbours plenty of interesting projects
4728
4729
\item Others are e.g. \href{https://bitbucket.org/}{Bitbucket}, \href{https://www.codebasehq.com/}{Codebase},~\ldots
0 commit comments