.. _linux_guide: ############################# Basic Linux Guide ############################# All of the CRC HPC infrastructure is running some form of Red Hat Enterprise Linux. RHEL (Red Hat Enterprise Linux) is a Linux distribution, which is a flavor of a Unix-like operating system centered around the Linux kernel. To begin using the CRC HPC infrastructure, you will need a basic understanding of how to operate and navigate a terminal in a Linux environment. This page will show you the basics, but we highly recommend learning on your own some basic Bash commands. The following is only intended to get you started, a more in depth guide can be found on the `Linux Journey `_ website. .......................... Navigating File System ================================= When first logging in, you will be placed into what is known as your Home space. This can be abbreviated with `~`. Your prompt will most likely look something like this:: [$USER@crcfe0X.crc.nd.edu ~]$ _ In your prompt you should see your ND NetID instead of ``$USER``. Paths """"""""""""""""""""""""""""""""" A path is simply the location in a file system of a particular file or directory (folder). There are two different basic types of paths, absolute (fully qualified) and relative. +----------+---------------------------------------+-------------------------------------------+ | Type | Def | Example | +==========+=======================================+===========================================+ | Relative | Path in relation to current directory | foo.txt | +----------+---------------------------------------+-------------------------------------------+ | Absolute | Fully qualified unambiguous path | /afs/crc/user/u/username/research/foo.txt | +----------+---------------------------------------+-------------------------------------------+ To see where you are currently in the file system hierarchy, also known as your **Current Working Directory** or ``CWD``, can be found with using the ``pwd`` command:: [crcuser@crcfe0X.crc.nd.edu ~]$ pwd /afs/crc.nd.edu/user/c/crcuser/ ............................................... Viewing Files and Directories """"""""""""""""""""""""""""""""" To view which files and directories are in your Current Working Directory, simply use the ``ls``:: [crcuser@crcfe0X.crc.nd.edu ~]$ ls Private Public YESTRDAY You can also pass a relative or absolute path to the ``ls`` command to view the contents of a particular directory without being within said directory:: [crcuser@crcfe0X.crc.nd.edu ~]$ ls Public myprogram.py mystuff researchstuff foobar.txt You can change which directory you're currently within by using the ``cd`` command. You can also pass in a relative or absolute path into this command and it will behave as expected:: [crcuser@crcfe0X.crc.nd.edu ~]$ cd Public [crcuser@crcfe0X.crc.nd.edu ~/Public]$ _ [crcuser@crcfe0X.crc.nd.edu ~/Public]$ cd /afs/crc/user/c/crcuser/Private [crcuser@crcfe0X.crc.nd.edu ~/Private]$ pwd /afs/crc/user/c/crcuser/Private ............................................... Commands ============================= In order to do anything within a terminal you will need to run commands. If you've been following along you already ran a few commands. The default shell for new CRC accounts is ``bash``. Thus, you will be running bash commands for both UGE jobs and while logged into the front ends. If you have an older account (pre May 2018) your default login shell will most likely be ``tcsh``. If you'd prefer to have either bash or tcsh as your login shell, feel free to drop us a line at CRCSupport@nd.edu and we'd be happy to change it over for you. If you are unsure what your login shell is, type ``echo $0``. To see a listing of basic commands, grab the sheet here: :download:`Linux Coding Cheat Sheet ` ............................................... Flags, Arguments, Options """"""""""""""""""""""""""""""""" Every command will have the name along with modifier flags, arguments, and options. These can drastically alter the result of the command or be more subtle. For example, to view hidden files you need to pass a flag to the ``ls`` command: .. code-block:: shell [crcuser@crcfe0X.crc.nd.edu ~/research]$ ls -a . .. .foo.txt research.txt foobar/ There are many, many flags to most commands. Normally, the ``--help`` flag is a standard for viewing a short help message about other options, but not always. .. code-block:: shell [crcuser@crcfe0X.crc.nd.edu ~]$ ls --help Usage: ls [OPTION]... [FILE]... List information about the FILEs (the current directory by default). Sort entries alphabetically if none of -cftuvSUX nor --sort is specified. Mandatory arguments to long options are mandatory for short options too. -a, --all do not ignore entries starting with . -A, --almost-all do not list implied . and .. --author with -l, print the author of each file -b, --escape print C-style escapes for nongraphic characters --block-size=SIZE scale sizes by SIZE before printing them; e.g., [ message clipped ] An argument to a command is simply something passed in via the command line to be harvested and used in someway by the command being ran. For example, the ``ls`` command can receive an argument of a path to list the contents of. Below, we are passing the argument of ``~/research/python_scripts`` to the command, which will then take that path and display the listing of contents. A command may do many different things with an argument, as there is no standard for what an argument is meant to do. .. code-block:: shell [crcuser@crcfe0X.crc.nd.edu ~]$ ls ~/research/python_scripts foo.py helper.py foobar.txt Normally there is a manual page for a command, although this may not always be the case. A manual page will give you the technical details about running and utilizing a certain command in terms of options, bugs, flags, and more. The command to view a manual page is ``man COMMAND_HERE``. This will normally bring up a pager such as ``less`` to view the page, where you can scroll up and down using the arrow keys, search for strings and more, or scroll with "J + K" keys. To exit the man page, you can press the "Q" key.:: LS(1) User Commands LS(1) NAME ls - list directory contents SYNOPSIS ls [OPTION]... [FILE]... DESCRIPTION List information about the FILEs (the current directory by default). Sort entries alphabetically if none of -cftuvSUX nor --sort is specified. Mandatory arguments to long options are mandatory for short options too. -a, --all do not ignore entries starting with . -A, --almost-all do not list implied . and .. --author with -l, print the author of each file -b, --escape print C-style escapes for nongraphic characters --block-size=SIZE scale sizes by SIZE before printing them; e.g., '--block-size=M' prints sizes in units of 1,048,576 bytes; see SIZE format below ............................................... FS Manipulation """"""""""""""""""""""""""""""""""" Organizing files and directories can free up time normally spent searching for files. If you'd like to create a directory to hold data etc for a certain portion of research, you can do so with ``mkdir``. Let's say you'd like to rename a directory or file, you can use the ``mv`` command to move that file / directory to a different name.:: [crcuser@crcfe0X.crc.nd.edu ~]$ mkdir research [crcuser@crcfe0X.crc.nd.edu ~]$ ls research Public Private YESTRDAY [crcuser@crcfe0X.crc.nd.edu ~]$ mv research malaria_research [crcuser@crcfe0X.crc.nd.edu ~]$ ls malaria_research Public Private YESTRDAY Copying files is another important task which can easily be accomplished. First you'll need to consider if you are copying a single file or a directory. To copy a single file, simply use ``cp src dest`` where src is the original file you'd like to copy and dest is the destination (this can include a path either relative or absolute). If you're copying a directory, you'll need to pass the ``-r`` flag to indicate you'd like to recursively copy files from said directory.:: [crcuser@crcfe0X.crc.nd.edu ~]$ ls -F foo.txt run.py* research/ [crcuser@crcfe0X.crc.nd.edu ~]$ cp foo.txt bar.txt [crcuser@crcfe0X.crc.nd.edu ~]$ ls -F foo.txt bar.txt run.py* research/ [crcuser@crcfe0X.crc.nd.edu ~]$ cp -r research test_research [crcuser@crcfe0X.crc.nd.edu ~]$ ls -F foo.txt bar.txt run.py* research/ test_research/ ............................................... Opening Files """"""""""""""""""""""""""""""""""""" Opening files within the terminal is normally the most efficient way to quickly see the results of some runs or check on output etc. There are several ways to see the contents of a file, with different methods being ideal in different situations. To simply dump the contents of a file to the screen, you can use ``cat``. Be careful doing so, as even if the file is a binary (compiled executable) or raw data file it will still dump gibberish to your screen. If the file is long and you'd like to easily scroll back and forth, you can open the file with a pager like ``less`` or ``more``. Note that these will seemingly take over your terminal and you'll need to exit them to get back to the cmd prompt. Normally this is as easy as pressing the "Q" key.:: [crcuser@crcfe0X.crc.nd.edu ~]$ ls -F foo.txt run.py* research/ [crcuser@crcfe0X.crc.nd.edu ~]$ cat foo.txt this is text from the file foo.txt this is more txt. I am text. Editing Files """"""""""""""""""""""""""""""""""""" Editing files from a terminal can be done in many ways. You can simply edit the file and transfer the edited file to the CRC servers, or edit in place through the terminal. There are several terminal text editors with varying degree of learning curves. Below they are organized from easiest to learn to hardest, normally the more difficult to learn indicates a more powerful editor in terms of features and abilities. * `nano `_ * `emacs `_ * `vim `_ * `ed `_ -- Old editor, exists but we do not recommend. Typically with each above, you can create a file for example `vim newfilename.txt`, or you can edit an existing file with the same command. Learning the ins and outs of a particular editor can take some time, but is almost always worth the effort to save time transferring files when a small change is needed. Most editors have the ability to customize the operation, look, and feel through configuration scripts and can add different plugins for additional modifications. It is not unheard of for some programmers to use one of the above editors as their preferred text editor for everyday use.