Duke Biology and NESCent users (aka SDG group) are maintaining a suite of phylogenetics, population genetics, and genomics software. See DSCR Installed Applications for a software list, including paths to executables. Many of these programs are installed at /opt/apps/sdg/bin. To avoid typing the entire path to a program whenever it is executed, it is useful to edit the UNIX shell variable $PATH. This tutorial will show you how to edit your shell variables in bash and tcsh. Because you may login with one type of shell and submit qsub scripts with another type of shell, you may need to edit both.
What is $PATH?
Whenever you type a command in UNIX (including Linux on the DSCR cluster and MacOSX), the system searches certain directories for an executable matching that name. This search path is stored in the environmental variable $PATH. Multiple directories can be specified. To see what your current $PATH is, use the command "echo":
This means that whenever a command is typed (even echo!), the system first searches in is /usr/local/bin, then /bin, and so on. Where does it find the command echo? To find the exact path found for an executable (without executing it), a useful command is "which":
So the system is going through three directories in the $PATH before it gets to one that contains the executable "echo."
How to change $PATH within one session.
It is useful to change the $PATH variable if you have executable programs in several standard locations. This will allow you to invoke the program from any directory with a simple command, rather than copying the executable into your current directory or specifying the entire path. Changing the $PATH from the command line is easy, but differs based on which shell language you are using. Most DSCR users will be using either BASH or TCSH. To find out which shell you are using:
To set your $PATH variable in Bash:
and in tcsh:
(NOTE: The lowercase form 'path' is required for tcsh set commands.)
Both commands will alter the $PATH variable by adding the SDG group directory to the beginning of the $PATH, so the system will search this directory first. Confirm that you have indeed changed the $PATH by re-entering:
Setting $PATH automatically
Determine which Shell You use.
When a user logs in to DSCR, the default UNIX shell is loaded. For many users this is the tcsh, though others prefer bash.
The shell is also loaded in qsub scripts you submit to the cluster (on the first line with the "shebang" such as #!/bin/bash).
If the default login shell is different from the shell you use in qsub scripts, you will need to edit your $PATH in both shell environments. If you would prefer to make your life simpler by only ever using one shell, you can change your default login shell through this help ticket on the OIT website.
Creating the profile
Rather than typing in the above commands every time you login, both bash and tsch provide a way to automatically set your $PATH variable when you login. Both files are "hidden," meaning the file names begin with a period. To see if these files exist for you:
If the files .bashrc and .tschrc do not exist, you will have to create them. Do not do this if the files already exist!
Edit these files with your favorite text editor.
On MacOSX, it is perhaps easiest with TextWrangler. Open TextWrangler and under File select "Open from FTP/SFTP server". Check the "SFTP" box and enter dscr-login-05.oit.duke.edu as the server. Provide your NetID and password, and after the connection completes navigate to your home directory (if necessary). At the bottom of the file browser is a checkbox: Show items starting with "." Check this box and open your profiles.
Setting up SDG environment
The SDG group is maintaining several programs and environments for users: including Python and R. To make it all run smoothly, we have created standard "SDG Group" profiles. If you are a basic user, all you need to do is add one line to one file! More advanced users can also use the profile and adapt it as necessary.
Open these files with the text editor as above. For bash users, add the following line at the end of your .bashrc file:
and in .tcshrc:
IMPORTANT! There must be one blank line at the end of your file, or it will not execute correctly.
For basic users, that's it!
If you have other directories you wish to check before the SDG directories (such as $HOME/bin), simply execute the "source" line before your additional modifications.
Save your profile, logout from the cluster in the Terminal, and log back in.
Confirm your new path with echo $PATH. Now, from any directory you can invoke any programs in /opt/apps/sdg/bin, such as:
What's In That Profile?
The following section is not necessary for basic users, but for more advanced users who want to know what's loaded when the sdg "master profile" is sourced.
Programs and packages installed and maintained by the SDG Group are located at /opt/apps/sdg/bin. This variable is also useful for executing scripts uploaded by SDG Group users to $SDG_ROOT/Scripts.
The Python Path is where Python looks for modules upon import. Modules installed specifically for the SDG version of Python (2.7) are located at /opt/apps/sdg/lib/python2.7/site-packages. Additional modules are loaded from the Python 2.6 packages in /opt/apps/lib/python2.6/site-packages.
To utilize the SDG version of R, packages have been installed at /opt/apps/sdg/lib/r-packages.
Several newer programs, compiled with gcc version 4.3 or later, require advanced libraries located at /opt/apps/lib64