New User Info
Obtaining a CRC Account
Quick Start Guide
Cluster Computing Format
Front End Systems
Proper Front End Usage
Logging in
Utilizing Front Ends
Available Software
File Systems
AFS
Panasas
File Transfers
Job Scripts
Parallel Environments
Queues
Host Groups
Job Submission and Monitoring
Why is my job waiting in the Queue?
Parallel Jobs
Job Deletion
Job Resource Monitoring
Job Arrays
Connecting to CRC Servers
Connecting from Linux
X11 Forwarding
Connecting from a Mac
X11 forwarding for Mac
Connecting from Windows
Connecting from off campus
Basic Linux Guide
Navigating File System
Paths
Viewing Files and Directories
Commands
Flags, Arguments, Options
FS Manipulation
Opening Files
Editing Files
Modules
Module Command
Searching Modules
Module Categories
Using Modules
Getting information about a module
User Created Modules
Creating a Private Module
What if my Software is a precompiled Binary?
Using Private Modules
Different Versions of Software
Considerations
Introductory Videos
Accessing the CRC systems
CRC front end systems
Connecting from a Windows
Connecting from Mac
Connecting from the browser
Utilizing front end systems
Modules
File systems
Transfering files
Job scripts
Parallel environments
Queues
Job submission
Job monitoring
Checking available resources
Job arrays
Training Events
New User Training
Register and Contacts
Help
FAQ
CRC Accounts
How Do I Request a CRC Account?
How Do I Find Some Quick Start Information?
I Can’t Login (New User / First Time Login)
I’ve Forgotten My Password
How Do I Request an Account for an External Collaborator?
How Can I Maintain My CRC Account After Graduating/Leaving Notre Dame?
I am Getting A “Module: Command Not Found” Message When I Log In To My CRC Account
Remote Access
Why Is My Remote Connection So Slow?
When I try to login I see the error: “Algorithm Negotiation Failed”, what does this mean?
When I try to connect CRC through VSCode, why does Remote SSH keep asking for password again and again?
Training
How Do I Receive Training?
CRC Software
What Software Is Available At The CRC?
How Do I Request Help Installing Software?
How Do I Request A Software Module To Be Installed/Updated?
Notes on sudo/root access
AFS Storage
How Do I Check My Available AFS Storage?
How Do I Request More AFS Storage Space?
How Do I Give A Particular User/Group Access To A Directory In My Personal AFS Space?
I Am Receiving A “Locking Authority File” Error
How Do I Add and Remove Users From The Shared Group Space I Administer?
I’m Not Seeing My Output When Running From AFS!
I Am Getting The Following Message Using Find
PANASAS Storage (/scratch365)
How Do I Check My Available (/scratch365) Storage?
How Do I Request More (/scratch365) Storage Space?
Jobs
My Job Is Aborted At Runtime Due To Excessive Disk-Swapping
Do Different Operating System Versions Affect Performance?
Unable to run job: error: no suitable queues’ Message When Submitting Job Script
How Can I Monitor The Behavior Of My Running Jobs?
My Job Has Created Zombies…What Does That Mean?
Applications
Xquartz gives me a failed request error:
I Need To Install ArcGIS On My System
I am trying to compile my code with mvapich2 but cannot find -lpsm_infinipath
I See Incomplete Images In COMSOL v4+
My Gaussian Jobs Are Being Aborted Due To /tmp Over-usage
Request of ArcGIS Installation and Support
Grants/Proposals
How Do I Acknowledge the CRC In My Proposals?
Does The CRC Provide An Example Of A Data Management Plan?
Getting Help
Steps to Getting Help
Setting up an Appointment
User Support Location
Training Opportunities
CRC Infrastructure
CRC Resources
Current Equipment
Systems Dashboard
Storage and server lookup
CRC Software
User Requested Software
Available Hardware
Interactive Front End(s)
CRC Owned Compute Clusters
Large Memory Systems
GPU Systems
Storage
AFS Storage
AFS Limitations and Notes
AFS Group Storage
AFS Website
AFS Permissions / ACLs
Panasas (scratch) Storage
Panasas Best Practices
Local /tmp Space
Storage Backups
Storage Comparison
UGE Environment
Queues
Parallel Environments
Host Groups
User Groups
Faculty Cluster Partnership Program
Purchasing Dedicated Servers
Bulk Purchasing
Grant Proposals
Maintenance
Typical Changes
Module Changes
Reoccuring Maintenance
Past Maintenance Periods
Resources
GPU
Job Submission Example
Installing Software on a GPU machine
CUDA And cuDNN Modules’ Avalability
Available Hardware For General Access
Resource and Job Monitoring
Other Resources For GPU Jobs
HTCondor
Visual Status of the ND Condor Pool
Online Short Course
About Condor
Getting Started With Condor at Notre Dame
Important note about AFS
Submitting Large Numbers of Jobs
To Submit Multicore SMP jobs to the ND Condor Pool
Example of Submitting 10 Matlab jobs
Example of Submitting an Amber job to a GPU node
Condor Status
More Information
Globus
What Is Globus?
Setting Up Globus at the CRC
Transferring Files to/from CRC
Prepare Endpoints/Collections
Select and Transfer File(s)
Sharing Files
Quick Guide
NDCMS
Logging In and Setting up your Software Environment
Setting up environment
Add your proxy DN and Notre Dame username (your username on earth.crc.nd.edu)
If You Forgot to Set Your AFS Permissions at First Login
PhEDEx
Requesting a PhEDEx Transfer to the Notre Dame Tier 3 Site (US_T3_NotreDame)
Monitoring Tier 3 components and Jobs
HTCondor Monitoring
HTCondor Job Monitoring
Storage
Home Area
Scratch Space
Accessing the scratch space from grid jobs
Hadoop Space
Lobster Working Storage
Storage space for PhEDEx
Apptainer/Singularity
Why Use Apptainer
Downloading Apptainer
Getting Started
Obtaining Containers
Create Your Own Custom Container
Using Pre-Built Containers
Accessing Containers
Using Apptainer in Batch Jobs
Apptainer on GPU Nodes
Using Apptainer within AFS
Using Modules within Apptainer containers
Important Notes
Helpful Resources
CAML
Access requirements
Resource description
Using the resource on campus
Jupyterhub service
Pre-requisites
Instructions
HTCondor Submission
HTCondor submission file
Setting up the tutorial examples
Tensorflow example
Pytorch example
Amber example
JAX example:
DeepSphere example
Popular Modules
Matlab
Toolboxes
Submitting Single Core Jobs
Submitting Multicore Jobs
MATLAB and Array Jobs
Memory Profiling
License Information
Tutorials and Training
Known issue with
parpool
and
matlabpool
Dynare Support
Further Information
Python
Basic Usage
Installing Python Packages Locally
Adding ${HOME}/.local/bin to your path
Python Virtual Environment
Job Submission Example
Multi-core Jobs
Further Information
R
Sample R Job
Installing Local Packages
Installing packages within R
Installing R packages from source code
Profiling R Code
Parallel Computing in R
Test for Rmpi
Related Software
Further Information
Tensorflow
General Description
Basic Usage
Tensorflow Python
Installing Python Packages Locally
Conda
General Description
Initial setup
Environment Management
Package Management
Python Management
Channel Management
Job Submission Using Conda Environments
CRC User
»
Index
Index