Running Jobs at Computational Facilities (pdf download)
This tutorial discusses, 1) how to submit jobs at JHU-MARCC with a sample job submission file, 2) how to run interactive jobs at JHU-MARCC, 3) useful commands related to job submission, 4) useful Linux commands, and 5) how to submit jobs at XSEDE with a sample job submission file.
1. How to Submit Jobs at JHU-MARCC
MARCC policy states that all users must submit jobs to the scheduler for processing. Interactive use of login nodes for job processing is not allowed.
- SLURM resource manager to manage resource scheduling and job submission.
- Partitions (different job queues) to divide types of jobs which will allow sequential/shared computing, parallel, GPU jobs, and large memory jobs.
For the complete list of partitions available for users please visit:
If you need further info or assistant, please contact MARCC support at,
Reach us via the PARADIM computation support forum,
Sample job submission scripts
A sample script to run a Quantum Espresso job in parallel partition using 24 cores with 5000MB memory in a single node would look like this,
Jobs are usually submitted via a script file. The sbatch command is used.
2. How to Run Interactive Jobs at JHU-MARCC
Users who need to interact with their codes while these are running can request an interactive session. This will submit a request to the queuing system that will allow interactive access to the node.
If you would like an interactive session, you can use the following command
Here we are requesting 24 CPUs, since -n 24 is the number of tasks, and -c 1 is the number of cores per task. We are asking for a session of 60 min (-t 60) and with a total memory of 5 GB (-m 5G).
This command opens a session where we will be able to execute pw.x directly from the command line without a job submission script.
3. Useful Commands for job Submission
$ sbatch my-script submit a job script
$ squeue list all jobs
$ sqme list all jobs belong to the current user
$ squeue -u [userid] list jobs by user
$ squeue [job-id] check job status
$ scancel [job-id] delete a job
$ scontrol hold hold a job
$ scontrol release release a held job
$ sacct show finished jobs
*Users can still use torque commands like qsub, qdel, qstat, etc…
4. Useful Linux Commands
- mkdir – make directories
Usage: mkdir [OPTION] DIRECTORY…
eg. mkdir paradim
- cd – change directories
Usage: cd [DIRECTORY]
eg. cd paradim
- ls – list directory contents
Usage: ls [OPTION]… [FILE]…
eg. ls, ls -l, ls paradim
- pwd – print name of current working directory
- vim – Vi Improved, a programmer’s text editor
Usage: vim [OPTION] [file]…
eg. vim myscript.txt
- cp – copy files and directories
Usage: cp [OPTION]… SOURCE DEST
eg. cp myscript.txt myscript_duplicate.txt, cp -r directory1 directory2
- mv – rename/move files
Usage: mv [OPTION]… SOURCE DESTINATION
eg. mv myscript.txt directory
eg. mv myoldscript.txt mynewscript.txt
- rm remove files or directories
Usage: rm [OPTION]… FILE…
eg. rm myoldscript.txt, rm -rf directory
- find – search for files in a directory hierarchy
Usage: find [OPTION] [path] [pattern]
eg. find myscript.txt, find -name myscript.txt
- history – prints recently used commands
- ps – report a snapshot of the current processes
Usage: ps [OPTION]
eg. ps, ps -el
- kill – to kill a process
Usage: kill [OPTION] pid
eg. kill -9 2275
5. How to Submit Jobs at XSEDE
Allocation for The Extreme Science and Engineering Discovery Environment (XSEDE) is available for collaborative research only. Please contact members of PARADIM theory staff available at /people to get access to XSEDE allocations.
If you already have an allocation with XSEDE, following is a sample job scrip to submit jobs.